loading - one big File vs. many small files

[Cero]

Which of these is generally faster?

Loading one big, in some cases huge, file; or many small files ?

The idea is, only loading content when you need it, and in doing so to conserve memory space.
But also the game may not slow down because of file loading during the game.

This question can be applied to any kind of media, but I'm talking mostly about sprites and text files.

[Orangy Tang]

Which of these is generally faster?

Loading one big, in some cases huge, file; or many small files ?

Generally one file will be faster, since there's a non-trivial overhead in opening a new file handle, and you'll usually have to wait for a disk seek too. Of course you can always pack your small files into one larger file, the easiest and most obvious way would be to pack your resources into a jar (which you might have to do anyway).

The idea is, only loading content when you need it, and in doing so to conserve memory space.
But also the game may not slow down because of file loading during the game.

Conserving memory space is a nice idea, but do you really have to? How many sprites are we talking about here? What size are they when loaded into ram? You may be making extra work for yourself when you don't really have to.

Also, if you're worried about loading slowing down your game then load your resources in a background thread.

[Cero]

Generally one file will be faster, since there's a non-trivial overhead in opening a new file handle, and you'll usually have to wait for a disk seek too. Of course you can always pack your small files into one larger file, the easiest and most obvious way would be to pack your resources into a jar (which you might have to do anyway).

Yeah well, when I distribute, I want to offer some kind exe wrapper for windows users. So I will see if that works with a jar of the whole project.

Conserving memory space is a nice idea, but do you really have to? How many sprites are we talking about here? What size are they when loaded into ram? You may be making extra work for yourself when you don't really have to.

We are talking about a game the size of Diablo 2, all 2D sprites... so there are lots of them.
And naturally you want to conserve as much memory as possible, so that also low spec PCs can play it, with no problem. As Diablo 2 does of course.
Right now, I load every sprite I need on map change.
Even so there are many sprites involved, it seems to be very fast.
So, as of now my main concern is not the sprites, but actually text files.

It's about Items and their data, which, in my case, mainly involve:
Name(String), Description Text(String), Artwork(BufferedImage), Icon / Symbol (BufferedImage)

Now what I'm doing is:   Whenever the game needs any data about an Item, it will load (ingame, not map change) the data.
Which involves reading in, one XML file per Item for all the text and at the moment 2 Images to be loaded from file.

Obviously once loaded it won't be loaded again.

On that note I am also not sure if I should unload all unnecessary resources on map change aswell.

Also, It's like I have any performance issues as of now at all; but I just want to make very sure it stays that way as the game grows.

Games published by our own members! Check 'em out!

[Scarzzurs]

It really sounds like doing some basic large-scale tests would be in your best interest. Just to see if you really have a problem to be solved or not.

However, I suspect the text files will be the smallest problem to pre-cache (their size is probably very small compared to the graphics / audio of the game).
Thus I would load/unload all resource-heavy files at map change (and maintain almost all low-end users). All other resources could be loaded and cached throughout the lifetime of the program...

It is also worth noting that certain resources like graphics and audio data often can be moved outside the Java heap and stored elsewhere (on graphics card / ram), in order to cause less work for the garbage collector...

[Edit]Oops, seems like this topic was rather old, although not containing any real conclussion. Sorry if i gravedug something long dead. Remove my post if need be, couldn't find the button myself[/Edit]

- Scarzzurs

[Cero]

It really sounds like doing some basic large-scale tests would be in your best interest. Just to see if you really have a problem to be solved or not.

However, I suspect the text files will be the smallest problem to pre-cache (their size is probably very small compared to the graphics / audio of the game).
Thus I would load/unload all resource-heavy files at map change (and maintain almost all low-end users). All other resources could be loaded and cached throughout the lifetime of the program...

It is also worth noting that certain resources like graphics and audio data often can be moved outside the Java heap and stored elsewhere (on graphics card / ram), in order to cause less work for the garbage collector...

[Edit]Oops, seems like this topic was rather old, although not containing any real conclussion. Sorry if i gravedug something long dead. Remove my post if need be, couldn't find the button myself[/Edit]

- Scarzzurs

Hey,

well it turns out loading many files works better for me.

As for the "load/unload all resource-heavy files at map change"... well, I do load all ressources missing at map change.
So every time there is a map change, certain assets are needed, checked if they are loaded, if not, they are loaded.
I don't really unload anything though. May wanna look into that...  I just don't want the map change to be more than 2 seconds or so. At the moment its fine.