Java-Gaming.org Hi !
Featured games (90)
games approved by the League of Dukes
Games in Showcase (769)
Games in Android Showcase (230)
games submitted by our members
Games in WIP (856)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  How to manage very large 3D arrays? (OutOfMemoryError)  (Read 2211 times)
0 Members and 1 Guest are viewing this topic.
Offline Llorn

Junior Newbie





« Posted 2014-05-07 17:45:41 »

Hi,

I just started developing my first game, a learning project of sorts. It'd describe the envisioned finished version as something between Minecraft, Dwarf Fortress, and Battle of Wesnoth; a blocky, tick-based simulation of a randomly generated fantasy world with thousands of unique, detailed characters in it, in which you can do pretty much anything. I got to a problem right at the beginning. I was trying to generate a basic random world, so that I have something to work with when setting up the display, but when I tried to represent the game world as a 24,576 x 24,576 x 150 3D array of Element objects(a class I made to eventually represent everything in the game), I kept getting the "Exception in thread "Display" java.lang.OutOfMemoryError: Java heap space".

I got those numbers from Dwarf Fortress, and that's not even a large world. And Minecraft worlds can be up to 6,000,000x6,000,000x256, so it should be possible, but how? I tried to google for how to save it in a file, but all the advice there was a bit off from what I wanted and I couldn't divine anything useful from it. So, how would you go aroung managing a game world like this?
Offline Bearded Cow

Senior Devvie


Medals: 3
Projects: 1
Exp: 1 year


¬..¬


« Reply #1 - Posted 2014-05-07 18:05:22 »

Split it up. I'm sure there is a lot more details on this if you look up 'chunking'.
Offline princec

« JGO Spiffy Duke »


Medals: 1059
Projects: 3
Exp: 20 years


Eh? Who? What? ... Me?


« Reply #2 - Posted 2014-05-07 18:46:03 »

One way to do chunking is to store smaller cubes in a Map, using the cube's offset coordinates as the key. Wrap it up in a facade that makes it look like one giant cube, but which handles "missing" cubes by constructing/generating/loading them on the fly.

Cas Smiley

Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline StrideColossus
« Reply #3 - Posted 2014-05-07 20:05:55 »

As previous posters have already suggested, divide your 'world' into chunks and create some sort of service or facade that loads chunks as required on the fly.

Some other things you might want to consider adding to this idea:

- caching: as the user moves around you will loading new chunks into memory (on a background thread) and discarding stale chunks, this is essentially a cache.  If your cache is slightly larger than the extent of your world you can retain chunks even though they are not currently in view since they are likely to be needed again due to locality (i.e. user will move back).  When you do need to discard from the cache (you will need to apply some sort of limit(s)) you could use an LRU algorithm, or distance to user, or cost-to-load, some combination of these, etc.

- pre-loading: similar to the above, you could pre-load chunks that the user is likely to see soon, e.g. you display (say) all chunks within 3 units-of-distance from the user, but you load all chunks to a radius of 5, or load the ones in the direction the user is facing, or some such algorithm.

- LOD (level of detail): not sure if this is relevant to your game, but if you are (for example) rendering a 3D environment the more distant chunks do not need to be at the maximum LOD.  Therefore you can load less detailed data / textures initially and progressively load more detailed data as the user gets closer.

Just some thoughts to consider.

- stride
Offline Roquen

JGO Kernel


Medals: 518



« Reply #4 - Posted 2014-05-07 20:06:03 »

My advice is to makes some some small games first.
Offline princec

« JGO Spiffy Duke »


Medals: 1059
Projects: 3
Exp: 20 years


Eh? Who? What? ... Me?


« Reply #5 - Posted 2014-05-07 20:40:40 »

Oh yes, and that Smiley

For yours and everyone else's sanity, please, please make Space Invaders first.

Cas Smiley

Offline Mac70
« Reply #6 - Posted 2014-05-07 20:43:20 »

If your entities don't have any special data, you can use byte arrays inside class with proper getters/setters instead of object arrays - this solution is using from 4 to 8 times less memory. If you need to add special data to some tiles, then you can just add them to proper list/map.
Offline Roquen

JGO Kernel


Medals: 518



« Reply #7 - Posted 2014-05-07 20:54:04 »

Procedural content.  Read this: http://www.java-gaming.org/topics/uniform-feature-points/28000/view.html.  Schemes of this nature are the basic building blocks.

You can store an (effectively) infinite amount of content in zero persistent memory.  The only maintained memory you need is for whatever schemes you want to support PC changes to the environment.
Offline Llorn

Junior Newbie





« Reply #8 - Posted 2014-05-09 15:12:50 »

Alright, first off, thanks for all the advice, much appreciated. Here are the things I checked out and how they turned out:

Chunking

The only way to divide the world into chunks that I could come up with was to create a 5D array, with the first two coordinates being coord of a 2D chunk grid comprising of 32x32(chosen arbitrarily) chunks, and the rest a normal 3D coords withing the chunk. When I tried to randomly generate a world like that(nothing too complicated, just a random number from 0 to 9 to represetnt ten different kinds of blocks), I got the same error. I think I'm missing something important here. I get the error because the array is too large, right? So how exactly do I go around chunking it into smaller pieces so that it doesn't crash when I try to generate a world? I could save each chunk in a separate variable intead of an array, but then I couldn't make worlds of arbitrary sizes, or could I? Here is the code for the chunked world, for reference:

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
      for (int chx = 0; chx < currentWorld.worldArray.length; chx++) {
         for (int chy = 0; chy < currentWorld.worldArray[0].length; chy++) {
            for (int ix = 0; ix < currentWorld.worldArray[0][0].length; ix++) {
               for (int iy = 0; iy < currentWorld.worldArray[0][0][0].length; iy++) {
                  for (int iz = 0; iz < currentWorld.worldArray[0][0][0][0].length; iz++) {
                     currentWorld.worldArray[chx][chy][ix][iy][iz] = new Element(random.nextInt(10));
                     System.out.println(currentWorld.worldArray[chx][chy][ix][iy][iz].ID);
                  }
               }
            }
         }
      }


Byte arrays
Tried to google and read up on them a bit, but I'm still not sure what do you mean or how to implement it. Are you saying that I should store the data about what's in each block in a byte instead of in other data types? So far, I've been planning on having either blocks or mobs filling the game world; Blocks will just have an integer ID referring to a definition of that block's properties(texture, hardness, drop, an so on), and mobs will have tons of properties like species, inventory, health, etc., etc. They will also be subject to a regular AI check, as soon as I set up the rest of the basics. So how would you implement this using byte arrays instead of a more straighforward solution like ints and Strings?

Simpler game
I do plan on making it simple... for now. It should look more or less like Dwarf Fortress with a graphic pack:



I don't want any of that 3D or real-time stuff. Just 2D graphics, 3D block-based space, and tick-based time. I don't plan on making it super-complex righ now, either. A hero slaughtering sheep on a giant, randomly generated cube world composed solely of "Rock" and "Dirt" blocks would be enough to make me happy for now Smiley I'd like to add a ton of things later(that's what I like about this porject the most, I can keep adding stuff indefinitely), but for now, I just want to have the basics set up.
Pages: [1]
  ignore  |  Print  
 
 

 
EgonOlsen (1674 views)
2018-06-10 19:43:48

EgonOlsen (1722 views)
2018-06-10 19:43:44

EgonOlsen (1161 views)
2018-06-10 19:43:20

DesertCoockie (1587 views)
2018-05-13 18:23:11

nelsongames (1187 views)
2018-04-24 18:15:36

nelsongames (1716 views)
2018-04-24 18:14:32

ivj94 (2541 views)
2018-03-24 14:47:39

ivj94 (1767 views)
2018-03-24 14:46:31

ivj94 (2847 views)
2018-03-24 14:43:53

Solater (972 views)
2018-03-17 05:04:08
Deployment and Packaging
by mudlee
2018-08-22 18:09:50

Java Gaming Resources
by gouessej
2018-08-22 08:19:41

Deployment and Packaging
by gouessej
2018-08-22 08:04:08

Deployment and Packaging
by gouessej
2018-08-22 08:03:45

Deployment and Packaging
by philfrei
2018-08-20 02:33:38

Deployment and Packaging
by philfrei
2018-08-20 02:29:55

Deployment and Packaging
by philfrei
2018-08-19 23:56:20

Deployment and Packaging
by philfrei
2018-08-19 23:54:46
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!