Java-Gaming.org Hi !
Featured games (90)
games approved by the League of Dukes
Games in Showcase (744)
Games in Android Showcase (225)
games submitted by our members
Games in WIP (825)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  Efficient Chunk Loading.  (Read 4168 times)
0 Members and 1 Guest are viewing this topic.
Offline zFollette

Junior Devvie


Exp: 2 years


I like jokes


« Posted 2014-01-18 03:30:40 »

Lets say I want to load a Map that is 20x20 chunks, each chunk is 10x10x2 blocks, that is a lot of blocks (80,000). So, in a small map, I would just initiate all of the chunks before the game starts, but now, I am interested in more efficient ways. I have tried to only initiate the chunks within a 2x2 area of the player, but RAM keeps stacking (from ~52mb to ~250mb) by the time I trek the whole map. Here is some code I have tried to implement. If anyone can give me a better suggestion, please do so.

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  
42  
43  
44  
45  
46  
47  
48  
49  
50  
51  
52  
53  
54  
55  
56  
57  
58  
private void setView() {
        int lowX = (getChunk((int) position.x, (int) position.z)[0]) - 2;
        int highX = (getChunk((int) position.x, (int) position.z)[0]) + 2;

        int lowY = (getChunk((int) position.x, (int) position.z)[1]) - 2;
        int highY = (getChunk((int) position.x, (int) position.z)[1]) + 2;

        if (lowX < 0) {
            lowX = 0;
        }

        if (highX > width) {
            highX = width;
        }

        if (lowY < 0) {
            lowY = 0;
        }

        if (highY > height) {
            highY = height;
        }

        if (view[0] != lowX || view[1] != highX || view[2] != lowY || view[3] != highY) {
            view[0] = lowX;
            view[1] = highX;
            view[2] = lowY;
            view[3] = highY;
            loadChunks();
        }
    }

    private void loadChunks() {
        for (int x = 0; x < width; x++) {
            for (int y = 0; y < height; y++) {
                if (x >= view[0] && x <= view[1] && y >= view[2] && y <= view[3]) {
                    chunk[x][y] = new Chunk(x, y);
                    chunk[x][y].setTexAmount(4);
                } else {
                    chunk[x][y] = null;
                }
            }
        }
    }

private void renderChunks() {
        int lowX = view[0];
        int highX = view[1];

        int lowY = view[2];
        int highY = view[3];

        for (int x = lowX; x < highX; x++) {
            for (int y = lowY; y < highY; y++) {
                chunk[x][y].renderChunk();
            }
        }
    }


If it matters, I am using Vertex Buffer Objects for all of my 3D rendering.

Also, a sub question: Would it be worth the code for me to convert my 2D HUD from immediate mode to VBO?

Humor will keep you alive.
Offline Opiop
« Reply #1 - Posted 2014-01-18 03:40:19 »

First off, why the random choice in chunk size? I believe it is actually less efficient not to use a power of two chunk size, but I don't know where I read that.

In my YouTube tutorial series, I can load literally millions of blocks at one time. How do I do it? Well, for one I use display lists. Second, I do a far amount of optimizations.

1.) Frustum culling. VERY IMPORTANT! The game would run very very slow without it.
2.) I create one static instance of every block type in my world instead of creating separate instances for every block. I have a feeling this is what you're doing, and its a very, very bad way of doing it!
3.) All your standard culling (back face culling, face removal, block removal)
4.) Unload chunks not in view, and load them back in when in view.

I'm sure I missed a couple, but with these pretty basic optimizations I can hold around 1000 chunks in memory at once. Not great, but Its decent I think. So far I've yet to actually hit a limit on how many chunks can be rendered at once.  display lists are amazing for this kind of work though, I would quite honestly recommend arrays objects instead of buffer objects if you don't like the idea of display lists and want to stay "modern".

Remember, just because buffer objects are the modern way of holding vertices it doesn't mean it's the best way. I found that buffer objects could barely render 16 chunks before slowing down, whereas I can render literally a 1000 display lists at once and see no performance hit. Sure, I probably didn't optimize the buffer objects well enough, but it seems display lists are doing the job, so what the hell! Might as well do something I enjoy.
Offline zFollette

Junior Devvie


Exp: 2 years


I like jokes


« Reply #2 - Posted 2014-01-18 03:45:57 »

First off, why the random choice in chunk size? I believe it is actually less efficient not to use a power of two chunk size, but I don't know where I read that.

In my YouTube tutorial series, I can load literally millions of blocks at one time. How do I do it? Well, for one I use display lists. Second, I do a far amount of optimizations.

1.) Frustum culling. VERY IMPORTANT! The game would run very very slow without it.
2.) I create one static instance of every block type in my world instead of creating separate instances for every block. I have a feeling this is what you're doing, and its a very, very bad way of doing it!
3.) All your standard culling (back face culling, face removal, block removal)
4.) Unload chunks not in view, and load them back in when in view.

I'm sure I missed a couple, but with these pretty basic optimizations I can hold around 1000 chunks in memory at once. Not great, but Its decent I think. So far I've yet to actually hit a limit on how many chunks can be rendered at once.  display lists are amazing for this kind of work though, I would quite honestly recommend arrays objects instead of buffer objects if you don't like the idea of display lists and want to stay "modern".

Remember, just because buffer objects are the modern way of holding vertices it doesn't mean it's the best way. I found that buffer objects could barely render 16 chunks before slowing down, whereas I can render literally a 1000 display lists at once and see no performance hit. Sure, I probably didn't optimize the buffer objects well enough, but it seems display lists are doing the job, so what the hell! Might as well do something I enjoy.

Thanks for the response. And yes, I am making a new instance of every block, unfortunately. By the way, you never got back to me on that private message. I will definitely try out some of your optimizations though.

EDIT: I misread you, I do not make a new instance for every block. Only every chunk. The only data I keep per block is in an array of Vector4f, x, y, z and texture.

Humor will keep you alive.
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Opiop
« Reply #3 - Posted 2014-01-18 03:53:02 »

Oh sorry! I've been very busy lately!

You shouldn't keep an instance of every texture. Your renderer should take parameters corresponding to what texture you want to render. I also assume you are using a spritesheet? If not, definitely do that right away! But yes, storing a new texture per chunk would add up eventually.

Really, if you want to be very efficient, you shouldn't be creating any new objects at all after the initial start up of the game. For instance, I make a single block for each block type, I make all the chunks, I make the entities and mobs and player and then I don't create new objects at all after that. Later on I will need to, but I really do believe that's one reason its running faster.
Offline zFollette

Junior Devvie


Exp: 2 years


I like jokes


« Reply #4 - Posted 2014-01-18 03:56:09 »

Oh sorry! I've been very busy lately!

You shouldn't keep an instance of every texture. Your renderer should take parameters corresponding to what texture you want to render. I also assume you are using a spritesheet? If not, definitely do that right away! But yes, storing a new texture per chunk would add up eventually.

Really, if you want to be very efficient, you shouldn't be creating any new objects at all after the initial start up of the game. For instance, I make a single block for each block type, I make all the chunks, I make the entities and mobs and player and then I don't create new objects at all after that. Later on I will need to, but I really do believe that's one reason its running faster.

I forgot to mention that I handle textures by number, not by instance.  Yes I use a spritesheet. I will make a version of my 3D rendering in display lists. I have never used them for large scale 3D before.

Humor will keep you alive.
Offline Opiop
« Reply #5 - Posted 2014-01-18 03:58:56 »

For voxels it seems they work very well. What do you mean by number though?
Offline Longarmx
« Reply #6 - Posted 2014-01-18 03:59:58 »

I would assume texture handle.

Offline Opiop
« Reply #7 - Posted 2014-01-18 04:00:58 »

Ah I see, I understand now. Why keep a texture handle per chunk though? I create a single static spritesheet in a spritesheet class then reference it whenever I need it. No point in having every chunk keep their own.
Offline zFollette

Junior Devvie


Exp: 2 years


I like jokes


« Reply #8 - Posted 2014-01-18 04:08:48 »

Ah I see, I understand now. Why keep a texture handle per chunk though? I create a single static spritesheet in a spritesheet class then reference it whenever I need it. No point in having every chunk keep their own.

I am sorry for my miss wording. I load the sheet and set each texture within the sheet to a number, lets say I have 5 textures, the numbers would be 0-5. I then  I can change the texture by putting a different number in for the coords of the cube in an array, I then re buffer the chunk, applying the changes. I never load the sheet more than once.

Humor will keep you alive.
Offline Opiop
« Reply #9 - Posted 2014-01-18 04:10:41 »

Ah I see, that makes sense. Can you profile your code? That would be greatly helpful to see why you have such a high memory usage.
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline zFollette

Junior Devvie


Exp: 2 years


I like jokes


« Reply #10 - Posted 2014-01-18 04:18:50 »

Ah I see, that makes sense. Can you profile your code? That would be greatly helpful to see why you have such a high memory usage.

What do you mean by profile? And wow man, thanks for the tip on display lists. Literally exponentially more efficient. But the RAM is still stacking, nevertheless.

Humor will keep you alive.
Offline Opiop
« Reply #11 - Posted 2014-01-18 04:22:09 »

Profiling basically means to monitor the JVM while it is running. You can see exactly (sort of) what is happening. How much memory is used, how much this thread is being used etc... I don't know of any good profilers, but a quick Google search should turn you up one!

No problem for the display list idea by the way! Just remember to only rebuild them once every time a chunk is changed and not ever else!
Offline zFollette

Junior Devvie


Exp: 2 years


I like jokes


« Reply #12 - Posted 2014-01-18 04:25:07 »

Profiling basically means to monitor the JVM while it is running. You can see exactly (sort of) what is happening. How much memory is used, how much this thread is being used etc... I don't know of any good profilers, but a quick Google search should turn you up one!

No problem for the display list idea by the way! Just remember to only rebuild them once every time a chunk is changed and not ever else!

Only common sense my friend. Haha, I enabled the basic GL_CULL_FACE, and that seems to save some RAM, but nothing extreme.

EDIT: I fixed my chunk loading method to initialize all of the chunks at the start, it seems like the display lists load almost instantanious compared to VBOs which take around 4 seconds to load 400 chunks. I am sitting at 80mb RAM with 80,000 tiles, the RAM is no longer stacking, at least I think so.

Humor will keep you alive.
Offline Longarmx
« Reply #13 - Posted 2014-01-18 04:27:30 »

To solve your ram problem, just unload the chunks once they are far enough away from the player.

Offline zFollette

Junior Devvie


Exp: 2 years


I like jokes


« Reply #14 - Posted 2014-01-18 04:30:52 »

To solve your ram problem, just unload the chunks once they are far enough away from the player.

Describe 'unload'. As in setting the chunk to null, so the GC can take care of it? If so, I did that, it sucked. If you mean 'don't render them when they get a distance away', I do that.

Humor will keep you alive.
Offline Opiop
« Reply #15 - Posted 2014-01-18 04:32:09 »

No no, just save them to disk when they are far away. Then remove the chunks from your array. You're using an array correct? I actually use an arraylist so I can literally remove the chunk from memory after I save it, then add it back when I load it back in.
Offline zFollette

Junior Devvie


Exp: 2 years


I like jokes


« Reply #16 - Posted 2014-01-18 04:34:55 »

No no, just save them to disk when they are far away. Then remove the chunks from your array. You're using an array correct? I actually use an arraylist so I can literally remove the chunk from memory after I save it, then add it back when I load it back in.

By save to disk, do you literally mean save it to my hard disk? I was planning on map loading, but not at this point, though it can be done in like 5 minutes.

Humor will keep you alive.
Offline Opiop
« Reply #17 - Posted 2014-01-18 04:35:58 »

Yes, save it to the hard drive. Trust me, it'll get rid of a lot of your issues, especially with memory management.
Offline zFollette

Junior Devvie


Exp: 2 years


I like jokes


« Reply #18 - Posted 2014-01-18 04:39:41 »

Yes, save it to the hard drive. Trust me, it'll get rid of a lot of your issues, especially with memory management.

Ok, I will do that tomorrow, and do you think storing an array of integer arrays is more efficient that storing an array of Vertex4fs?

Humor will keep you alive.
Offline Opiop
« Reply #19 - Posted 2014-01-18 04:42:23 »

Are you storing the voxel positions? Don't do that. It probably actually is the reason you have such high memory usage! You can just extract the voxel positions from the array they sit in. I keep a byte of tile ids in each of my chunks, and my renderer then renders the appropriate texture from the id. I never ever store a voxel position. Its just a huge waste of memory!
Offline zFollette

Junior Devvie


Exp: 2 years


I like jokes


« Reply #20 - Posted 2014-01-18 04:48:30 »

Are you storing the voxel positions? Don't do that. It probably actually is the reason you have such high memory usage! You can just extract the voxel positions from the array they sit in. I keep a byte of tile ids in each of my chunks, and my renderer then renders the appropriate texture from the id. I never ever store a voxel position. Its just a huge waste of memory!

Interesting, I will definitely try that out and see how it goes. I can load 1000 chunks at 124mb of RAM, if it is any comparison to you.

Humor will keep you alive.
Pages: [1]
  ignore  |  Print  
 
 

 
Ecumene (145 views)
2017-09-30 02:57:34

theagentd (213 views)
2017-09-26 18:23:31

cybrmynd (292 views)
2017-08-02 12:28:51

cybrmynd (284 views)
2017-08-02 12:19:43

cybrmynd (294 views)
2017-08-02 12:18:09

Sralse (287 views)
2017-07-25 17:13:48

Archive (966 views)
2017-04-27 17:45:51

buddyBro (1092 views)
2017-04-05 03:38:00

CopyableCougar4 (1663 views)
2017-03-24 15:39:42

theagentd (1425 views)
2017-03-24 15:32:08
Java Gaming Resources
by philfrei
2017-12-05 19:38:37

Java Gaming Resources
by philfrei
2017-12-05 19:37:39

Java Gaming Resources
by philfrei
2017-12-05 19:36:10

Java Gaming Resources
by philfrei
2017-12-05 19:33:10

List of Learning Resources
by elect
2017-03-13 14:05:44

List of Learning Resources
by elect
2017-03-13 14:04:45

SF/X Libraries
by philfrei
2017-03-02 08:45:19

SF/X Libraries
by philfrei
2017-03-02 08:44:05
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!