Direct ByteBuffer costs on memory

[elias]

This is about ByteBuffers, but I'm posting it there because it's most important when working with lwjgl that makes extensive use of direct buffers.

This little test program shows the problem of direct buffer memory overhead.

1   2   3   4   5   6   7   8   9   10   11   12   13   14   15   16   17   18   19   20   21

import java.nio.*; public class TestMem {     public static void main(String[] args) {         ByteBuffer[] bufs = new ByteBuffer[100000];         for (int i = 0; i < bufs.length; i++)             bufs[i] = ByteBuffer.allocateDirect(1);         try {             Thread.sleep(60000);         } catch (Exception e) {             e.printStackTrace();         }         bufs = null;         System.gc();         try {             Thread.sleep(10000);         } catch (Exception e) {             e.printStackTrace();         }     } }

Running this program and watching memory use in e.g. 'top' on linux shows that the program sits on about 400mb of memory! This is against the theoretical 100k that I'm actually allocating. Changing this to indirect buffers solves the problem (still ~50-100 bytes overhead on each ByteBuffer though), but lwjgl don't like indirect buffers.

Only possible solution as I see it is to allocate one big chunk and slice()'ing it up as needed, in effect doing your own memory management.

I already submitted a RFE to sun, but I doubt they will improve this, as they already state that direct buffers should only be used for large, long-lived buffers.

Just FYI.

- elias

[vrm]

idea for v1 :
int buff=org.lwjgl.Sys.malloc(100);
org.lwjgl.Sys.free(buff);

[Herkules]

I have a déjà vu....

Games published by our own members! Check 'em out!

[princec]

We could very easily add a malloc.... but there's no real reason for a direct ByteBuffer to be incurring this overhead because it too should just boil down to a malloc in the end. It's best that someone with broadband grabs the source code and has a look at the native code behind it.

<Later> Ah no, don't worry about it. It's most likely because they're allocated on 4k page boundaries - and that's a Good Thing. So just don't go allocating too many things under 4k, ok?

I suspect they will not fix this, and I think they'd be right not to fix it.

Cas

[Jacko]

While you're talking about direct byte buffers....

I got hold of a copy of the next GCJ release 3.3 which has had nio put in. So I was pretty gutted when allocating a direct byte buffer gave the message "not implemented". (Have a look on the java message boards at gcc.gnu.org if you want to get a copy, 30 Mb download, set up already for windows, so no tricky cross compiling needed)

If there was a malloc in lwjgl, I assume this remove the need for using direct byte buffers?

Would we also need a free() or could the wrapper class look after this and clean up after us?

[elias]

a malloc() would require us to reimplement buffers altogether, along with alignment issues, endianess etc. Or else java couldn't do anything useful with the memory allocated.

Cas, please enlighten me, what's the good thing about 4k pages, apart from some sense of protection? I could imagine that keeping seperate buffers within 4k pages could eliminate the nasty side effects with buffer overruns almost completely. But is there any performance gain as such?

btw, my rfe will be listed at http://developer.java.sun.com/developer/bugParade/bugs/4820023.html
in a few days.


- elias

[princec]

We could easily write a complete replacement for java.nio.Buffer and its derivatives except of course we'd not have bounds checking enabled (Except in debug mode)

As far as LWJGL is concerned all we want is a legal address and area of memory. We could do it, and remove the dependency on 1.4 in the process... (I'd have to alter the method signature of Sys.getDirectBufferAddress() to merely take an Object and do a cast on the native side, which would avoid having 2 versions of LWJGL hanging about).

Anyone up for it?

Elias: 4k pages are generally more efficient on Intel hardware which uses 4k pages in its MMU. Not a great deal more efficient it has to be said. But it does highlight the fact that your buffers should generally be large and long-lived.

Cas

[elias]

How would we read/write the buffers from lwjgl, then? I could imagine it's a lot of hassle for so little - I implemented a simple memory manager yesterday that simply allocated one large chunk and slice()'ed that in smaller bits as needed. That's also very handy if you're working with GL_NV_vertex_array_range.

- elias

[princec]

That's what I do in SPGL as well. I use a simple binary tree for allocating and deallocating chunks. Works best for powers of 2 (I wonder if that's how malloc() works?)

Cas

[cfmdobbie]

So the obvious test is to allocate 25, 4K buffers and check the RAM usage then.  What happens in that case?

Games published by our own members! Check 'em out!

[cfmdobbie]

That's what I do in SPGL as well. I use a simple binary tree for allocating and deallocating chunks. Works best for powers of 2 (I wonder if that's how malloc() works?)

Probably.  In the current Linux memory manager that's exactly how it works.  When a request comes in you round up to the nearest free power of two, split it in half, put the first half onto the appropiate list, chop the unused space off the end of the second half and distribute it between the appropiate lists, then return the allocated chunk.

It treats blocks in pairs (the "buddy" system) and checks for a free "buddy" when it frees a block, joins the blocks and moves them to a "higher" list.

Very cool, very fast and very easy to understand.  Nice!

[ap_kelly]

No, please no! Don't take lwjgl down the malloc/free route, one of the beauties of Java is the fact that you don't have to do these kinds of things any more. Why would anyone use Java in this case, they may as well use C/C++.

One thing I'd like to see lwjgl do is hide the fact that these nio buffers are used and implement the calls to OpenGL as the OpenGL documentation suggests i.e. by passing in Float[] rather than pointers to a FloaftBuffer to the various calls. This would certainly make porting existing OpenGL applications written in other languages a whole lot easier.

Regards,

Andy.

[princec]

Don't worry, we don't need malloc() really because direct bytebuffers do what we want. But passing in float[]s to GL functions - NO! Agh! Major performance hit!

Cas

[gregorypierce]

One thing I'd like to see lwjgl do is hide the fact that these nio buffers are used and implement the calls to OpenGL as the OpenGL documentation suggests i.e. by passing in Float[] rather than pointers to a FloaftBuffer to the various calls. This would certainly make porting existing OpenGL applications written in other languages a whole lot easier.

While I agree with you there are 2 reasons why I don't think lwjgl should go in this direction:

1) passing in float[]s mean taking stuff out of the Java stack and moving it into native memory used by the OpenGL driver. Pain and suffering performance wise - especially on architectures where java byte ordering is different than system byte ordering.

2) It would be a maintenance nightmare to have to wrap every individual function call and implement all of the various possible data types that could be passed into these functions. To be honest there is very little gained and for people who've written applications where they were managing their own memory in C/C++, its actually more natural to do it this way than to wrap the function calls.


Its not the fist time its come up, but the performance and maintenance implications far outweight the marginal increase in ease of use. If it were possible to have both, I would do it myself

Now if only I could get Apple to release JDK1.4.1 for OSX... I simply refuse to port any more until I can tell whether the bugs are in Java, LWJGL, or my own code!

[GergisKhan]

1	Now if only I could get Apple to release JDK1.4.1 for OSX... I simply refuse to port any more until I can tell whether the bugs are in Java, LWJGL, or my own code!

Do you have ANY idea how excited I am about this?  I'm biding my time, because the moment this happens, I'm switching to OpenGL.  

There.  That should make a few people happy.

Going from fake 3D in 2D to full 3D once LWJGL is ported, and I'm willing to help out once I get my Mac, too!