Java-Gaming.org Hi !
Featured games (83)
games approved by the League of Dukes
Games in Showcase (522)
Games in Android Showcase (127)
games submitted by our members
Games in WIP (590)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: 1 [2] 3 4 ... 7
  ignore  |  Print  
  Once again! fast MappedObjects implementation  (Read 31641 times)
0 Members and 1 Guest are viewing this topic.
Offline Riven
« League of Dukes »

« JGO Overlord »


Medals: 835
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #30 - Posted 2011-06-26 16:00:38 »

Well, I put a lot of bytecodes in the callsite which makes the method-body rather big, and makes it harder for HotSpot to find the patterns. I'm now rewriting it to method-calls, so HotSpot can inline it, effectively ending up with the same bytecode...

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Riven
« League of Dukes »

« JGO Overlord »


Medals: 835
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #31 - Posted 2011-06-26 16:18:24 »

I'm getting reasonably comparable performance now:

JDK 1.6.0u26 x86
1  
2  
instance took: 355ms
mapped took: 460ms


JDK 1.6.0u26 x64
1  
2  
instance took: 355us
mapped took: 455us


both running server VMs, which brings overhead to 'only' 28%

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Riven
« League of Dukes »

« JGO Overlord »


Medals: 835
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #32 - Posted 2011-06-26 16:58:23 »

I'm a fan of the 'release early' so here goes:
Quote
http://indiespot.net/files/published/mappedobject-0.1.jar (view source)
Main-class: eden.mapped.TestMappedObject
JARs asm-3.2.jar and asm-util-3.2.jar (asm.ow2.org) must be on the classpath.

Code quality is crap, but it works.
Feedback / bug reports are appreciated.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Spasi
« Reply #33 - Posted 2011-06-26 18:10:06 »

I need to add -noverify for it to work. I'm on 1.7.0_b146. Else I'm getting this:

1  
Caused by: java.lang.VerifyError: Expecting a stack map frame in method eden.mapped.TestMappedObject.testWriteFieldAccess(Leden/mapped/MappedVec3;)V at offset 19


Results are similar to yours, 351us instanced vs 453us mapped.

One thing you can try is "baking" MappedObjectUnsafe into MappedObject and transforming the static methods to instance ones, like so:

1  
2  
3  
4  
5  
6  
7  
8  
9  
public void fput(float value, long addr)
   {
      INSTANCE.putFloat(baseAddress + addr, value);
   }

   public float fget(long addr)
   {
      return INSTANCE.getFloat(baseAddress + addr);
   }


That's the only difference I can think of between my implementation and yours (note: I'm not doing any bytecode transformation). I don't know if it will make any difference on performance, but it will make the forked bytecode shorter.
Offline Riven
« League of Dukes »

« JGO Overlord »


Medals: 835
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #34 - Posted 2011-06-26 18:33:44 »

I need to add -noverify for it to work. I'm on 1.7.0_b146. Else I'm getting this:

1  
Caused by: java.lang.VerifyError: Expecting a stack map frame in method eden.mapped.TestMappedObject.testWriteFieldAccess(Leden/mapped/MappedVec3;)V at offset 19

Interesting... I'll look into that.

Results are similar to yours, 351us instanced vs 453us mapped.

One thing you can try is "baking" MappedObjectUnsafe into MappedObject and transforming the static methods to instance ones, like so:

1  
2  
3  
4  
5  
6  
7  
8  
9  
public void fput(float value, long addr)
   {
      INSTANCE.putFloat(baseAddress + addr, value);
   }

   public float fget(long addr)
   {
      return INSTANCE.getFloat(baseAddress + addr);
   }


That's the only difference I can think of between my implementation and yours (note: I'm not doing any bytecode transformation). I don't know if it will make any difference on performance, but it will make the forked bytecode shorter.
What's the performance like in your implementation?

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Spasi
« Reply #35 - Posted 2011-06-26 19:00:17 »

I'm doing multithreaded matrix-matrix multiplication on direct FloatBuffers. It's 10% faster than the same code on float[].
Offline Spasi
« Reply #36 - Posted 2011-06-26 19:05:20 »

Anyway, I just tried it and performance is the same.

(also baseAddress in my previous reply should have been viewAddress)
Offline Riven
« League of Dukes »

« JGO Overlord »


Medals: 835
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #37 - Posted 2011-06-26 19:15:28 »

I'm doing multithreaded matrix-matrix multiplication on direct FloatBuffers. It's 10% faster than the same code on float[].

Interesting, although that's not quite the same as object-field access. I'll see how my mappedobjects compare to float[] performance.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Riven
« League of Dukes »

« JGO Overlord »


Medals: 835
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #38 - Posted 2011-06-26 19:37:16 »

1  
2  
3  
4  
5  
6  
7  
instance took:         357us

mapped took:           447us
backing array took:    454us

plain array took:      384us
plain unsafe took:     379us


I guess I hit the limit of what indirect access to arrays and buffers can achieve...  Sad

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Riven
« League of Dukes »

« JGO Overlord »


Medals: 835
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #39 - Posted 2011-06-26 20:43:33 »

Having said that, any suggestions on the API?

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline concerto49

Junior Devvie





« Reply #40 - Posted 2011-06-27 03:36:04 »

Keep in mind it uses sun.misc.Unsafe (in a safe way) and Janino for the code generation.

Is there a reason why Janino was used for code generation and not say ASM? Is it a better lib? Do you know how they compare?

High performance, fast network, affordable price VPS - Cloud Shards
Available in Texas, New York & Los Angeles
Need a VPS Upgrade?
Offline Spasi
« Reply #41 - Posted 2011-06-27 19:15:08 »

I used the debug JVM to have a look at the generated assembly. The good news is that memory is directly accessed as expected and SSE registers are used to do the math in both cases. The difference between mapped and instanced is in the loop unrolling:

- Mapped is unrolled only 2 times vs 4 for instanced.
- There's some heavy operation reordering in the instanced case, whereas for mapped it's two times the same thing and afaict in the exact same order as the corresponding bytecode instructions.

So I guess Unsafe's disadvantage is that it limits the extend of JVM optimizations that can be applied. That doesn't mean that mapped objects are not useful of course. There are huge memory savings to be had and you can always offload expensive computation to OpenCL. Wink

Riven, I'll need to use this on some real code first to have any input on the API. It looks great so far though. Btw, have you given any thought to nested structs or anything more complex like that?
Offline Riven
« League of Dukes »

« JGO Overlord »


Medals: 835
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #42 - Posted 2011-06-27 19:30:07 »

I used the debug JVM to have a look at the generated assembly. The good news is that memory is directly accessed as expected and SSE registers are used to do the math in both cases. The difference between mapped and instanced is in the loop unrolling:

- Mapped is unrolled only 2 times vs 4 for instanced.
- There's some heavy operation reordering in the instanced case, whereas for mapped it's two times the same thing and afaict in the exact same order as the corresponding bytecode instructions.
I manually unrolled the mapped object benchmark, and unfortunately saw no performance increase.

So I guess Unsafe's disadvantage is that it limits the extend of JVM optimizations that can be applied. That doesn't mean that mapped objects are not useful of course. There are huge memory savings to be had and you can always offload expensive computation to OpenCL. Wink
Well, the real savings (performance wise) are not having to copy everything to/from the buffers. It's far from cheap, as you're trashing the CPU cache. Back in the day (2006, when I was doing rendering) the copying around of data was a fair chunk of the available time in the render-loop.

Riven, I'll need to use this on some real code first to have any input on the API. It looks great so far though. Btw, have you given any thought to nested structs or anything more complex like that?
It's a shame that AALOAD and AASTORE don't give you the type. It would have been great to access structs like java arrays:
1  
vecs[0].x = vecs[1].y;

as opposed messing with mapped.view(int)

As for interleaving mapped objects (not nesting) I'm planning to add support for stride (independent of sizeof) and offset (which is basically an increased baseAddress).

You could make a VTN VBO using:
1  
2  
3  
4  
int stride = (3+2+3)*4;
Vec3 vertices  = Vec3.map(bb, 0*4, stride);
Vec2 texcoords = Vec2.map(bb, 3*4, stride);
Vec3 normals   = Vec3.map(bb, 5*4, stride);

Maybe I'll merge them somehow, so you can .view(index) them all in one go.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Spasi
« Reply #43 - Posted 2011-06-27 19:55:39 »

Indeed, tried it as well, you get the extra unrolling (actually 2 unrolls at the Java level result in 4 unrolls at the assembly level, 4 unrolls at Java is 8 at assembly), but again you don't get the operation reordering.

I agree about the data copy overhead of course, but I recently did a normal (OOP) quad-tree vs serialized quad-tree (with the nodes being "mapped objects") and besides the win on tree traversing and bounds calculation (even more win if you consider how easy it was to parallelize), the normal code was taking forever to start-up and burned loads more memory, due to the instantiation of all those "Node" objects in the tree.
Offline Riven
« League of Dukes »

« JGO Overlord »


Medals: 835
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #44 - Posted 2011-06-27 21:07:40 »

Added support for stride:
Quote
http://indiespot.net/files/published/mappedobject-0.2.jar (view source)
Main-class: eden.mapped.TestMappedObject
JARs asm-3.2.jar and asm-util-3.2.jar (asm.ow2.org) must be on the classpath.

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
         MappedVec3 v = MappedVec3.map(bb);
         MappedVec2 t = MappedVec2.map(bb);
         MappedVec3 n = MappedVec3.map(bb);

         int stride = v.sizeof + t.sizeof + n.sizeof;
         if (stride != 32)
            throw new IllegalStateException();

         MappedVec3.configure(v, stride, 0);
         MappedVec2.configure(t, stride, v.sizeof);
         MappedVec3.configure(n, stride, v.sizeof + t.sizeof);

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Riven
« League of Dukes »

« JGO Overlord »


Medals: 835
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #45 - Posted 2011-06-28 06:53:33 »

Quote
http://indiespot.net/files/published/mappedobject-0.3.jar (view source)
Main-class: eden.mapped.TestMappedObject
JARs asm-3.2.jar and asm-util-3.2.jar (asm.ow2.org) must be on the classpath.

Two bugfixes:
1. Stride is now copied when calling MappedObject.dup(...)
2. The ByteBuffer that is passed to MappedObject.map(...) is stored in a field, to prevent the GC from freeing the memory that is used by the mapped objects.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Spasi
« Reply #46 - Posted 2011-06-28 10:27:36 »

Think you can add a MappedObject.sizeof(Matrix4f.class), where Matrix4f is any subclass of MappedObject? The transformer would then replace the method call with the int value of sizeof that it already knows. You would then be able to do this:

1  
2  
ByteBuffer buffer = ByteBuffer.allocateDirect(nodes * MappedObject.sizeof(Matrix4f.class));
Matrix4f.map(buffer);


instead of

1  
2  
ByteBuffer buffer = ByteBuffer.allocateDirect(nodes * (16 * 4)); // hard-coded & error-prone byte size
Matrix4f.map(buffer);
Offline Riven
« League of Dukes »

« JGO Overlord »


Medals: 835
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #47 - Posted 2011-06-28 11:10:05 »

What I do is basically a big hack, which has nothing to do how java works Smiley

I can do this:
1  
2  
3  
4  
public class MappedObject
{
   public static final int SIZEOF = -1;
}


And rewrite GETSTATIC to push an int on the callsite:
1  
2  
int a = Matrix3f.SIZEOF; // after transform: int a = 48;
int b = Matrix4f.SIZEOF; // after transform: int b = 64;

The compiler will accept it, because SIZEOF is defined in the supertype MappedObject.

I use exactly the same trick to implement MappedObject.map(...) -- I just look what the type of INVOKESTATIC is:
1  
2  
Matrix3f m3 = Matrix3f.map(bb); // after transform: Matrix3f m3 = new Matrix3f(); m3.init(bb, align, 48);
Matrix4f m4 = Matrix4f.map(bb); // after transform: Matrix4f m4 = new Matrix4f(); m4.init(bb, align, 64);


You might get a compile-time warning though, that you should read SIZEOF or call map(...) from the supertype. That's a fair tradeoff.

To answer your question: will do, but as said it will be a static field, as opposed to a method call.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Riven
« League of Dukes »

« JGO Overlord »


Medals: 835
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #48 - Posted 2011-06-28 11:19:54 »

Quote
http://indiespot.net/files/published/mappedobject-0.4.jar (view source)
Main-class: eden.mapped.TestMappedObject
JARs asm-3.2.jar and asm-util-3.2.jar (asm.ow2.org) must be on the classpath.

Changes:
1. added <? extends MappedObject>.SIZEOF
2. removed mappedObject.sizeof

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Spasi
« Reply #49 - Posted 2011-06-28 16:29:32 »

Oh, I love this. Results on a ridiculously large quad-tree (2048x2048):

1  
2  
3  
4  
5  
360.32ms [ Naive QuadTree ]
126.13ms [ Serial QuadTree ]
125.00ms [ Buffer QuadTree ]
124.88ms [ Buffer TLocal QuadTree ]
125.34ms [ Mapped QuadTree ]


Naive is the usual object-oriented implementation. Serial is the serialized version, using float arrays. Matrix multiplication then becomes:

1  
2  
3  
4  
5  
public static void mul4f(final float[] a, final int pa, final float[] b, final int pb, final float[] t, final int p) {
   float m00 = a[pa + 0 * 4 + 0] * b[pb + 0 * 4 + 0] + a[pa + 1 * 4 + 0] * b[pb + 0 * 4 + 1] + a[pa + 2 * 4 + 0] * b[pb + 0 * 4 + 2] + a[pa + 3 * 4 + 0] * b[pb + 0 * 4 + 3];
   ...
   t[p + 0 * 4 + 0] = m00;
   ...


Buffer is my attempt at mapped objects + Unsafe (the FloatBuffer below is NOT java.nio.FloatBuffer, it's my hacked version of it):

1  
2  
3  
4  
5  
public static void mul4f(final FloatBuffer a, final int pa, final FloatBuffer b, final int pb, final FloatBuffer t, final int p) {
   float m00 = a.get(pa + 0 * 4 + 0) * b.get(pb + 0 * 4 + 0) + a.get(pa + 1 * 4 + 0) * b.get(pb + 0 * 4 + 1) + a.get(pa + 2 * 4 + 0) * b.get(pb + 0 * 4 + 2) + a.get(pa + 3 * 4 + 0) * b.get(pb + 0 * 4 + 3);
   ...
   t.put(p + 0 * 4 + 0, m00);
   ...


Buffer TLocal adds something like Riven's view() on the FloatBuffer:

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
public static void mul4f(final FloatBuffer a, final int pa, final FloatBuffer b, final int pb, final FloatBuffer t, final int p) {
   a.setBaseOffset(pa);
   b.setBaseOffset(pb);

   float m00 = a.get(0 * 4 + 0) * b.get(0 * 4 + 0) + a.get(1 * 4 + 0) * b.get(0 * 4 + 1) + a.get(2 * 4 + 0) * b.get(0 * 4 + 2) + a.get(3 * 4 + 0) * b.get(0 * 4 + 3);
   ...
   t.setBaseOffset(p);

   t.put(0 * 4 + 0, m00);
   ...


Finally, Mapped uses Riven's library-of-awesomeness:

1  
2  
3  
4  
5  
public static Matrix4f mul4f(Matrix4f left, Matrix4f right, Matrix4f dest) {
   float m00 = left.m00 * right.m00 + left.m10 * right.m01 + left.m20 * right.m02 + left.m30 * right.m03;
   ...
   dest.m00 = m00;
   ...


Which is basically standard Java code (actually a copy-paste of LWJGL's Matrix4f.mul code). Fun fact: The naive implementation peaks at 834 MB memory used, whereas the Mapped implementation at 346 MB.

Requests:

- Add a static malloc(int count) method. It would be the equivalent of:

1  
MappedClass.map(ByteBuffer.allocateDirect(count * MappedClass.SIZEOF));


- Make dup() an instance method. So instead of:

1  
2  
3  
4  
5  
private static final ThreadLocal<Matrix4f> localMatrices = new ThreadLocal<Matrix4f>() {
   protected Matrix4f initialValue() {
      return Matrix4f.dup(localMatrixData);
   }
};


we do this:

1  
2  
3  
4  
5  
private static final ThreadLocal<Matrix4f> localMatrices = new ThreadLocal<Matrix4f>() {
   protected Matrix4f initialValue() {
      return localMatrixData.dup();
   }
};


It would then be possible to "generify" such code. I've changed your implementation to try this and it seems to work:

1  
2  
3  
4  
5  
// MappedObject.java, line 38
public final <T extends MappedObject> T dup()

// MappedObjectTransformer.java, line 199
if ( opcode == INVOKEVIRTUAL && methodName.equals("dup") && className.equals(mappedType.className) && signature.equals("()L" + jvmClassName(MappedObject.class) + ";") ) {


- Maybe add a method/field to retrieve the currently mapped index (the value you pass to the view method)?

- It would be nice if you could add some extra code for constructors. Basically the current implementation requires an empty constructor (no params, no touching of fields or methods that use the fields, else the JVM crashes). The library should either verify that that's the case, or you could move the init method to the MappedObject constructor and make sure all subclasses call it accordingly (when .map is called). That would require quite a bit of bytecode transformation though and I'm not sure how hard it is to do with ASM.
Offline Riven
« League of Dukes »

« JGO Overlord »


Medals: 835
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #50 - Posted 2011-06-28 17:01:31 »

1  
2  
3  
4  
5  
360.32ms [ Naive QuadTree ]
126.13ms [ Serial QuadTree ]
125.00ms [ Buffer QuadTree ]
124.88ms [ Buffer TLocal QuadTree ]
125.34ms [ Mapped QuadTree ]

Nice!



- Add a static malloc(int count) method. It would be the equivalent of:
Easy enough.



- Make dup() an instance method. So instead of:
I deliberately made it a static method, and was thinking of making view(i) static too. Every instance method I define in MappedObject can't be used by the end-user's type, which might end up quite restricting...

I agree though that both view(...) and dup(...) should be either instance methods or static methods.



- Maybe add a method/field to retrieve the currently mapped index (the value you pass to the view method)?
I might even.... make it a field? Pointing
1  
vec3.view++

GETFIELD would look like: (viewAddress-baseAddress)/stride
PUTFIELD would look like: viewAddress=baseAddress+stack.popInt()*stride



- It would be nice if you could add some extra code for constructors. Basically the current implementation requires an empty constructor (no params, no touching of fields or methods that use the fields, else the JVM crashes). The library should either verify that that's the case, or you could move the init method to the MappedObject constructor and make sure all subclasses call it accordingly (when .map is called). That would require quite a bit of bytecode transformation though and I'm not sure how hard it is to do with ASM.
The problem of a non-default constructor in the supertype is that you *must* declare it in the subtype, or your get a compiletime error. I'd rather not have everybody define annoying constructors in they MappedObject subclasses, so that's why I moved the code to the init() method. No matter how intelligent the transformer, I can't solve these annoying language rules Smiley

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Spasi
« Reply #51 - Posted 2011-06-28 17:01:55 »

I was thinking that maybe you can leave the Java code like it is now, but make the transformer change the constructors on the fly. So you'd go from this:

1  
2  
3  
4  
5  
public MappedObject() {}

public Matrix4f() {
   setIdentity();
}


to this:

1  
2  
3  
4  
5  
6  
7  
8  
public MappedObject(ByteBuffer buffer, int align, int sizeof) {
   init(buffer, align, sizeof);
}

public Matrix4f(ByteBuffer buffer, int align, int sizeof) {
   super(buffer, align, sizeof);
   setIdentity();
}


I just remembered another important request: Add a .copy(source) method, or .copy(source, target) if you want it static, that gets transformed to Unsafe's copyMemory(source.viewAddress, target.viewAddress, MappedClass.SIZEOF). This allows uber-fast copies between mapped objects.

Hmm, is there any chance to support primitive arrays as mapped fields, or would that over-complicate things? They would obviously require a fixed size (via an annotation perhaps?).
Offline Spasi
« Reply #52 - Posted 2011-06-28 17:06:42 »

Aye, view as a field would be great.
Offline Riven
« League of Dukes »

« JGO Overlord »


Medals: 835
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #53 - Posted 2011-06-28 17:11:51 »

I was thinking that maybe you can leave the Java code like it is now, but make the transformer change the constructors on the fly. So you'd go from this:

1  
2  
3  
4  
5  
public MappedObject() {}

public Matrix4f() {
   setIdentity();
}


to this:

1  
2  
3  
4  
5  
6  
7  
8  
public MappedObject(ByteBuffer buffer, int align, int sizeof) {
   init(buffer, align, sizeof);
}

public Matrix4f(ByteBuffer buffer, int align, int sizeof) {
   super(buffer, align, sizeof);
   setIdentity();
}

Why didn't I think of this... Clueless

I just remembered another important request: Add a .copy(source) method, or .copy(source, target) if you want it static, that gets transformed to Unsafe's copyMemory(source.viewAddress, target.viewAddress, MappedClass.SIZEOF). This allows uber-fast copies between mapped objects.
Sure. I'll probably also add copy(MO, MO, count)

Hmm, is there any chance to support primitive arrays as mapped fields, or would that over-complicate things? They would obviously require a fixed size (via an annotation perhaps?).
The problem there is that an array is simply an 'anonymous' reference on the stack. It's quite hard to identify those. I can make assumptions on access patterns, but how would I identify this error:
1  
2  
3  
byte[] arr = mapped.payload;
arr[0] = 4;
arr[1] = 3;

While this would be fine:
1  
2  
mapped.payload[0] = 4;
mapped.payload[1] = 3;

As there would be no byte[] at runtime, I'd have to push a pointer on the stack, and use that in every subsequent array access. Problem is that I really can't track that, as that 'arr' can be passed into a method, which also has regular byte[]s passed in. It really gets much more complex than this relatively simple example.

Maybe you have an idea on how to solve this...?

Edit:
I could 'solve' it like this:
1  
2  
3  
4  
5  
6  
7  
public class Packet extends MappedObject
{
    public int used;

    @MappedByteArray(length=32)
    public ByteBuffer payload;
}


It would be a real ByteBuffer instance, that has its private address-field modified by the 'view' field. To keep the GC from trippin' I'd have to restore the address field before the ByteBuffer would be collected.

It's a horrible hack, but it'd work.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Spasi
« Reply #54 - Posted 2011-06-28 17:38:44 »

Yes, that would be relatively clean, you won't have to touch any code that uses the payload this way. Also... you could map the mapped buffer and get nested structs! Grin
Offline Riven
« League of Dukes »

« JGO Overlord »


Medals: 835
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #55 - Posted 2011-06-28 17:52:23 »

Yes, that would be relatively clean, you won't have to touch any code that uses the payload this way. Also... you could map the mapped buffer and get nested structs! Grin
Nope. the mapped.viewAddress is independant of ByteBuffer.address, so once you modify the view-field the nested mapped object would be detached.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Spasi
« Reply #56 - Posted 2011-06-28 18:04:17 »

Yes, obviously you'd have to remap every time you change the view index.

Btw, I'm not sure how you plan to allocate the mapped buffers, but it could potentially become expensive on e.g. lots of .dup(). Unless you always allocate 1 byte and hack Buffer.capacity as well. Since there's no way to create a direct ByteBuffer anywhere in memory from Java (afaik), we use JNI's NewDirectByteBuffer in LWJGL (see safeNewBuffer in common_tools.h). We use it on functions like GL15.glMapBuffer.
Offline Riven
« League of Dukes »

« JGO Overlord »


Medals: 835
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #57 - Posted 2011-06-28 19:33:10 »

I wouldn't allocate anything on dup(...) or on 'embedding' a nested buffer, I'd just call ByteBuffer.slice() or ByteBuffer.duplicate() on some global (static final) ByteBuffer, they have 'private' address & capacity fields, which you can manipulate, to point it to the desired memory region.

I feel like I didn't quite get the question, because this seems too obvious.


BTW: direct-allocating 1 byte still malloc()s 4096 bytes (1 page) of memory.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Riven
« League of Dukes »

« JGO Overlord »


Medals: 835
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #58 - Posted 2011-06-28 21:47:04 »

Quote
http://indiespot.net/files/published/mappedobject-0.5.jar (view source)
Main-class: eden.mapped.TestMappedObject
JARs asm-3.2.jar and asm-util-3.2.jar (asm.ow2.org) must be on the classpath.

Changes:
1. the new way to manipulate the index is the 'view' field, probably renamed to 'index' (?)
2. added support for malloc(count)
3. added support for copyTo(target)
4. added support for copyRange(target, instances)
5. added support for the creation of arbitrarily ranged ByteBuffers
6. major code cleanup
7. added readme & license
8. made dup(...) an instance method.

Bugfixes:
1. copy the align & buffer fields on dup(...)
2. made MappedObject.configure(...) stricter: view must be zero, or else 'view' can get corrupt as the stride is manipulated
3. worked around an issue where mapped.view++ did a GETFIELD MO.view instead of GETFIELD V3.view, which resulted in fireworks

Todo:
1. mapped buffers Pointing (and thus losely nested mapped objects)
2. support user-defined constructors
3. optional bounds-checks of 'view' field
4. figure out what to do if copyTo(...) / copyRange(...) result int a memcopy with an overlap?
5. rewrite field byte-offset calculation code, to prevent misaligned fields and unions (overflowing 'sizeof')



Please keep in mind that mapped.view++ is currently a rather heavy operation:
1  
2  
3  
int temp = (mapped.viewAddress - mapped.baseAddress) / mapped.stride;
temp++;
mapped.viewAddress = mapped.baseAddress + temp * mapped.stride;

I might, some day, optimize it to:
1  
mapped.viewAddress += mapped.stride;
Yawn

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Spasi
« Reply #59 - Posted 2011-06-29 00:07:10 »

I wouldn't allocate anything on dup(...) or on 'embedding' a nested buffer, I'd just call ByteBuffer.slice() or ByteBuffer.duplicate() on some global (static final) ByteBuffer, they have 'private' address & capacity fields, which you can manipulate, to point it to the desired memory region.

OK, that's "hackier" than I thought, it should work fine.

BTW: direct-allocating 1 byte still malloc()s 4096 bytes (1 page) of memory.

This should be fixed in JDK 7. There's also a flag to force the old behavior (-XX:+PageAlignDirectMemory -Dsun.nio.PageAlignDirectMemory=true).
Pages: 1 [2] 3 4 ... 7
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

trollwarrior1 (32 views)
2014-11-22 12:13:56

xFryIx (73 views)
2014-11-13 12:34:49

digdugdiggy (52 views)
2014-11-12 21:11:50

digdugdiggy (46 views)
2014-11-12 21:10:15

digdugdiggy (40 views)
2014-11-12 21:09:33

kovacsa (64 views)
2014-11-07 19:57:14

TehJavaDev (69 views)
2014-11-03 22:04:50

BurntPizza (67 views)
2014-11-03 18:54:52

moogie (82 views)
2014-11-03 06:22:04

CopyableCougar4 (82 views)
2014-11-01 23:36:41
Understanding relations between setOrigin, setScale and setPosition in libGdx
by mbabuskov
2014-10-09 22:35:00

Definite guide to supporting multiple device resolutions on Android (2014)
by mbabuskov
2014-10-02 22:36:02

List of Learning Resources
by Longor1996
2014-08-16 10:40:00

List of Learning Resources
by SilverTiger
2014-08-05 19:33:27

Resources for WIP games
by CogWheelz
2014-08-01 16:20:17

Resources for WIP games
by CogWheelz
2014-08-01 16:19:50

List of Learning Resources
by SilverTiger
2014-07-31 16:29:50

List of Learning Resources
by SilverTiger
2014-07-31 16:26:06
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!