Java-Gaming.org Hi !
Featured games (91)
games approved by the League of Dukes
Games in Showcase (806)
Games in Android Showcase (239)
games submitted by our members
Games in WIP (868)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  LibStruct  (Read 14588 times)
0 Members and 1 Guest are viewing this topic.
Offline basil_

« JGO Bitwise Duke »


Medals: 418
Exp: 13 years



« Posted 2014-09-09 22:08:47 »

Riven, do you mind any feedback on Structs ?
Offline Riven
Administrator

« JGO Overlord »


Medals: 1371
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #1 - Posted 2014-09-09 22:18:40 »

Spill it! And as long as I don't have to post a wall of text introducing LibStruct to people thinking Objects are just fine, I might even offer some support, patches and new features Smiley

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings!
Offline basil_

« JGO Bitwise Duke »


Medals: 418
Exp: 13 years



« Reply #2 - Posted 2014-09-10 12:26:22 »

sweet. here's my wall of text then Smiley

life is good. agent, the two demos and the tests are running just fine. digging through the code trying to understand what's going on; gotta say, this is one of the most interesting codes i've seen. love it! right now i'm trying to get to fly with my exsiting things, primary particles, meshes and bvh. whoever thinks "objects are just fine" - never tried to move his/her shit to the GPU properly.

there are no objects !


also i can almost see Structs glueing GL-buffers, cl_memory, network-streams, whatnots-based-on-bytebuffer - in a very nice way, hopefully allowing us to get all that more organised. the speed-increase from reduced cache-misses is a nice extra Wink.

i'd love to use Structs for 3d mesh-abstraction. on one hand just working with classes/objects, on the other hand not dealing with the storage (heap, offheap, GL, CL, network), rendering the storage replaceable without hacking too much code. for instance, it's not trivial to extend a mesh abstraction into openCL without rewritting the whole most of the damn thing. not taking about the processing, just the managment.

what i'm not getting yet is; there are multiple concepts going on at the same time and it's not clear which api-pieces are related to which.

bytebuffer mapped arrays and the StructAllocationStacks are super useful so far. just love the way
stack.save()
and
stack.restore()
works. both ideas are nicely differentiated.

//

please correct me, thats what i (hopefully) understand so far but unable to relate :

StructGC
- allocating "mapped" objects with
new
(transparent) or
Struct.calloc(svec3d.class)

- automatic memory managment, garbage collecting
- not convenient to access underlying bytebuffers.
- all that does not apply to mapped-byteBuffers nor StructAllocationStacks.

StructMemory
- sun.misc.Unsafe nitty gritty. setting memory.
- mapping bytebuffers to arrays.
- initialising
StructAllocationStack
s.
-
emptyArray()
, is not a "regular" array but a "struct-array", holding ints/handles. (?)
-- confuses me since it comes with <struct@0> elements.
-- in contrast to
Struct.calloc(some.class,3)
which allocates "objects" ready to use. writing this down unconfuses me actually.

//

anyway, the usage of
emptyArray
is very unclear to me.

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
svec3d[] array = Struct.emptyArray(svec3d.class,1);
array[0] = Struct.calloc(svec3d.class); // that just updates the pointer ?

array[0] = null; // exception. why is it not possible to use null.

// is this a valid workaround for that ? :
svec3d[] array = Struct.emptyArray(svec3d.class, 3);
svec3d _nullElement = array[0];

array[0] = Struct.calloc(svec3d.class);
array[1] = Struct.calloc(svec3d.class);
array[2] = Struct.calloc(svec3d.class);
   
array[1] = _nullElement; // <- argh!

Structs can or cannot be stored in "regular" object-arrays ?

getting very confused by this :
1  
2  
3  
4  
5  
6  
7  
ArrayList<svec3d> list = new ArrayList<>();
list.add(Struct.calloc(svec3d.class));
System.out.println(list.get(0)); // all good, works fine ... or should it not ?

// this crashes with "java.lang.IllegalStateException: found=REFERENCE, required=STRUCT_ARRAY"
Object[] objects = new Object[1];
objects[0] = Struct.calloc(svec3d.class);
aint these two things the same ?

//

on generics :

is it not possible to use mapped classes with generics ? just to show what i mean :
1  
2  
3  
4  
Callable<svec3d> callable = new Callable<svec3d>()
{
  public svec3d call() throws Exception { return null; /* or Struct.malloc(svec3d.class); */ }
};
generates :
1  
2  
3  
java.lang.IllegalStateException: LibStruct failed to rewrite classpath:
[...]
Caused by: java.lang.IllegalStateException: must define how struct return values are handled

adding
@TakeStruct
or
@CopyStruct
yields :
1  
2  
3  
4  
java.lang.IllegalStateException: LibStruct failed to rewrite classpath:
[...]
Caused by: java.lang.IllegalStateException
   at net.indiespot.struct.transform.StructEnv$2$1.visitInsn(StructEnv.java:367)
feels like i'm using the api in a wrong way.


i ran into another issue that is probably related to this.
1  
2  
3  
System.out.println(String.valueOf(Struct.emptyArray(svec3d.class,1))); // works
System.out.println(String.valueOf(Struct.emptyArray(svec3d.class,1)[0])); // works
System.out.println(Arrays.toString(Struct.emptyArray(svec3d.class,1))); // crashes with "Bad type on operand stack"


all that cos' of the idea to have a generic list of struct objects. similiar to your Point, Line or TriangleList. very confusing. could one stick to Object[] arrays and use something similar to ArrayList or is it better to use a List using
Struct.emptyArray()
(like in your example) ? get's me back to generics :

something simple like ..
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
class list<E>
{
  list(Class<E> c)
  {
    // this.array = Struct.emptyArray(c, 3); // even not doing anything in this class
  }
}

list<svec3d> l = new list<>(svec3d.class); //
// crashes with java.lang.IllegalStateException: found=STRUCT_TYPE, required=[REFERENCE, STRUCT_ARRAY, NULL]
not passing the Class to ctor : no crash.

--

some suggestions :

for StructAllocationStack i've added this into StructMemory. i know, not the best way but works for now :
1  
2  
3  
4  
5  
6  
7  
8  
public static ByteBuffer getBuffer(StructAllocationStack stack)
{
  synchronized ( immortal )
  {
    for(Holder h : immortal) if(h.stack() == stack) return h.buffer();
  }
  throw new IllegalStateException();
}
with that it does not matter where i come from. StructAllocationStack or a Struct.map()-ed ByteBuffer created in advance. both can be easily sent to GL for instance.

maybe it'd be better to have a "constructor" for the stack like
1  
2  
ByteBuffer bb;
stack = Struct.createStructAllocationStack(bb)
is it required to have the backing ByteBuffer to be "imortal" for more reasons then just telling java-GC to let it alone ?

the more i think about this the less i like it tho'. maybe it's just enough to grab the pointer at some point and stick it into a custom ByteBuffer. it's about other API's in the end.

//

a method like
Struct.map(some.class, bb)
could be useful, at least convenient :
1  
public static <T> T map(Class<T> structType, long pointer) // pointer to object
and maybe :
1  
public static <T> T[] map(Class<T> structType, long pointer, int numElements) // pointer to object-array

maybe better use a "view"
1  
public static <T,A> A view(long pointer, Class<A> asType,int offsetMultipleOf4)
and maybe use int-handles instead of long-pointers.

seems almost trivial since we need the pointer anyway when mapping a bytebuffer.
long addr = StructMemory.alignBufferToWord(buffer)
. would alignment cause trouble ?

that would open Struct up a bit and bring more pointers from who knows where from onto the table.

//

finally, what does
Struct.free()
do when not using the "default" allocation-stack. does it do anything to StructAllocationStack elements ?

free
allows "dealloc" when i understand the source. did you plan any "dealloc" on the StructAllocationStack's ? would become more a StructAllocationList i guess. when i start dealing with the sparseness of a bytebuffer by myself, moving "removed" objects around, trying to minimisde fragmentation - it feels counterproductive since StructGC is doing this already.

reading
StructGC.local_heaps
could be sufficient unless you advice not to touch these.

i like the clean view on the memory when using a pointers and unsafe but i also appreciate automatic memory management. would be nice to get these two closer together.

//

anyway, these are just my impressions.
again, just using Struct.map() and StructAllocationStack right of the bat is very very nice and sufficient - in a controlled environment tho'. right now it's not too easy to get struct-objects to work seamlessly with existing regular OO-code. those rewritten classes behave a bit different Smiley

now if you add a glsl-compiler to Structs, you made https://code.google.com/p/aparapi/  Tongue

as a side note : wrapping java-interfaces around memory is possible with java.lang.reflect.Proxy. one can achieve some of the functionality Structs provides - with "basic" java - but never at the performance we get when rewritting classes. thanks for making that work Riven!

o/ (sorry for my english)
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline basil_

« JGO Bitwise Duke »


Medals: 418
Exp: 13 years



« Reply #3 - Posted 2014-09-10 12:36:30 »

have to correct myself already ...

1  
2  
3  
ArrayList<svec3d> list = new ArrayList<>();
list.add(Struct.calloc(svec3d.class));
System.out.println(list.get(0));
works but is wrong :

1  
svec3d v = list.get(0);
generates :
1  
java.lang.ClassCastException: java.lang.String cannot be cast to svec3d

makes the discussion about Object[] and Struct.emptyArray obsolete. .. or is the toString() replacement gone too far ?
Offline Riven
Administrator

« JGO Overlord »


Medals: 1371
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #4 - Posted 2014-09-10 15:18:40 »

Nice to see you poking around. Pointing

I see you have a metric ton of misconceptions, which I will happily clearify in a few hours Smiley

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings!
Offline basil_

« JGO Bitwise Duke »


Medals: 418
Exp: 13 years



« Reply #5 - Posted 2014-09-10 15:30:49 »

thanks Smiley

*edit*

tried something like that :

Struct.java, next to
public static long getPointer(Object struct)

1  
2  
3  
4  
5  
public static <T> T fromPointer(Class<T> structType, long pointer)
{
  throwFit();
  return null;
}


and without knowing what i'm doing, StructEnv.java
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
else if(name.equals("fromPointer"))
{
  flow.stack.pop();
  flow.stack.set(0,VarType.MISC);
  flow.stack.set(1,VarType.MISC); // didn't crash the vm yet

  owner = jvmClassName(StructMemory.class);
  desc  = "(J)" + wrapped_struct_flag;
  name  = "pointer2handle";
}


returns a working object struct out of that. just what i needed Smiley
Offline Riven
Administrator

« JGO Overlord »


Medals: 1371
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #6 - Posted 2014-09-10 20:05:24 »

Chapter 1: different types of acquiring structs and their (automatic) disposal


There are a few ways to allocate and free structs, roughly following the way C handles structs:



Stack Allocation

First we have stack-allocation, using the
new
keyword:

1  
2  
3  
4  
5  
6  
7  
8  
9  
public void execute() {
   // here, the stack position is saved (let's say it's 53489448)

   Vec3 a = new Vec3(); // struct at position 53489448
   Vec3 b = new Vec3(); // struct at position 53489460
   Vec3 c = new Vec3(); // struct at position 53489472

   // here the stack position is restored to 53489448
}
The stack, is like the C stack: the address it's at, is saved upon method entry, and is restored upon method termination. Once the stack is popped, all structs created within that method are considered freed, and the references pointing to them must not be used. This means that you would* not be able to return a struct that you stack-allocated within a method, because after the method is terminated, the callsite will receive a reference to a struct that is considered 'undefined memory'.

Another misconception is that stack-allocation is that saving and restoring the stack happens on arbitrary scopes. This is not the case: the following loop will grow the stack, until the JVM crashes:
1  
2  
3  
4  
5  
6  
7  
8  
9  
public void kaboom() {
   // here, the stack position is saved (let's say it's 53489448)

   while(true) {
      new Vec3(); // struct at position 53489448 + (n++)*12
   }

   // here the stack position is restored to 53489448
}
Moving the while-loop body into it's own method, would solve the problem, as the stack would be saved and restored each time the method in the loop body is called.

N.B.: Never free() a stack allocated struct: it is conceptually equal to a double-free in C, leading to memory corruption.


How do we make a stack-allocated struct, available to the callsite? We let LibStruct copy it to a stack-allocated struct in the callsite:
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
@CopyStruct
public Vec3 add(Vec3 a, Vec3 b) {
   Vec3 sum = new Vec3();
   sum.x = a.x + b.x;
   sum.y = a.y + b.y;
   sum.z = a.z + b.z;
  return sum; // copy into callsite happens here, the 'sum' struct goes out of scope
}

Vec3 a = new Vec3(1,2,3); // stack allocated explicitly
Vec3 b = new Vec3(4,5,6); // stack allocated explicitly
Vec3 c = add(a, b); // 'c' is stack allocated implicitly





Mapping structs on byte-buffers

Let's say we have a region in memory in which we want to place our structs. For games, this typically is a VBO.
1  
2  
ByteBuffer bb = ByteBuffer.allocateDirect(...);
Vec3[] mapped = Struct.map(Vec3.class, bb, stride, offset);

The references to the structs in the Vec3[], point to memory addresses in the byte-buffer. You are free to overwrite these references:
1  
2  
mapped[13] = mapped[15];
mapped[14] = new Vec3(); // stack allocated, beware!
The references to these structs are valid, as long as the memory the ByteBuffer is pointing to, is not garbage collected. Structs to not contain a reference to this byte-buffer, so it's up to you to keep the byte-buffer referenced.

N.B.: Never free() a mapped struct: it is conceptually equal to a double-free in C, leading to memory corruption.



Manually allocated and freed structs

If you wish a bit of convenience, you can let LibStruct manage the allocation and free-ing of structs for you.
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
@TakeStruct
public Vec3 add(Vec3 a, Vec3 b) {
   Vec3 sum = Struct.malloc(Vec3.class);
   sum.x = a.x + b.x;
   sum.y = a.y + b.y;
   sum.z = a.z + b.z;
  return sum; // no copy is made, the reference is passed as is
}

Vec3 a = new Vec3(1,2,3); // stack allocated
Vec3 b = new Vec3(4,5,6); // stack allocated
Vec3 c = add(a, b); // 'c' is manually allocated

Struct.free(c); // and must be freed manually

N.B.: Only free() a malloc()ed struct.



Arrays of structs

With
new Vec3[n]
, you are guaranteed to receive a tightly packed sequence of structs.
With
Struct.map(...)
, you are guaranteed to receive a tightly packed sequence of structs.
With
Struct.malloc(Vec3.class, n)
the structs are not guaranteed to be tightly packed, as the memory manager can run out of space in it's current TLAB (thread local allocation buffer) and continue allocating instances on the next TLAB.
With
Struct.emptyArray(Vec3.class, n)
, you get an array of null references, for you to decide how to fill with references, not with data - writing into these (arr[13].x = 0.0f) will crash the JVM with a memory access violation.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings!
Offline Riven
Administrator

« JGO Overlord »


Medals: 1371
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #7 - Posted 2014-09-10 21:01:15 »

Chapter 2: why using structs as objects will break in horrible ways at runtime




A bunch of static methods

When LibStruct encounters a struct, like a simple Vec3, it will turn this:
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
public class Vec3
{
   public float x,y,z;

   public Vec3 add(Vec3 that) {
      ...
   }

   public String toString() {
      return "("+x+", "+y+", "+z+")";
   }
}

into:
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
public class Vec3
{
   // note: no fields!

   public static int add(int this, int that) {
      ...
   }

   public static String toString(int this) {
      // HotSpot recognizes this pattern, and turns all
      // these operations into the LEA x86 instruction.
      // it also allows us to address 16GB (!), instead of 4GB of structs.
      float x = Struct.unsafe.getFloat(((this & 0xFFFF_FFFFL)<<2)+0);
      float y = Struct.unsafe.getFloat(((this & 0xFFFF_FFFFL)<<2)+4);
      float z = Struct.unsafe.getFloat(((this & 0xFFFF_FFFFL)<<2)+8);
      return "("+x+", "+y+", "+z+")";
   }
}

In short: your fields won't exist, and all your methods will be static, including methods intended to override other methods, like if functions like hashCode() and toString() are declared in your struct. This is tricky territory, if, and only if, you're going to use your structs as if they are objects.

A simple callsite, looking like this:
1  
2  
3  
4  
Vec3 a = ...
Vec3 b = ...
Vec3 c = a.add(b);
String s = c.toString();

is rewritten into:
1  
2  
3  
4  
int a = ...
int b = ...
int c = Vec3.add(a,b);
String s = Vec3.toString(c);




Unexpected duplicate method signatures

Let's say you have 2 struct types: Point and Triangle, now this innocent looking code:
1  
2  
3  
4  
5  
public class NonStructType
{
   public void dup(Point p) {...}
   public void dup(Triangle t) {...}
}

will be rewritten to:
1  
2  
3  
4  
5  
public class NonStructType
{
   public void dup(int p) {...}
   public void dup(int t) {...}
}
leading to a duplicate method signature at runtime, causing the bytecode verifier to bark. Beware. Make your method names more explicit if the need arises.



The usage of toString and incorrect parameter arguments

As previously explained, struct variables are nothing but integers holding references to memory addresses.

Let's say we have this code:
1  
2  
Vec3 a = ...;
System.out.println(a);

behind the scenes, this bytecode is generated by javac:
1  
2  
3  
ALOAD 0
GETSTATIC java/lang/System.out
INVOKESTATIC java/lang/PrintStream.println(Ljava/lang/Object;)V

(maybe) you can imagine the problem of rewriting that 'a' variable to an int:
1  
2  
3  
ILOAD 0
GETSTATIC java/lang/System.out
INVOKESTATIC java/lang/PrintStream.println(Ljava/lang/Object;)V

we just attempted to push an
int
argument into a parameter defined as java.lang.Object. The bytecode verifier will instantly reject this mockery. So why don't we just rewrite the parameter to accept an int,like so:
1  
2  
3  
ILOAD 0
GETSTATIC java/lang/System.out
INVOKESTATIC java/lang/PrintStream.println(I)V

Well, in this case, PrintStream.println(int) actually exists, but we're just lucky. The vast majority of methods with Object parameters, do not have an overloaded method with an int, and even if so, you'd wonder whether you could just call a totally different method, just because its parameter types fit your need.

Long story short: this is a problem that cannot be solved correctly, because the callsite is actually making a mistake! It is trying to pass a struct to a method that expects an object. The Java Compiler won't complain though, because the compiler doesn't know about LibStruct, and the code is perfectly valid at compile-time (as opposed to rewrite-time).

People (the first users of LibStruct) however, kept making this mistake, so I hacked up LibStruct quite badly, so that in case it detects this problem, it injects code that converts the struct argument (at runtime, an int argument), into a String argument, with a plain description of your struct: "<struct@...address...>"

With this arcane hack, you can finally do what should be forbidden in the first place:
1  
System.out.println(a); // prints: "<struct@394843575>"




Java Collection classes and Generics
The collection classes in the java.util-package only deal with Objects. You can't use them with structs. Structs are not objects, using them as such will only lead to misery. Case closed. Same with generics: Generics won't work for primitives, only for (types of) objects, and structs are not objects.

How would the following code work?
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
interface Producer<T> {
   T produce();
}
class StringProducer implements Producer<String> {
   String produce() {
      return "yarn";
   }
}
class Vec3Producer implements Producer<Vec3> {
   Vec3 produce() {
      return Struct.malloc(Vec3.class);
   }
}

String s = new StringProducer().produce();
Vec3 v = new Vec3Producer().produce();

If at all possible, the last line would be rewritten to:
1  
int v = new Vec3Producer().produce();

which would require support for primitive-generics, and a Producer that had a magically overloaded produce() method that returned an
int
, breaking the contract of Producer. Both problems are not something we should seek to solve, as struct(type)s simply should not be used as object(type)s.


The only reason the following works:
1  
2  
3  
List<Vec3> vecs = new ArrayList<Vec3>();
vecs.add(new Vec3(1,2,3)); // really?
Vec3 vec = vecs.get(0); // kaboom

is due to the [struct-argument to object-parameter] hack, actually adding a String ("<struct@9384753>") to the type-erased ArrayList. Upon calling vecs.get(0), the Java Compiler will have sneaked in a cast-check to Vec3, to add a safety net around type-erasure, and at runtime your code will throw a ClassCastException (String cannot be cast to Vec3):
1  
2  
3  
4  
5  
List vecs = new ArrayList();
vecs.add("<struct@9384753>"); // really?
Vec3 vec = (Vec3)vecs.get(0); // kaboom
// or rather
int vec = (int)vecs.get(0); // doubly kaboom




With that out of the way, just don't. Don't use structs as if they are objects. For these kinds of problems, think of them as really wide primitives, without Java 5's auto-boxing.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings!
Offline Riven
Administrator

« JGO Overlord »


Medals: 1371
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #8 - Posted 2014-09-10 21:20:13 »

More chapters will be written after a good night's sleep. Pointing

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings!
Offline basil_

« JGO Bitwise Duke »


Medals: 418
Exp: 13 years



« Reply #9 - Posted 2014-09-11 09:01:48 »

thanks alot! looks like this is the lost documentation. Pointing

clarifying the transformation into statics explains most of my head scratching.

wide primitives nails it nicely. keeping that in mind makes it much easier to work with. should i still hope for primitive-generics in java ? Emo

Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Spasi
« Reply #10 - Posted 2014-09-11 14:37:47 »

should i still hope for primitive-generics in java ?

Yes, it's part of Project Valhalla.
Offline Riven
Administrator

« JGO Overlord »


Medals: 1371
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #11 - Posted 2015-01-23 00:56:11 »

a method like
Struct.map(some.class, bb)
could be useful, at least convenient :
1  
public static <T> T map(Class<T> structType, long pointer) // pointer to object
and maybe :
1  
public static <T> T[] map(Class<T> structType, long pointer, int numElements) // pointer to object-array

maybe better use a "view"
1  
public static <T,A> A view(long pointer, Class<A> asType,int offsetMultipleOf4)
and maybe use int-handles instead of long-pointers.

seems almost trivial since we need the pointer anyway when mapping a bytebuffer.
long addr = StructMemory.alignBufferToWord(buffer)
. would alignment cause trouble ?

that would open Struct up a bit and bring more pointers from who knows where from onto the table.

Quite a bit has changed in LibStruct, and I incorporated your request.
https://github.com/riven8192/LibStruct/blob/master/src/net/indiespot/struct/cp/Struct.java#L261

1  
2  
3  
Vec3 vec = Struct.fromPointer(long pointer, Vec3.class);
Vec3[] vecs = Struct.fromPointer(long pointer, Vec3.class, int length);
Vec3[] vecs = Struct.fromPointer(long pointer, int stride, int length);


Note that struct arrays are actually
int[]
filled with references, behind the scenes. It introduces a bit of overhead to have this indirection between the array index and the struct pointers, as we're fetching values from memory that we can calculate instead (memory is slow). Let's imagine we allocate an Vec3[]:
1  
Vec3[] vecs = Struct.mallocArray(Vec3.class, 10001);

This causes an int[] to be created like so:
1  
2  
3  
4  
int[] vecs = new int[10001];
long pointer = malloc((long)sizeof * vecs.length);
for(int i=0; i<vecs.length; i++)
   vecs[i] = pointer2handle(pointer + (long)i*sizeof);
basically we have an int[] with a lot of predictable (redundant) values. It has its merits, as it allows you to do:
vecs[13] = vecs[138];
or
vecs[14] = new Vec3();
but often you simply want to 'map' an array to a fixed range of memory.

Therefore (yet) another API has been introduced to get rid of this level of indirection, by adding convenience methods that help addressing contiguous structs:
1  
2  
3  
4  
5  
6  
7  
8  
// old skool:
Vec3[] vecs = Struct.mallocArray(Vec3.class, 10001);
for(int i=0; i<vecs.length; i++) {
   Vec3 vec = vecs[i]; // indirection
  // compiled to: int vec = vecs[i];
   vec.x = i * 0.125f;
}
Struct.free(vecs); // puts some strain on the LibStruct GC, which has to free 10001 handles

1  
2  
3  
4  
5  
6  
7  
8  
9  
// die hard mode:
int count = 10001;
Vec3 base = Struct.mallocArrayBase(Vec3.class, count); // no int[], just an int
for(int i=0; i<count; i++) {
   Vec3 vec = Struct.index(base, Vec3.class, i); // direct pointer
  // compiled to: int vec = base + (12 >> 2) * i;
   vec.x = i * 0.125f;
}
Struct.free(base); // no strain on the LibStruct GC, which has to free only 1 handle

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
// fully in control:
Bytebuffer bb = ...;
long addr = StructUnsafe.getBufferBaseAddress(bb);
for(int i=0; i<10001; i++) {
   // doing the math yourself - this is actually not as fast as
   // Struct.index(...) due to converting ints to and from longs
   Vec3 vec = Struct.fromPointer(addr + (long)i * Struct.sizeof(Vec3.class), Vec3.class);
  // compiled to: int vec = (int)((addr + (long)i * 12) >> 2);
   vec.x = i * 0.125f;
}
// no GC whatsoever, as we never malloc'd.


Hopefully this 'opens up' LibStruct for more general purpose projects Smiley


Breaking news: the structs returned by mallocArray/mallocArrayBase are guaranteed (since a few weeks) to be in a contiguous block of memory. All memory handles (ints) are treated as unsigned integers (longs) and bitshifted 2 positions to the left. This allows you malloc & address up to 16GB (OS/hardware limits apply) with 32 bit handles.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings!
Offline ra4king

JGO Kernel


Medals: 508
Projects: 3
Exp: 5 years


I'm the King!


« Reply #12 - Posted 2015-01-23 06:31:45 »

Very nice job Riven, this will allow for a nice little optimization in my game.

Pages: [1]
  ignore  |  Print  
 
 

 
Riven (587 views)
2019-09-04 15:33:17

hadezbladez (5529 views)
2018-11-16 13:46:03

hadezbladez (2410 views)
2018-11-16 13:41:33

hadezbladez (5790 views)
2018-11-16 13:35:35

hadezbladez (1233 views)
2018-11-16 13:32:03

EgonOlsen (4669 views)
2018-06-10 19:43:48

EgonOlsen (5688 views)
2018-06-10 19:43:44

EgonOlsen (3205 views)
2018-06-10 19:43:20

DesertCoockie (4104 views)
2018-05-13 18:23:11

nelsongames (5125 views)
2018-04-24 18:15:36
A NON-ideal modular configuration for Eclipse with JavaFX
by philfrei
2019-12-19 19:35:12

Java Gaming Resources
by philfrei
2019-05-14 16:15:13

Deployment and Packaging
by philfrei
2019-05-08 15:15:36

Deployment and Packaging
by philfrei
2019-05-08 15:13:34

Deployment and Packaging
by philfrei
2019-02-17 20:25:53

Deployment and Packaging
by mudlee
2018-08-22 18:09:50

Java Gaming Resources
by gouessej
2018-08-22 08:19:41

Deployment and Packaging
by gouessej
2018-08-22 08:04:08
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!