Java-Gaming.org    
Featured games (91)
games approved by the League of Dukes
Games in Showcase (580)
games submitted by our members
Games in WIP (500)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  C-like contiguous arrays  (Read 2024 times)
0 Members and 1 Guest are viewing this topic.
Offline reza_rob2

Senior Newbie




Java games rock!


« Posted 2004-05-08 00:35:01 »

Hi everyone,

I need C-like, RAM/Cache-contiguous, arrays, as in:

malloc(sizeof(struct s)  *  <very_big_number>);

Is it correct that for this, I must go to JNI?  Is there any other way, that is relatively clean?  I suppose using a StringBuffer might be hackish and slow?

Thanks so much for any insight.
Offline tom
« Reply #1 - Posted 2004-05-08 01:51:43 »

Hava a look at nio buffers. You can allocate and manipulate native memory.

http://java.sun.com/j2se/1.4.2/docs/api/java/nio/ByteBuffer.html

Offline reza_rob2

Senior Newbie




Java games rock!


« Reply #2 - Posted 2004-05-08 12:22:39 »

Thanks Tom.  This is very useful.

Is there any way to do:

1  
arbitrary_object.set_to(nio_buffer, index);


In C we can just say

1  
s = malloc(sizeof(s) * 10);


and that way, it is very easy to give structure to the buffer, and work with it.

What is the best solution for this in Java?

Really appreciate your help.
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline tom
« Reply #3 - Posted 2004-05-08 13:06:48 »

There is no such thing in java. You have to write functions that encode and decode the state of the object to and from the buffer. And you have to keep track of offsets yourself.

Offline Orangy Tang

JGO Kernel


Medals: 51
Projects: 11


Monkey for a head


« Reply #4 - Posted 2004-05-08 13:07:48 »

Quote
I need C-like, RAM/Cache-contiguous, arrays, as in:

Out of interest, why? It almost sounds like you're trying to directly port a C memory pool, when a proper OO object pool would work just as well (if not better). Unless you really are doing something freaky that is.

[ TriangularPixels.com - Play Growth Spurt, Rescue Squad and Snowman Village ] [ Rebirth - game resource library ]
Offline reza_rob2

Senior Newbie




Java games rock!


« Reply #5 - Posted 2004-05-08 22:25:47 »

Arrrrgg!  I don't know what happened, but for some reason I've stopped being notified for activity on this thread.

I'm really glad I checked manually.

Quote
Out of interest, why? It almost sounds like
you're trying to directly port a C memory pool, when a proper OO
object pool would work just as well (if not better). Unless you really
are doing something freaky that is.


Dear Orangy Tang,

The issue is RAM blowup (and cache discontinuity??)

My algorithms may certainly be improvable.  However, I don't think
that what I'm doing, would be unreasonable in C(resource-wise.)

Our last discussion(another thread,) surely helps a lot.

(I'm not trying to port a C program, quite the contrary: I'm
considering porting existing (and so-so working) Java code into
JNI/C.)

If you like to know, it's a terrain rendering algorithm.  I got the idea from:

http://www.flipcode.com/tutorials/geomipmaps.pdf

(Just glanced at it;  no patience to read the details fully.)


The quad tree, is probably a major cause of the RAM blowup.
-----------------------------------------------


The general question about Java (in the long run, and my specific
problem aside) is this:

Suppose you have many nested objects:

class Quad {
     short [] x_range = new short[2]; // small array; overhead
     short [] y_range = new short[2]; // small array; overhead
     QuadNode [][] children;          // small array(s); overhead

     // more small stuff; overhead
};

Moreover, you must new() everything. In C, I use large array;
malloc overhead =~ 0.

So, what is the right way to do this in Java?

If I'm being freaky, let me know please.

Regards, Reza.


PS:  

I _will_ look at the canyon demo's source code.  Ken pointed that out,
on the JOGL thread, and the demo runs fast :-)

Looking forward to it.
Offline tom
« Reply #6 - Posted 2004-05-09 01:26:45 »

The obvious solution to this kind of memory bloat is to use less objects. A smal optimization would be to replace x_range with minx, maxx.

But usually you have to look at the big picture. Try changing the design so you don't need many smal objects. Instead store the data in big arrays. In your example, you could possible have a Grid class with all the ranges of all the Quads in the grid stored in one short[] array.

Offline Nipsu

Innocent Bystander




rock!


« Reply #7 - Posted 2004-05-09 08:43:21 »

When your arrays have fixed sizes, why not combine them in single, continuos array and use constant offset values to access needed portion of the data? Each reference to array is 4 bytes (in 32bit environments) so in the case of three fixed arrays with same type this approach saves 8 bytes / object. That's nearly a megabyte when you have 100k objects of this type.

class NotSmart
{
 short[] x = new short[2];
 short[] y = new short[2];
 short[] z = new short[2];  
}

class Smart
{
 private short[] xyz = new short[6];
 public short getX(int index)  
 {
   return xyz[index];
 }
 // the server vm will optimize method calls away.
 // it will also optimize the +2 too since that's a constant.
 // basically this is the same as direct array access.
 // (not in client vm though).
 public short getY(int index)  
 {
   return xyz[index + 2];
 }
 .
 .
 . etc
}

I suggest that if u need to store and handle primitive types efficiently take a look at Stephanio Vigna's Fastutil project: http://freshmeat.net/projects/fastutil/
Offline swpalmer

JGO Coder




Where's the Kaboom?


« Reply #8 - Posted 2004-05-10 02:47:39 »

Sounds like someone should cast a vote for the structs RFE.  Anyone have the URL handy?

Also, 'Cas did a terrain rendering demo ages ago that was very fast and as far as I know only used very basic OpenGL bindings, no native code or the algorithm.

Offline princec

JGO Kernel


Medals: 282
Projects: 3
Exp: 16 years


Eh? Who? What? ... Me?


« Reply #9 - Posted 2004-05-10 09:51:03 »

Structs RFE: http://developer.java.sun.com/developer/bugParade/bugs/4820062.html
We need all 3 of your votes!!!! Remove your generics votes, we've got that now Wink

Terrain demo - nvidia only (sorry): http://www.puppygames.net/downloads/shaven_puppy_demo.zip

Cas Smiley

Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline reza_rob2

Senior Newbie




Java games rock!


« Reply #10 - Posted 2004-05-10 23:53:49 »

Quote


I read almost all comments regarding this RFE.  I'm not sure where there is more discussion.

I'd appreciate clarification of this point:

1.  Do "child" structures point to the Buffer Object; or only the top parent structure (instances) point back to the BufferObject?

2.  Are all of these structures garbage collected?

3.  Are all structures "passed by reference?"

It seems that the answers are yes.  If so, then is this still compact enough in memory, when we consider the overhead of really small structures?  Or, is the point of the RFE, just to avoid (per structure) new() overhead?  (I can see the RFE achieving this point.)

If auto-garbage-collected, then what happens to the empty memory slot?

I'm a bit rusty on the details.

Thank you.
Offline princec

JGO Kernel


Medals: 282
Projects: 3
Exp: 16 years


Eh? Who? What? ... Me?


« Reply #11 - Posted 2004-05-11 08:36:31 »

The precise semantics haven't really been thrashed out but the general idea is:

A Struct is an abstract class that has a byte buffer offset in it and a reference to a byte buffer. All its fields, which cannot be Object fields, only primitive types, are laid out in a well-defined fashion in memory and instead of being kept on the heap they actually are mapped to offsets into said byte buffer. You can have as many Struct instances as you want but generally you only need one or two, and you can just slide them around the byte buffer by changing the offset. The JVM ensures that the Struct cannot wander off the ends of the buffer.

Cas Smiley

Offline reza_rob2

Senior Newbie




Java games rock!


« Reply #12 - Posted 2004-05-12 02:47:52 »

Could we just say, "we want C# like structs?"

That way, understanding the RFE may be easier for the Sun engineers.

I mean, probably, I would _never_ trust a M$ product for my cross-platform needs.  However, if Sun does it for Java, I think it would be very useful.
Offline princec

JGO Kernel


Medals: 282
Projects: 3
Exp: 16 years


Eh? Who? What? ... Me?


« Reply #13 - Posted 2004-05-12 09:48:54 »

I don't know how C# structs work but if they work like we want them to work - purely as sliding mapped objects inside byte buffers - great. But we don't want to go making serious VM changes. I'm perfectly happy with using new() where it's necessary and letting the VM do escape analysis in the future etc. - don't want any of this by value stuff getting into Java to complicate things.

Cas Smiley

Pages: [1]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

xsi3rr4x (48 views)
2014-04-15 18:08:23

BurntPizza (44 views)
2014-04-15 03:46:01

UprightPath (60 views)
2014-04-14 17:39:50

UprightPath (42 views)
2014-04-14 17:35:47

Porlus (58 views)
2014-04-14 15:48:38

tom_mai78101 (82 views)
2014-04-10 04:04:31

BurntPizza (140 views)
2014-04-08 23:06:04

tom_mai78101 (240 views)
2014-04-05 13:34:39

trollwarrior1 (200 views)
2014-04-04 12:06:45

CJLetsGame (207 views)
2014-04-01 02:16:10
List of Learning Resources
by SHC
2014-04-18 03:17:39

List of Learning Resources
by Longarmx
2014-04-08 03:14:44

Good Examples
by matheus23
2014-04-05 13:51:37

Good Examples
by Grunnt
2014-04-03 15:48:46

Good Examples
by Grunnt
2014-04-03 15:48:37

Good Examples
by matheus23
2014-04-01 18:40:51

Good Examples
by matheus23
2014-04-01 18:40:34

Anonymous/Local/Inner class gotchas
by Roquen
2014-03-11 15:22:30
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!