Java-Gaming.org    
Featured games (91)
games approved by the League of Dukes
Games in Showcase (579)
games submitted by our members
Games in WIP (500)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  Java2D & Native Buffers  (Read 4829 times)
0 Members and 1 Guest are viewing this topic.
Offline Frederick

Senior Newbie





« Posted 2008-02-24 16:22:51 »

Hey!

This is a java2d - opengl related question.

Basically I want to draw into a BufferedImage and pass that image as OpenGL texture.
I know there is a method to create a ByteBuffer from BufferedImage - it looks like this:

1  
2  
byte[] imageArray = ((DataBufferByte) image.getRaster().getDataBuffer()).getData();
ByteBuffer imageBuffer = ByteBuffer.wrap(imageArray);


However this won´t work for me, because I need the buffer to be direct, the buffer retrieved
by this method is not direct. Unfortunately my JNI methods (yes thats right - I rolled my own   Grin )
won´t accept non direct buffers. A hacky and a performancewise not so optimal solution may
be to copy the non-direct buffer into a direct buffer, but I really wouldn´t like my little program
to spend time on copying  Wink

So is there a way to wrap a BufferedImage around a direct buffer ? Technically that should be
no problem...

Offline Riven
« League of Dukes »

JGO Overlord


Medals: 605
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #1 - Posted 2008-02-24 20:00:35 »

Unfortunately, you'll have to copy the byte[] into a direct buffer yourself.

The problem is, that as long as we cannot pin down objects (the GC must not move these objects in the heap) we can't make a direct buffer point to these memory-locations. Further, I highly doubt anybody at Sun would be willing to change the public API to give everybody and their dog access to BufferedImage internals, as those seem to get rehauled every few years..

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Frederick

Senior Newbie





« Reply #2 - Posted 2008-02-25 03:00:54 »

Okay I am a bit frustrated by now !

I found a way to solve that problem.

Sun engineers have nicely abstracted all properties of a graphics surface.

The key class is WritableRaster which uses a DataBuffer. A DataBuffer is an encapsulation of the actual data representation. For example the most used implementation of DataBuffer is the DataBufferByte which is implemented by a byte array. So the way to go is implement an own version of Data Buffer:

Quote
public class DataBufferDirectByte extends DataBuffer {
    protected ByteBuffer directByteBuffer;
    public DataBufferDirectByte(ByteBuffer directByteBuffer) {
        super(TYPE_BYTE, directByteBuffer.capacity());
        this.directByteBuffer = directByteBuffer;   
    }
   
    public int getElem(int bank, int i) {
        return directByteBuffer.get(i);
    }

    public void setElem(int bank, int i, int val) {
        directByteBuffer.put(i, (byte)val);
    }

    public ByteBuffer getDirectByteBuffer() {
        return directByteBuffer;
    }
}

Very Easy uh ?  Wink
Afterwards one has to wade through lots of other classes, there need to be surfaceformats and colormodels defined before it is finally possible to build an ImageBuffer.

This looks like this:
Quote
int[] offsetarray = {0,1,2,3};
            PixelInterleavedSampleModel smodel =
                    new PixelInterleavedSampleModel(DataBuffer.TYPE_BYTE, 256,256,4,256*4,offsetarray);
            WritableRaster r = (WritableRaster) Raster.createRaster(smodel,
                    new DataBufferDirectByte(BufferUtils.allocateDirectByteBuffer(256*4)), new Point(0,0));
           
            ColorModel m = ImageUtil.createColorModel(r.getSampleModel());
            BufferedImage i = new BufferedImage(m,r,false, null
);
The BufferUtils class was written by me.

But then happens the big #!$5$!*
Quote
ByteComponentRasters must have byte DataBuffers
java.awt.image.RasterFormatException: ByteComponentRasters must have byte DataBuffers
        at sun.awt.image.ByteComponentRaster.<init>(ByteComponentRaster.java:142)

A class somewhere inside the jdk jungle complains that our newly introduced DataBuffer is not of byte type. What the *!& ? We have defined it to be of byte type by calling the constructor of the superclass with the apropriate parameter;
1  
super(TYPE_BYTE, directByteBuffer.capacity());


The type might be queried then with the DataBuffer.getDataType method.

Then I have a look at the implementation of this "ByteComponentRasters" class and what do i find Huh

1  
2  
3  
4  
5  
6  
 
if (!(dataBuffer instanceof  DataBufferByte)) {
                  throw new RasterFormatException(
                           "ByteComponentRasters must have "
                                  + "byte DataBuffers");
              }


That makes me wanna cry ! They actually confused their design ! They are checking the actual class types, instead they should
have been using:
switch(dataBuffer.getDataType);

Oh nooo! I actually think that this is a bug and was not meant this way. No point in abstracting everything, so the system may nicely be extended, and then to allow only one type.

 Angry Angry Angry Angry

No more motivation to continue...

What do you think about this ?
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline tom
« Reply #3 - Posted 2008-02-25 08:23:21 »

You could access the BufferedImage array directly from your JNI method?

Offline Frederick

Senior Newbie





« Reply #4 - Posted 2008-02-25 12:57:46 »

Hey Tom I am trying to do that  Grin

But I believe the only way to access *ANYTHING*  via JNI is to put
it into direct buffers. I would never pass java objects down to c++, for
perfomance reasons. I believe it was a wrong decision from sun
(at least for my purposes) design JNI like it is now. It would have been
better if C++ could directly access java class fields and methods, even
if that would mean recompiling the dll if a new java version comes
around.

If I remember correctly the issue with accessing java arrays from c++
is that the arrays gets copied or something. I will have a look at that later.
Have to go to uni,

Thanks
Frederick
Offline Noya

Senior Newbie





« Reply #5 - Posted 2008-02-25 13:01:13 »

Quote from: Frederick
public class DataBufferDirectByte extends DataBuffer

If I remember correctly, instanceof checks the whole inheritance tree. So you could just write "extends DataBufferByte". It's not clean, but hey it should work Wink
Offline tom
« Reply #6 - Posted 2008-02-25 14:51:21 »

There is a way to access java arrays from JNI c code without the array being copied. There is a special array mehtod you can use from JNI. I think it's called getPrimitiveArrayCritical, or something like that. The VM has to support pinning and a few other restrictions. But you should atleast check it out.

EDIT: http://download.java.net/jdk7/docs/technotes/guides/jni/spec/functions.html (search for GetPrimitiveArrayCritical)

Offline Frederick

Senior Newbie





« Reply #7 - Posted 2008-02-25 18:15:18 »

Hey!

Quote
If I remember correctly, instanceof checks the whole inheritance tree. So you could just write "extends DataBufferByte". It's not clean, but hey it should work Wink

I wouldn´t really like that, because I switched to Java because I want to program nice and cleanly  Wink
That way I would inherit an array data structure and methods I don´t need - kind of a mess.

Quote
There is a way to access java arrays from JNI c code without the array being copied. There is a special array mehtod you can use from JNI. I think it's called getPrimitiveArrayCritical, or something like that. The VM has to support pinning and a few other restrictions. But you should atleast check it out.

Thanks for the link, I might give that a try. I plan to use only modern VMs so no problem with pinning I hope. Still I would like to use the buffers, but...

Maybe I should file this as a bug report ? I would tend to call it a bug, because it is definetly not a feature *lol* Also it would please me to know, that I can render to any surface I like and the jdk supports this functionality - unfortunately only theoretically.

Offline trembovetski

Senior Member




If only I knew what I'm talking about!


« Reply #8 - Posted 2008-02-25 23:18:27 »

It is not a bug. You need to define your own Raster type to handle your DataBuffer.

However, the problem is that even when you make it all work - that is,
create a BufferedImage with your direct data buffer, the only way
Java2D would be able to render to/from it is using get/setElem,
which will be excruciatingly slow - since Java2D doesn't know
how to access your data other than through get/set elem methods.

What we really need is proper support for DataBuffers with out of heap
data placement. I believe there's an RFE somewhere already.
Unfortunately this will have to wait at least until Java 7.

Dmitri
Offline Frederick

Senior Newbie





« Reply #9 - Posted 2008-02-26 11:58:57 »

Quote
It is not a bug. You need to define your own Raster type to handle your DataBuffer.

Okay that´s too much work then. I had thought, because the raster has a pluggable DataBuffer it would suffice to exchange it.

Quote
Java2D would be able to render to/from it is using get/setElem,
which will be excruciatingly slow

 I really hope the JVM will inline getter/setter methods ?

Quote
What we really need is proper support for DataBuffers with out of heap
data placement.

I would really appreciate stack allocation too, but i don´t understand how it could help in this case.  Huh Huh Smiley

...I think I will stick with toms suggestion - everything else seems to be too complicated...
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Frederick

Senior Newbie





« Reply #10 - Posted 2008-02-26 14:32:48 »

Implemented the method with "getPrimitiveArrayCritical" - works fine so far, but of course I can´t check if an internal copy is made - lets hope the best  Wink
Offline CaptainJester

JGO Knight


Medals: 12
Projects: 2


Make it work; make it better.


« Reply #11 - Posted 2008-02-26 15:34:52 »

Why don't you just do this?
1  
2  
3  
4  
//image needs to be initialized as new BufferedImage(width, height, BufferedImage.TYPE_4BYTE_ABGR);
byte[] imageArray = (byte[]) image.getRaster().getDataElements(0, 0, image.getWidth(), image.getHeight(), null);
ByteBuffer imageBuffer = ByteBuffer.allocateDirect(imageArray.length);
imageBuffer.put(imageArray);

As long as you use the correct type in the constructor you can cast it to a byte array.

Offline Riven
« League of Dukes »

JGO Overlord


Medals: 605
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #12 - Posted 2008-02-26 16:14:20 »

Because we wanted to avoid that data-copy...

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline trembovetski

Senior Member




If only I knew what I'm talking about!


« Reply #13 - Posted 2008-02-26 20:05:34 »

Okay that´s too much work then. I had thought, because the raster has a pluggable DataBuffer it would suffice to exchange it.

 I really hope the JVM will inline getter/setter methods ?

It's not only the cost of getter/setter methods, it's the fact that with such
image we'll use the most generic rendering loops available, written
in java. The vm isn't yet up to the task of competing with highly optimized
native code (which is used when one of the image types we know
about is used) in this case.

For example, a fillRect will translate to tons of setElem calls (one per pixel),
while for 'normal' image it will be a bunch (or a single one) of quick memsets.

Dmitri
Offline Frederick

Senior Newbie





« Reply #14 - Posted 2008-02-27 12:19:34 »

Ahh okay I see  Grin

I had always thought that rasterization algorithms were written in java, because people
referred to swing as a pure java solution and being therefore more sluggish than SWT, which
has native drawing code.
So that is not entirely true.

Good to hear that Java2D uses native code  Smiley I also believe that native code is superior for that
kind of things, because one can use SIMD, which is cool, one can care for memory alignment and
cache utilization and of course graphics hardware acceleration.

So thank you all - you are awesome
wish me luck, so that I actually get something done  Wink
Frederick 

[EDIT: major orthographic corrections  Roll Eyes]
Offline trembovetski

Senior Member




If only I knew what I'm talking about!


« Reply #15 - Posted 2008-02-27 17:49:29 »

Just a note: ideally we would have all our loops written in Java and
let hotspot worry about proper translation to machine code
(using SIMD or whatever is available) - at least, when rendering
to a heap-based image (like BufferedImage).

We do have a project
which works in this direction - but since it requires a cooperation from
the super-busy vm team, it's not progressing much..

Dmitri
Offline Frederick

Senior Newbie





« Reply #16 - Posted 2008-02-28 11:32:50 »

Hi trembovetski  Smiley

I heard that auto-vectorization is really a hard problem and that actually no existing compiler is good at it at the moment. Good to hear that there is work done on it. Personally I wouldn´t mind to write that code by myself, maybe for a particle simulation, skinning or something like that. Maybe java performance is just good enough - time will show - I haven´t pushed my machine to the limit up to now  Grin
But when time has come it would be cool to have the opportunity.
Then there is always the tradeoff between speedup and native calling overhead, caused by JNI, to be considered. I suppose SIMD matrix multiplication wouldn´t  be  much faster than a java version because of that !?

What I really would appreciate is some improved capability to talk to native code, it would be COOL when java could at least partially have memory layout compatible to c++, maybe using structs or something like that. I had to do some workarounds in that area. But I understand  that this is not the main application area of java.

Thank you for your nice comments !
Offline Linuxhippy

Senior Member


Medals: 1


Java games rock!


« Reply #17 - Posted 2008-03-06 12:44:37 »

There was some discussion about pinning in the VM: As far as I can say nobody should expect pinning to be available anymore.
Hotspot itself does not support pinning, as it would destroy the bump-the-pointer allocation strategy - and because allocation for typical java programs is much more important than java->C data transport I don't think things will change.

The get*Critical Methods themself block the GC, therefor allow to access the data without copying it.

By the way, my old Athlon-Thunderbird-1Ghz (PC100 ram) copied arround with about 800mb/s - so nothing to really worry about anymore Smiley

lg Clemens
Offline trembovetski

Senior Member




If only I knew what I'm talking about!


« Reply #18 - Posted 2008-03-06 18:29:58 »

I have been told that using Get/Set<Type>ArrayRegion may actually be better than
GetPrimitive* .

You might want to try it.
Offline Frederick

Senior Newbie





« Reply #19 - Posted 2008-03-11 20:52:19 »

Hey Trembovetski - Thanks  Wink

I will definately give that a try... at the moment I am quite busy with learning, so that will have to wait a little. Big thanks, that you asked somebody for it  Cheesy

@Linuxhippy: 800mb/s seems impressive first but it is not so if you divide that value by the fps-count. Lets say by 40 - then we will have 20MB per frame. For 2D Animation like video etc. one has send data constantly to the gpu (I am not sure that I want to do THAT  Wink). Things like this will keep adding up if one does not care. I am not a hardcore optimizer and i would always prefer simplicity rather than speed - but needless copying is something one really should avoid, I think.
Pages: [1]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

xsi3rr4x (31 views)
2014-04-15 18:08:23

BurntPizza (28 views)
2014-04-15 03:46:01

UprightPath (43 views)
2014-04-14 17:39:50

UprightPath (26 views)
2014-04-14 17:35:47

Porlus (43 views)
2014-04-14 15:48:38

tom_mai78101 (64 views)
2014-04-10 04:04:31

BurntPizza (124 views)
2014-04-08 23:06:04

tom_mai78101 (224 views)
2014-04-05 13:34:39

trollwarrior1 (190 views)
2014-04-04 12:06:45

CJLetsGame (198 views)
2014-04-01 02:16:10
List of Learning Resources
by SHC
2014-04-18 03:17:39

List of Learning Resources
by Longarmx
2014-04-08 03:14:44

Good Examples
by matheus23
2014-04-05 13:51:37

Good Examples
by Grunnt
2014-04-03 15:48:46

Good Examples
by Grunnt
2014-04-03 15:48:37

Good Examples
by matheus23
2014-04-01 18:40:51

Good Examples
by matheus23
2014-04-01 18:40:34

Anonymous/Local/Inner class gotchas
by Roquen
2014-03-11 15:22:30
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!