Java-Gaming.org    
Featured games (91)
games approved by the League of Dukes
Games in Showcase (580)
games submitted by our members
Games in WIP (499)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  Offscreen Canvas3D Update for hand-made Occlusion  (Read 1800 times)
0 Members and 1 Guest are viewing this topic.
Offline Konterfei

Senior Newbie





« Posted 2005-04-26 11:07:52 »

Hello everybody.

Didn't find a solution searching here or in google. Perhaps someone can forward me to some hints or help me out a little bit. I'm just starting off with Java3D. Normal display and manipulation of my universe works fine, but now I'm getting into tricky parts...

My overall goal is to display java3D-objects above an image or video, where the 3d stuff is occluded by real-life-objects out of the (video)image. Of course I don't get the 3d-info out of my image, but I rather have a prepared mask for occlusion or representative 3d-models for that (which I don't want to draw but only use for occlusion).

Unfortunatelly I didn't find out how it would be possible to use 3d-models only for filling the z-Buffer, but not the display... (java3D seems to be TOO high-level for that).
Any idea?

My other plan is to grab the canvas3D-image as a BufferedImage, do an AND-operation with my mask-image and underlay the result with the background-image. This works fine for the first frame, but (and here I get to the point) if I have any Behaviours in my world changing positions for example the Canvas3D does not update itself... and my grabbed image of the 3D-world is always the first one...
Functions like renderOffScreenBuffer() or startRenderer() don't seem to change a thing...

Any advice how to still get a 3D-world in a Canvas3D to run, which is NOT displayed directly (only drawn offscreen)?

Here some sourcecode:

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  
42  
43  
44  
45  
46  
47  
48  
49  
50  
51  
52  
53  
54  
55  
56  
public class Something extends Frame implements KeyListener
{

BufferedImage bImage;
...

setLayout(new FlowLayout());
GraphicsConfiguration config = SimpleUniverse.getPreferredConfiguration();

// initialise canvas OFFSCREEN (flag = true)
canvas3D = new Canvas3D(config, true);
canvas3D.getScreen3D().setSize(350, 350);
canvas3D.getScreen3D().setPhysicalScreenWidth(0.254 / 90.0 * xsize);
canvas3D.getScreen3D().setPhysicalScreenHeight(0.254 / 90.0 * ysize);

...

// creates some objects and Behaviours which don't need user interaction
BranchGroup scene = createSceneGraph();
SimpleUniverse simpleU = new SimpleUniverse(canvas3D);
simpleU.getViewingPlatform().setNominalViewingTransform();
simpleU.addBranchGraph(scene);

...

bImage = new BufferedImage(350, 350, BufferedImage.TYPE_4BYTE_ABGR);
ImageComponent2D buffer = new ImageComponent2D(ImageComponent.FORMAT_RGBA, bImage);
canvas3D.setOffScreenBuffer(buffer);
canvas3D.renderOffScreenBuffer();
canvas3D.waitForOffScreenRendering();
bImage = canvas3D.getOffScreenBuffer().getImage();

// I add the buffered image to some panel, afterwards I see the first frame of the world in it
// ImagePanel extend Canvas
ImagePanel panel = new ImagePanel(bImage);
add(panel);

...

// after this I can see the Canvas in my Frame displaying the BufferedImage-copy of the 3dworld... but only the first frame

// I defined a keyListener to my Frame which triggers this:

public void update3DView()
{
      canvas3D.startRenderer();
      canvas3D.renderOffScreenBuffer();
      canvas3D.waitForOffScreenRendering();
     
      bImage = canvas3D.getOffScreenBuffer().getImage();
     
      // whatever I try it does not trigger an update
     panel.repaint();
      canvas3D.repaint();
      repaint();
}


Well. That's it so far.
Don't know where the error could lie...
some missing calls for the canvas3D/theUniverse, some missing AWT update, ... hmm.

If I use the same universe created by createSceneGraph() with an onscreen Canvas3D all animation is triggered perfectly.

Thanks in advance,
cu,

Konterfei.
Offline Jeff

JGO Coder




Got any cats?


« Reply #1 - Posted 2005-04-26 17:44:37 »

As i udnerstand it, what yoi uare trying to do is a matte.

You REALLY want the stencil buffer for this.  Unfortunately Java3D doesnt have stencil buffer support yet... its coming.

Edit:  Just talked to Kevin on the J3D team. He suggests you loo at a  "Depth Only Raster."

He thinks you can use it t do your mattes, but he says it hasnt got a lot of use so if you have problems, let us know!

Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Offline Konterfei

Senior Newbie





« Reply #2 - Posted 2005-04-27 07:48:46 »

Thanks so far. I'll give it a try. Have no clue how to use it yet, but I'll see... It's "javax.media.j3d.Raster" you are suggesting, right?

Quote
but he says it hasnt got a lot of use


what does he mean by that?

Don't know if "matte" covers it by 100%: I'm not only trying to put some 2D-image in the background of a scene, but important is the possibility to occlude the 3d-objects partly. I'll put an example here:



top left: background image, 2D
top right: my 3d-scene with a rotating object
bottom left: a pre-defined 2D-mask
bottom right: my goal I'd like to achieve:

right now I did it "by hand": capturing the 3d-frame from an offscreen-canvas and checking pixel by pixel if the mask is set to 1 or not. if so, I would occlude my 3d-object... Well, yes. I'd like a stencil buffer. ;-)

So, this is working so far, but the problem of the not-updating-offscreen-canvas remains... I always only see the 1st frame (no renderOffscreenBuffer helps...). If I could do the same now using an onscreen Canvas3D plus a Raster then I'll be fine, I guess.

Perhaps this helps to point out the problem?

Greetings.
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Konterfei

Senior Newbie





« Reply #3 - Posted 2005-04-27 12:23:12 »

OK, great! The Raster works fine in onscreen-mode.

But can I capture the whole thing as an image (to save it into jpg/avi files)? I've read that I have to use an offscreen Canvas3D for that... Or?

... and then I have the update-problem again.
Offline Jeff

JGO Coder




Got any cats?


« Reply #4 - Posted 2005-04-27 20:39:19 »

Quote

Don't know if "matte" covers it by 100%: I'm not only trying to put some 2D-image in the background of a scene, but important is the possibility to occlude the 3d-objects partly. I'll put an example here:


Yup. In film and video this is called a Matte Smiley

If you change the occluded area on a frame by frame basis, its called a "traveling matte".

Smiley

JK

Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Offline Jeff

JGO Coder




Got any cats?


« Reply #5 - Posted 2005-04-27 20:41:03 »

Hm. Ive never done any off-screen rendering msyelf so I dont' have an answer  on that off the top of my head.

BUT  I'll ask Kevin or Paul to look at your post next time Im in the office and maybe they can either tell you what to try or tel lyo uwhat mre info they need to tel lyou whats wrong...

Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Offline Konterfei

Senior Newbie





« Reply #6 - Posted 2005-04-28 07:27:14 »

ok, thanks so far. I'll play around with the new discovered Raster for now....

I'm puzzled by the constructor of Raster offering a DepthComponent as well. Can I define a z-value for each pixel of my raster, which will affect occlusion? Would be great!! Cheesy
But I'm not sure: the API only mentions the parameter of the DepthComponent with the sentence

Quote
a Raster may be used to readback pixel data (color and/or z-buffer) from the frame buffer in immediate mode.

So this sounds more like only getting the z-buffer but without influencing it...

Best greetings,
Konterfei.
Offline Konterfei

Senior Newbie





« Reply #7 - Posted 2005-04-29 15:08:10 »

OK. I'm gone for the long weekend (Monday is holiday in spain).

If you can ask about rasters & offscreen I would be deep in your debt. ;-)

Cu.
Offline Jeff

JGO Coder




Got any cats?


« Reply #8 - Posted 2005-04-29 21:55:34 »

Yup. I asked Kevin today. hes going to look at your post and either post a response or emai lme a reaspons and ill post it.

Have a great weekend!

JK

Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Offline kcr

Senior Newbie





« Reply #9 - Posted 2005-04-29 23:45:20 »

Quote
If you can ask about rasters & offscreen I would be deep in your debt. ;-)


Depth rasters can be used for drawing and/or readback. It does give you a separate Z value for each pixel, but you can't currently use a Raster with both RASTER_DEPTH and RASTER_COLOR at the same time (OpenGL doesn't support that directly so it was never implemented). This means you can't really have a separate Z per pixel while you are drawing the raster. As long as you only have one of these mattes, and draw it before everything else in the scene, it shouldn't matter. I recommend that you try to use two raster objects, one with RASTER_DEPTH and the other with RASTER_COLOR. Draw the depth raster first. Draw the color-only raster second, with depth test and depth write enable disabled. Draw all 3D objects after this and they should be correctly Z-buffered with the image.

For Canvas3D capture, it depends on what you need to do. If all you want is to take a snapshot of the current window at the actual resolution, and if you aren't worried about other windows occluding the Canvas3D, then you don't need offscreen rendering. Just read the contents of the window from the postSwap method, and you have your image. If you want to render it at a higher resolution than the current window/screen will allow, or if you don't want the problem of occluding windows, you will need to use offscreen. Take a look at the PrintCanvas3D example program (you can get it from the https://j3d-examples.dev.java.net project).

-- Kevin
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Konterfei

Senior Newbie





« Reply #10 - Posted 2005-05-02 14:58:30 »

Thanks a lot you two. Just reading at home (I know I shouldn't). :-)

Will try it tomorrow in the office and will let you know if it works out. Kevin's information should be sufficient. Sounds promising!

CU.
Offline Konterfei

Senior Newbie





« Reply #11 - Posted 2005-05-04 08:14:58 »

Hey again.

I tried it out, but so far I get the same result as Mark McKay (just found his posting in the Java3D interest list yesterday) in January, 21th 2004:
http://archives.java.sun.com/cgi-bin/wa?A2=ind0401&L=java3d-interest&F=&S=&P=15470

My code is quite similar. As I understand it I don't need the RASTER_COLOR Raster for testing purpose (although I tried that as well): I have a Raster using RASTER_DEPTH but the pixels are all flat at the plane attached to the raster's origin... The moving 3d objects get occluded as soon as they move behind the x-y-plane lying at z-value 0.0f... I expected to see the 3D objects and sometimes (when they move backwards) to see black spots/areas on them -- where the raster has depthvalue 0.0f. (0.0f in the depthComponent not being the 3d space position but the closest z-value to the camera, right?).

I used DepthComponentFloat and my depthmask just had values 1.0f and 0.0f... But it does not influence anything.

Without the background RASTER_COLOR the important part looks like this:

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  
42  
BranchGroup objRoot = new BranchGroup();
OrderedGroup orderGrp = new OrderedGroup();


      // this just loads an image into BufferedImage, working fine
     // the image only has black and white areas for defining depthmask
     BufferedImage biMaskTemp =
            new ImageConverter().readIntoBufferedImage(frame_mask,this);

      // DepthCompononentFloatPlus extends DCFloat by just allowing
     // the additional definition of the depth data directly from the BufferedImage
     m_depthMask =
            new DepthComponentFloatPlus(frame_width, frame_height, biMaskTemp);
            
      // holds the image we want to draw, actually we DON'T want to draw it,
     // since this  time we only want to set z-values
     ImageComponent2D img = loadSomeImage();

      // origin of raster lies at z-value 0.0f!
     // m_depthMask is frame_width*frame_height big and has only 1.0fs and 0.0fs inside
     raster = new Raster(new Point3f(-1.0f, 0.75f, 0.0f),
                      Raster.RASTER_DEPTH, 0, 0, frame_width, frame_height, img, m_depthMask);
      raster.setClipMode(Raster.CLIP_IMAGE);
      

      Appearance app = createAppearance();
      RenderingAttributes ra = new RenderingAttributes();
      ra.setDepthBufferEnable(true);
      ra.setDepthBufferWriteEnable(true);
      app.setRenderingAttributes(ra);

      Shape3D shapeI = new Shape3D(raster, app);
      shapeI.setCapability(Shape3D.ENABLE_PICK_REPORTING);
      shapeI.setBounds(new BoundingSphere(new Point3d(), 0.5));
      transGroupRaster = new TransformGroup();
      transGroupRaster.addChild(shapeI);

orderGrp.addChild(transGroupRaster);

// ... draw 3d geometry and add to orderGrp ...

objRoot.addChild(orderGrp);



Does this code help? Did I get the idea right?
Perhaps you (or someone else) has a hint...

Greetings,
Konterfei.
Offline Konterfei

Senior Newbie





« Reply #12 - Posted 2005-05-09 12:38:31 »

Hey there.

Any news? I unfortunatelly didn't get it to work to change the z-value of single raster pixels... they always stick to the origin of the raster... hmmm.

Greetings,
Konterfei.
Offline Jeff

JGO Coder




Got any cats?


« Reply #13 - Posted 2005-05-09 20:12:15 »

Sorry been real busy. Ill ask Kevin and/or paul to take another peek at this this week.

JK

Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Pages: [1]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

xsi3rr4x (42 views)
2014-04-15 18:08:23

BurntPizza (38 views)
2014-04-15 03:46:01

UprightPath (54 views)
2014-04-14 17:39:50

UprightPath (36 views)
2014-04-14 17:35:47

Porlus (53 views)
2014-04-14 15:48:38

tom_mai78101 (75 views)
2014-04-10 04:04:31

BurntPizza (134 views)
2014-04-08 23:06:04

tom_mai78101 (234 views)
2014-04-05 13:34:39

trollwarrior1 (195 views)
2014-04-04 12:06:45

CJLetsGame (203 views)
2014-04-01 02:16:10
List of Learning Resources
by SHC
2014-04-18 03:17:39

List of Learning Resources
by Longarmx
2014-04-08 03:14:44

Good Examples
by matheus23
2014-04-05 13:51:37

Good Examples
by Grunnt
2014-04-03 15:48:46

Good Examples
by Grunnt
2014-04-03 15:48:37

Good Examples
by matheus23
2014-04-01 18:40:51

Good Examples
by matheus23
2014-04-01 18:40:34

Anonymous/Local/Inner class gotchas
by Roquen
2014-03-11 15:22:30
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!