Java-Gaming.org Hi !
Featured games (83)
games approved by the League of Dukes
Games in Showcase (524)
Games in Android Showcase (127)
games submitted by our members
Games in WIP (592)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  LWJGL/JavaFX Integration  (Read 13556 times)
0 Members and 1 Guest are viewing this topic.
Offline Spasi
« Posted 2012-11-13 17:41:05 »

Links:

- Source (Github repo, LWJGL license)
- JavaFX applet demo (+webstart link)
- JWS package (to run locally if you have trouble with the applet above)
- Windows installer (built with JavaFX native packaging)

Minimum requirements: Java 7 with JavaFX 2.2+, OpenGL 1.5+ with support for pixel buffer objects and framebuffer objects.

edit: It currently does not work on MacOS X.

Screenshot:



What's going on in the demo?

1) An LWJGL 3D scene is rendered to an offscreen framebuffer and then displayed inside a JavaFX node. The integration is lightweight and you can have any other JavaFX node on top of the 3D scene.

2) A JavaFX node is rendered to an offscreen image, copied to a GL texture and then displayed inside the 3D scene. In this demo, the node happens to be a WebView that renders java-gaming.org. You can interact with the WebView and the texture will update accordingly.

Please report any problems (won't start, crashes, bad performance) and feel free to check out the source code. I've tried to make it general enough so that it can be used in windowing toolkits other than JavaFX (e.g. with AWT).

If you try the windows installer and run into problems, try running the executable from the command-line with /Debug.
Offline CodeBunny

Senior Devvie


Medals: 4
Projects: 3



« Reply #1 - Posted 2012-11-13 17:59:40 »

It runs (windows 8 on samsung series 5 laptop), but it took a while to start up and also only runs at 20-30 FPS. Is this on purpose?
Offline Spasi
« Reply #2 - Posted 2012-11-13 18:08:42 »

Performance depends on how efficient the GPU and driver are at asynchronous PBO transfers. There are also several ways to implement the transfers and each GPU vendor favors a different one. The current implementation has only been optimized for AMD GPUs, so I'm actually hoping that someone else will contribute an optimized path for NV and/or Intel.

What's the GPU in that laptop?
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline CodeBunny

Senior Devvie


Medals: 4
Projects: 3



« Reply #3 - Posted 2012-11-13 18:10:55 »

I believe it's an integrated intel card.
Offline kappa
« League of Dukes »

JGO Kernel


Medals: 78
Projects: 15


★★★★★


« Reply #4 - Posted 2012-11-13 20:06:45 »

something seems borked on the host of the applet/jws page, can't download the jnlp or run applet.

It did run earlier though when I testing on an old computer, however it failed to run there as the PBO support was broken (though the driver claimed it was supported). Just an old ATI card with old broken drivers.
Offline Danny02
« Reply #5 - Posted 2012-11-13 23:33:18 »

nice, but really slow here. I get only 9fps on my laptop with fedora running. Have a Nvidia 550 but think it is deactivated and uses probably the intel one
Offline gouessej
« Reply #6 - Posted 2012-11-14 01:11:43 »

Hi

Nice job but do you think we could find a solution to do it without PBO? I already investigated a lot but I found no safe and satisfying solution on the long term.

Offline Spasi
« Reply #7 - Posted 2012-11-14 10:32:15 »

For JavaFX in particular, I would wait for it to be open-sourced first. Last I heard the target for the OS release is February. Then we can properly explore how we could hack into their GL pipeline, or even DX one with WGL_NV_DX_interop.

This solution is meant to be a lightweight alternative, if everything else fails. Assuming it can run fast enough of course. I'll test on Intel later today.
Offline Danny02
« Reply #8 - Posted 2012-11-14 15:52:13 »

just read about a quite handy GL extension which could speed up things on intel cards quite dramatically:
INTEL_map_texture
Offline ra4king

JGO Kernel


Medals: 355
Projects: 3
Exp: 5 years


I'm the King!


« Reply #9 - Posted 2012-11-15 04:25:34 »

Spasi, please have my babies. This is awesome!

Runs great at 235 FPS with triple buffering and 32x MSAA Smiley

Also the installer works beautifully! I was so surprised by how seamless and easy it was to install and uninstall.

This is all on Windows 8 x64 with a GTX 580 Smiley

Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Spasi
« Reply #10 - Posted 2012-11-17 22:03:02 »

Update:

- Organized the source so that it's clear what is library and what is demo code.
- Added choice boxes with alternative implementations.
- Replaced CountDownLatches with Semaphores (reduces garbage).
- Misc optimizations. The default implementation now runs at ~160 fps on a desktop Sandy Bridge IGP, ~1550 fps on my Radeon.
- Added an implementation that utilizes the INTEL_map_texture extension (thanks Danny02!). Performance goes up to ~275 fps on my i5.
- Added untested implementation for Nvidia GPUs. It does ReadPixels on a device allocated buffer then copies its contents to a host allocated buffer using the ARB_copy_buffer extension. This is supposed to double the readback bandwidth on consumer NV cards.

Details:

Intel. Well, the drivers are a pain. There's no way to perform an asynchronous read-back to a PBO, everything blocks. INTEL_map_texture allows for CPU-cached textures with linear layout, but performance is horrible if they're accessed by the GPU. After a lot of pain, I ended up using an extra buffer for GPU access and BlitFramebuffer for asynchronously copying from the tiled texture to the linear one. Anything else and it would block.

NV. I could only test on an old 7900 GS and performance is horrible, slower than the i5 IGP. The ARB_copy_buffer implementation is slower than the default too. I would really appreciate it if someone could test and report numbers on a more recent model.

Java 8. The setPixels/writePixels calls have worse performance (around 50%) on the latest JavaFX builds, so you should test on Java 7. I also had to update the installer above, but it's buggy on 7, you'll have to edit the shortcut and add a /Debug argument.

Performance. Well, it's almost usable. It's a horrible horrible hack, but with buffering and multi-threading (LWJGL render thread, JFX application thread, JFX render thread) there's enough overlap to hide the readback/upload overhead and have something playable. Keep in mind though that without Display.sync throttling you can swamp the JFX threads with the image update calls and cause UI freezes. On my setup it happens if I maximize the window to 1080p and disable syncing.
Offline concerto49

Junior Devvie





« Reply #11 - Posted 2012-11-17 22:22:11 »

Thanks. This is what we have been looking for all along. JavaFX is a great GUI tool. Will be trying this integration very soon and migrate some of our existing programs to it.

High performance, fast network, affordable price VPS - Cloud Shards
Available in Texas, New York & Los Angeles
Need a VPS Upgrade?
Offline gouessej
« Reply #12 - Posted 2012-11-18 01:18:26 »

NV. I could only test on an old 7900 GS and performance is horrible, slower than the i5 IGP. The ARB_copy_buffer implementation is slower than the default too. I would really appreciate it if someone could test and report numbers on a more recent model.
I can test on my Quadro NVS 285 but I'm not sure it will be faster.

Offline ra4king

JGO Kernel


Medals: 355
Projects: 3
Exp: 5 years


I'm the King!


« Reply #13 - Posted 2012-11-18 01:27:40 »

Can confirm the bugginess on Java 7 in the new build. I couldn't run it directly anymore, I had to use /Debug.

However with no sync it still runs at ~230 FPS

Offline HeroesGraveDev

JGO Kernel


Medals: 296
Projects: 11
Exp: 3 years


┬─┬ノ(ಠ_ಠノ)(╯°□°)╯︵ ┻━┻


« Reply #14 - Posted 2012-11-18 01:32:21 »

AWESOME!

(Unfortunately useless to me because I don't know how to use/don't use JavaFX)

Offline Spasi
« Reply #15 - Posted 2012-11-18 12:45:44 »

I can test on my Quadro NVS 285 but I'm not sure it will be faster.

I wouldn't be surprised if it is. One of the advantages of Quadro over the consumer line is greatly improved readback performance.

However with no sync it still runs at ~230 FPS

Could you please test again with no multisampling? I'm interested in the raw readback/upload performance without anything heavy skewing the fps. Also, make sure you don't resize the window or the panels, for better comparison with my numbers. Does the ARB_copy_buffer implementation perform better or worse?
Offline deathpat
« Reply #16 - Posted 2012-11-18 13:05:31 »

I just tested with my nvidia GPU ( GTX 560M ), running in the applet with a screen res of 1920 * 1080 and no antialiasing.
I have 270 FPS with asynchronous PBO and 450 FPS with ARB_copy_buffer ... so it definitely performs better on my PC with ARB_copy_buffer Smiley

work in progress : D A E D A L U S
Offline Roquen
« Reply #17 - Posted 2012-11-18 13:11:23 »

I have an old notebook with dual GeForce 9400s.  Window's launcher, default size = ~60Hz.  8 tap AA = ~30-40Hz.  (vsync enabled on both...I guess I should have disabled...bad me.  Original drop)
Offline Riven
« League of Dukes »

« JGO Overlord »


Medals: 833
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #18 - Posted 2012-11-18 13:41:50 »

Framerate is (obviously) heavily impacted by the number of buffers, as the uploads are async. I see linear growth in fps with increasing the number of buffers.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline deathpat
« Reply #19 - Posted 2012-11-18 13:43:21 »

I tested again with the windows installer, keeping the default window size:
I got 450 FPS with asynchronous PBO and 710 FPS with ARB_copy_buffer

work in progress : D A E D A L U S
Offline kappa
« League of Dukes »

JGO Kernel


Medals: 78
Projects: 15


★★★★★


« Reply #20 - Posted 2012-11-18 14:05:06 »

Here are my results from a laptop with a Nvidia Gefore 8400GS

All tests run with vsync turned off and MSAA x 1, Asynchronous PBO for both gives - No buffering runs at 30fps and double and triple buffer runs at 60fps. Similar results for the other options.

Is it possible you could also add a second JNLP to the download bundle with those JavaFX parameters that unlock its 60fps cap?

Also maybe adding another option demonstrating glReadPixels would be nice for performance comparison (and if set as default would allow demo to run on gfx cards which don't support PBO's properly as glReadPixels should work just about everywhere).
Offline ra4king

JGO Kernel


Medals: 355
Projects: 3
Exp: 5 years


I'm the King!


« Reply #21 - Posted 2012-11-18 20:54:55 »

GTX 580 with MSAA x1.

No Buffering:
Texture V | Render >Asynchronous PBOARB_copy_buffer
Asynchronous PBO480 FPS770-780 FPS
ARB_map_buffer_range480 FPS770-780 FPS

Double Buffering:
Texture V | Render >Asynchronous PBOARB_copy_buffer
Asynchronous PBO590-595 FPS1080-1085 FPS
ARB_map_buffer_range590-595 FPS1080-1085 FPS

Triple Buffering: same numbers as Double Buffering.

Offline Krux

Junior Newbie





« Reply #22 - Posted 2013-04-21 22:03:34 »

Hello

I found this with google and tried it, and I have to say I really like it, because it works. But on my Intel card it is slow, but that video card sucks anyway. I think the performance could be better if you exploit the fact that GUI elements update rather seldom compared to the 3D Scene. So instead of moving the 3D Scene into an ImageView, you might render all GUI elements that have changed into an offscreen texture and draw thaw with LWJGL manualy. This might be done, if you can replace or modyfy the scene Renderer from JavaFX with one of your own.

I tried to do something similar with swing, but I miserably failed. I could copy the swing component to an OpenGL texture, that was no problem, but I was not able to do it on change events. Also all interacitivity was lost if I wanted to bring swing elements into the 3D space. I was not able to find a hook to replace the renderer or to replace the user input. If you can do this in javafx, I will like you very much Wink
Offline Spasi
« Reply #23 - Posted 2013-04-22 11:28:15 »

Hi Krux,

First of all the integration in this demo is 2-way; you have both JavaFX nodes rendered in a 3D scene and the 3D scene itself rendered as a JavaFX node. This is the worst possible scenario in terms of performance and probably unrealistic in a real application. If you only need JavaFX content in an LWJGL window that's possible of course.

You're correct that GUI elements don't need to be redrawn as frequently. Again, I chose a WebView on purpose because I needed to test the most demanding node possible to see if this approach is viable. I do in fact use a texture to cache the JavaFX rendering, but in this demo it's being updated whenever I detect a change in the HTML content *or* every 4 frames (at 60fps), for carets etc. In a real app you could reduce the update rate considerably.

JavaFX does have a fully working robot implementation, but it's currently undocumented. You can use com.sun.glass.ui.Application.GetApplication().createRobot() to get an instance and start firing events. I'm afraid I can't help you with transforming 3D scene coordinates to 2D node coordinates, that's scene/engine specific, but it should be easy as long as you know the transformation matrix of the surface that contains the JavaFX texture.
Offline Krux

Junior Newbie





« Reply #24 - Posted 2013-04-22 12:45:51 »

thanks for the info, no problem with math here, but I am still not shure how to lock the mouse to do some first person stuff with mouse, keyboard and for those, who prefer it, even gamepad. I think I am going to explore your sourcecode a little bit more.
Offline Krux

Junior Newbie





« Reply #25 - Posted 2013-04-25 21:54:52 »

Ok, I have integrated your code into my project, and I really like it. The problem is, that my cpu usage is pretty high. I have an Itel Gpu and Linux, which gives me software javafx. But I have the impression, that my render Screen in rendered on the GPU, pulled back into main memory and merged with all the other JavaFx screens, and then pushed back into the game. My question is, if I had a better GPU with better drivers will my framebuffer remain on the GPU?

here an image if you care. The Button is JavaFx  Grin
Offline Spasi
« Reply #26 - Posted 2013-04-26 13:18:57 »

This is normal and a better GPU won't eliminate the framebuffer read-back. The point of this demo is to enable integration with zero changes to JavaFX and to showcase what such an integration could be used for. The overhead you're seeing is probably too much to use in production, especially on low-end hardware.

Possible changes to JavaFX that would make it more viable:

- Better PixelReader/Writer API to allow for zero-copy read-back loops. This would reduce memory/CPU pressure, but won't eliminate the read-back (+ upload).
- Exposure of the internal GL context to enable GPU copies via WGL/GLX make_current_read or NV_copy_image.
- Exposure of the internal GL context and also of any FBOs or textures used for JavaFX rendering. These could be used directly without any copies via context object sharing.

The last 2 would also require rendering synchronization APIs and a flag that forces OpenGL rendering, even on Windows.
Offline Krux

Junior Newbie





« Reply #27 - Posted 2013-04-27 09:07:15 »

Thanks for the info so far. But I think there can be done something with this overhead. Currently I have these ideas:

I could make the JavaFx pipeline optional. This means as long as my mouse is grabbed by Lwjgl I would render without framebuffer directly with Display.swapBuffers. But as soon as the mouse is released (by pressing Esc) my screen will render into a framebuffer once and the GUI will shows up. But I am not shure if this is as easy as it sounds, because this requires that the JavaFx window to turn into an LWJGL window and vice versa.

A different approach to reduce the render overhead would be to move textures only in one direction. So instead of rendering the scene into a framebuffer, the GUI could be rendered into a window sized rgba texture. Then I can merge the GUI texture with the scene rendering within my rendering loop. This could also benefit from the fact, that (as I've mentioned earlier) GUI updates are commonly less frequent than scene updates. This requires offscreen rendering with javafx and translation of LWJGL events into JavaFx events.
Pages: [1]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

toopeicgaming1999 (57 views)
2014-11-26 15:22:04

toopeicgaming1999 (50 views)
2014-11-26 15:20:36

toopeicgaming1999 (10 views)
2014-11-26 15:20:08

SHC (24 views)
2014-11-25 12:00:59

SHC (24 views)
2014-11-25 11:53:45

Norakomi (26 views)
2014-11-25 11:26:43

Gibbo3771 (24 views)
2014-11-24 19:59:16

trollwarrior1 (36 views)
2014-11-22 12:13:56

xFryIx (75 views)
2014-11-13 12:34:49

digdugdiggy (52 views)
2014-11-12 21:11:50
Understanding relations between setOrigin, setScale and setPosition in libGdx
by mbabuskov
2014-10-09 22:35:00

Definite guide to supporting multiple device resolutions on Android (2014)
by mbabuskov
2014-10-02 22:36:02

List of Learning Resources
by Longor1996
2014-08-16 10:40:00

List of Learning Resources
by SilverTiger
2014-08-05 19:33:27

Resources for WIP games
by CogWheelz
2014-08-01 16:20:17

Resources for WIP games
by CogWheelz
2014-08-01 16:19:50

List of Learning Resources
by SilverTiger
2014-07-31 16:29:50

List of Learning Resources
by SilverTiger
2014-07-31 16:26:06
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!