Java-Gaming.org    
Featured games (81)
games approved by the League of Dukes
Games in Showcase (489)
Games in Android Showcase (112)
games submitted by our members
Games in WIP (553)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1] 2
  ignore  |  Print  
  JPCT vs JME vs Xith3D vs ... benchmark  (Read 16089 times)
0 Members and 1 Guest are viewing this topic.
Offline zammbi

JGO Coder


Medals: 4



« Posted 2009-04-09 10:18:04 »

Thought this might come of some interest of people who don't read the JPCT forum.
A basic benchmark(Maybe someone could double check the results), I was shocked of the winner. Original thread: http://www.jpct.net/forum2/topics/lwjgl-jni-similiar-to-swt-jni/809/45/view.html

Latest tests:

Quote
Now for the tank...i've converted it to OBJ format first to ease testing. So i could reuse my existing test applications without changing anything except the filename and maybe the scaling.

I wasn't able to load it with Java3D (it bombed out with a nullpointer exception when loading the materials), so i skipped this. The numbers for JME are taken with the build-in stats tool and backed up by FRAPS (had to write the fps to a file, because FRAPS' onscreen display stops at 999). I've added 5% to the JME score to compensate for the stats tool's display. This seems to be reasonable judging from the results with the car. Of course, lockMeshes() has been used.
For 3DzzD, i've tested it in IE8 and Firefox3 and got the same results. An applet isn't a very good environment to test performance IMHO but it's all i got, so...

Here are the numbers:

jPCT: 3500fps
3DzzD: 2100fps
JME: 2000fps
xith3D: 1950fps


Quote
I've managed to load the model into Java3D...i have no idea, if there is something like lockMeshes() in Java3D too, but anyway...
So on the Core2 Quad@3Ghz, ATI Radeon HD 4870, Java6, Vista Ultimate, we now have:

jPCT: 580fps
JME: 525- 530fps
xith3D: 510fps
Java3D: 120fps (with some visual glitches, but that could be a bug in the loader)

Old tests:

Quote
At first, the results of rendering that high polygon car model that paul provided...

...on a Core2 Duo@3Ghz, some lowly NVidia Quadro GPU:

jPCT: 78 fps
jME: 34 fps
xith3D: 22 fps

The same thing rendered on my main machine, a Core2 Quad@3Ghz, ATI Radeon HD 4870:

jPCT: 580 fps
xith3D: 222 fps
jME: 30 fps

And finally, the famous Quake3-level from the xith/jME-benchmark...but just loaded as a model...i haven't converted the whole benchmark thing and only loaded it into xith and jPCT on the Quad core:

xith3D:
fps view: 91 fps
level view: 69 fps

jPCT:
fps view: 1000 fps
level view: 1000 fps

Quote
The car-model loaded on a Core2Duo @ 2.66Ghz, Intel GMA 950 onboard graphics:

jPCT: 6fps
JME: 4fps
xith3D: crashed due to absence of glActiveTextureARB on that machine (which is a bogus message, because jPCT uses that too and runs fine)

The same model loaded on a Pentium 4M @2.2Ghz, Geforce 2MX Go:

jPCT: 3fps
xith3D: 1fps
JME: unknown (but feeled like 1+fps)...i wasn't able to enable the fps stats, just got a gray overlay instead

and finally on an Athlon X2 @2.2Ghz, Radeon HD 3650 AGP:

jPCT: 200fps
xith3D: 110fps
JME: 28fps


Current project - Rename and Sort
Offline gouessej
« Reply #1 - Posted 2009-04-09 11:23:20 »

Hi!

I'm highly suspicious. Can someone double-check?

Offline EgonOlsen
« Reply #2 - Posted 2009-04-09 12:01:58 »

No need to be suspicious. I haven't added a Thread.sleep(xxx) or something.

I've used this model: http://www.paulscode.com/Models/triantObj.zip

This is the jPCT-app that displays it (an older version with a small bug, but that doesn't affect performance): http://www.jpct.net/download/beta/car.exe (it's a self-extracting exe for Windows, i was lazy...)

For xith3D and jME, i've simply used the obj-loader-demos that come with the engines and replaced the model. I've used the latest stable releases for both.


Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline DzzD
« Reply #3 - Posted 2009-04-09 12:30:17 »

great, dont know really why but I am not surprised,

where can we check the car model (the first test case ?), I want to get my stats too ! Smiley

Offline EgonOlsen
« Reply #4 - Posted 2009-04-09 12:41:34 »

where can we check the car model (the first test case ?), I want to get my stats too ! Smiley
There's a link to it in my previous post.

Offline Orangy Tang

JGO Kernel


Medals: 56
Projects: 11


Monkey for a head


« Reply #5 - Posted 2009-04-09 13:02:00 »

Very interesting - would love to see the results for 3DzzD as well. With JME looking increasingly ropey jPCT and 3DzzD are looking like the prefered alternatives right now IMHO.

[ TriangularPixels.com - Play Growth Spurt, Rescue Squad and Snowman Village ] [ Rebirth - game resource library ]
Offline gouessej
« Reply #6 - Posted 2009-04-09 13:23:40 »

Very interesting - would love to see the results for 3DzzD as well. With JME looking increasingly ropey jPCT and 3DzzD are looking like the prefered alternatives right now IMHO.
Why do you say that JME is looking increasingly ropey? There are still some developments on it, some people are even preparing JME 3.0, someone else is repairing the JOGL renderer and the AWT input system (as I have not enough time to do it myself), I am still working on the portal culling and some other people are submitting other features.

Offline brackeen

Junior Member





« Reply #7 - Posted 2009-04-09 13:29:35 »

That is really impressive.

Any chance of jPCT going open source?
Offline gouessej
« Reply #8 - Posted 2009-04-09 13:33:51 »

Watch this:
http://www.jmonkeyengine.com/jmeforum/index.php?topic=10879.msg82316#msg82316


Quote
Well, you have to wonder which tests he is running (where's the code and the models, etc.?) and what versions of xith and jme, etc.  Easy to claim a benchmark when you are the engine's author and you don't give out the exact code for others to try themselves.

That said, the 91 vs 1000 Q3 test thing is not very surprising if you follow that thread along...  He added a feature to jpct that compiles everything down to just the fewest number of batches needed to render that scene.  In other words, at a very elementary level imagine the Q3 level is a big triangle soup...  go through that soup and separate out all the triangles that are using the same states.  Combine those into one big mesh.  Now instead of several thousand "bsp faces" you have a couple dozen meshes.  Your polygon chomping card loves you. Smiley

Great idea except it changes the nature of the "benchmark" since jme and xith are purposely processing it as an old fashioned BSP with visibility calcs, etc.

Then again, those numbers are kind of odd:

Quote
xith3D:
fps view: 91 fps
level view: 69 fps

jPCT:
fps view: 1000 fps
level view: 1000 fps

maybe i just don't know what "fps view" is, but on my older PC here at home (P4 2.4G, nVidia 7800) I get in the 200-500FPS range flying through the level for both Xith and jME.

edit: maybe those numbers are from altering the xith test to not use visibility sets.  That would fall in line with the numbers.

Offline Orangy Tang

JGO Kernel


Medals: 56
Projects: 11


Monkey for a head


« Reply #9 - Posted 2009-04-09 13:36:43 »

Why do you say that JME is looking increasingly ropey? There are still some developments on it, some people are even preparing JME 3.0, someone else is repairing the JOGL renderer and the AWT input system (as I have not enough time to do it myself), I am still working on the portal culling and some other people are submitting other features.

Because there's no quality control any more. I may be wrong but as far as I know:
 - main developers have gone, so there's lots of code no-one really has a deep understanding of.
 - No proper releases, just instructions to "get the latest from svn"
 - No nightly builds or automated testing
 - Allowing anyone to check in to the source repository means that there's zero guarantees that it'll actually work.
 - All the demos on the website are still JME 1.0
 - Broken links in javadoc (which I'm not even sure is recent, nor if it's 1.0 or 2.0).

It's a shame because it used to be a production quality engine, but it seems like it's being run by amateurs now.

[ TriangularPixels.com - Play Growth Spurt, Rescue Squad and Snowman Village ] [ Rebirth - game resource library ]
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline gouessej
« Reply #10 - Posted 2009-04-09 13:42:30 »

Because there's no quality control any more.
It is wrong. We have to show our source code before submitting it and it has to respect the coding conventions.

- Allowing anyone to check in to the source repository means that there's zero guarantees that it'll actually work.
It is wrong too, I had to ask for the right to check in.

- All the demos on the website are still JME 1.0
At least lots of demos are still in JME 1.0.

Offline Orangy Tang

JGO Kernel


Medals: 56
Projects: 11


Monkey for a head


« Reply #11 - Posted 2009-04-09 14:00:38 »

Manually eyeballing code is not quality control, and still leaves all of the other points unaddressed. Individually they're not insurmountable problems, but they're indications of the decay and break down of the project as a whole.

[ TriangularPixels.com - Play Growth Spurt, Rescue Squad and Snowman Village ] [ Rebirth - game resource library ]
Offline EgonOlsen
« Reply #12 - Posted 2009-04-09 14:05:25 »

I agree that the Quake3-benchmark isn't an apples-to-apples comparison. That's why i only ran it once on one machine. On the other hand, the visual outcome is the same, just faster, so it doesn't really matter to the user. The "fps-view" simply is a defined view inside the level like a player would see it. "Level view" means, that the whole level is visible. And that view gave me the results that i've posted. I would have tried a newer version of xith3d than the latest stable, but the link to it was dead.


Offline gouessej
« Reply #13 - Posted 2009-04-09 15:16:38 »

From http://www.jmonkeyengine.com/jmeforum/index.php?topic=10879.msg82325#msg82325

Quote
- The car model is really nice  grin
- It weighs in at about 1.1million triangles, so let's all be happy for Java engines showing these framerates.
- I get around 20FPS with jME just loading the car and dumping it in the scene.
- Just by doing model.lockMeshes(), that jumps to about 250FPS
- This is on a single core AMD Sempron 3400 with a 3650 AGP graphics card.

Offline zammbi

JGO Coder


Medals: 4



« Reply #14 - Posted 2009-04-09 16:04:22 »

That's a little better heh. Though would be nice to now compare it with the other 2 engines.

Current project - Rename and Sort
Offline EgonOlsen
« Reply #15 - Posted 2009-04-09 17:04:36 »

Yes, that looks better. As said, i have no clue of JME or xith3D. But i expected the demos to be written in a way that shows off the engines' full capabilities. Obviously, this isn't the case... Huh

Offline tom
« Reply #16 - Posted 2009-04-09 18:09:37 »

Very interesting - would love to see the results for 3DzzD as well. With JME looking increasingly ropey jPCT and 3DzzD are looking like the prefered alternatives right now IMHO.
Aviatrix3D is also a good alternative. Not very well known on this forum, but I think it is the best java scenegraph at the moment.

As for the jPCT benchmark. A large model should be rendered at the same speed on all engines that supports some kind of compiling. It boils down to a few display list or VBO calls. The Q3 model is a different matter. It has thousands of objects with an average of 10 triangles and around 100 unique states. Combining the objects can improve the performance. Although you don't always want to merge objects as culling the objects separately might be better. It all depends on the scene.


Offline EgonOlsen
« Reply #17 - Posted 2009-04-09 20:18:10 »

I've retested the JME-version with a call to lockMeshes() on the loaded object (tested on the Quadcore mentioned in the first post). It's @520 fps now, which is almost on par with jPCT. When compiling the object in dynamic mode on jPCT (which seems to be similar to what JME does without lockMeshes()), jPCT puts out 30fps...exactly the same as JME does. So i guess it's almost a draw now... Grin
Does anybody know if xith3D has something similar like lockMeshes() in JME? Is there some OBJ-loader in JAVA3D to include this in the benchmark?

Offline EgonOlsen
« Reply #18 - Posted 2009-04-09 20:48:24 »

I've managed to load the model into Java3D...i have no idea, if there is something like lockMeshes() in Java3D too, but anyway...
So on the Core2 Quad@3Ghz, ATI Radeon HD 4870, Java6, Vista Ultimate, we now have:

jPCT: 580fps
JME: 520fps
xith3D: 220fps
Java3D: 120fps (with some visual glitches, but that could be a bug in the loader)

Offline tom
« Reply #19 - Posted 2009-04-09 21:28:44 »

Java3D BranchGroup has a compile() method. Javadoc:

http://download.java.net/media/java3d/javadoc/1.5.0/javax/media/j3d/BranchGroup.html#compile()

Offline DzzD
« Reply #20 - Posted 2009-04-09 22:26:34 »

Ok.. I will have trouble to compete with that model, 3DzzD only handle 3ds (and 3dz ) files for now and this model have object with more than 65536 face wich make my converter unable to convert the obj to 3ds Sad ....

so here is my tech/bench demo :

http://demo.dzzd.net/View3D/TANK.html

only 80000 faces but within a fully unsigned Java 1.1 Applet Smiley, once loaded you can switch to Hardware by pressing the H keys



you can download the 3ds model and its texture here :
http://demo.dzzd.net/View3D/TANK.ZIP

the software rendering have the followig feature enabled :
- real per pixel phong shading
- diffuse texture with advanced bilinear filtering (& more... some kind of anysio) + mipmap
- more than double precision z (not a fixed zbuffer)
- edge antialiasing up to 1/256em pixels precision


the hardware rendering is :

- gouraud shading
- diffuse  texture set to high quality (but depend on your gpu setting) + mipmap
- fixed 32 bit precision z buffer

the funny or not funny ... is that on this particular model the hardare rendering is slower than the software on my AMD 1.6 + ATI X800....

EDIT:
you can go back to software rendering with S key, the framerate is printed in the java console every 200/250 frames



Offline renanse

Junior Member




Intelligence is light to a dark world.


« Reply #21 - Posted 2009-04-10 01:46:04 »

I've managed to load the model into Java3D...i have no idea, if there is something like lockMeshes() in Java3D too, but anyway...
So on the Core2 Quad@3Ghz, ATI Radeon HD 4870, Java6, Vista Ultimate, we now have:

jPCT: 580fps
JME: 520fps
xith3D: 220fps
Java3D: 120fps (with some visual glitches, but that could be a bug in the loader)

Cool, thanks for being nice enough to post updates. Smiley

I noticed in another thread you mentioned using the in-app FPS counter to measure speed.  If you are just using the model loading tests, they have debugging and a render-to-texture graphing system which is probably where some of that 60FPS difference between jME and jPCT is going to.  Of course in a real application you'd not have those things (but they are pretty to look at in samples.)  Also, I'd suggest using a standardized method of measuring FPS (like fraps).

Renanse  (ruh-NON-say)
Offline DzzD
« Reply #22 - Posted 2009-04-10 01:57:05 »

very good looking demo, I tried it on my crapy computer I got 17fps (1.6Ghz + ATIX800), and it hang for 1 or 2 seconds every 10 seconds.

if you provide JME and/or xith demo I will test them too and post results back.

Offline bobjob

JGO Knight


Medals: 10
Projects: 4


David Aaron Muhar


« Reply #23 - Posted 2009-04-10 03:07:52 »

I would like to also test straight from LWJGL, as a speedy VBO object.

I would like to test the tank, DzzD can you please post the view settings?

like perspective if any, or is it Ortho?
intial scale, position from camera and such.

and how do I view the FPS in the tank applet demo?

My Projects
Games, Webcam chat, Video screencast, PDF tools.

Javagaming.org with chat room
Offline EgonOlsen
« Reply #24 - Posted 2009-04-10 06:47:33 »

Also, I'd suggest using a standardized method of measuring FPS (like fraps).
I tried fraps, but it actually slows down things on my machine. With FRAPS enabled, jPCT is at 530 and JME at 480...(not only judging from the FRAPS values but also backed up by the internal counters) Huh

Offline renanse

Junior Member




Intelligence is light to a dark world.


« Reply #25 - Posted 2009-04-10 12:26:49 »

I tried fraps, but it actually slows down things on my machine. With FRAPS enabled, jPCT is at 530 and JME at 480...(not only judging from the FRAPS values but also backed up by the internal counters) Huh
Yeah, it does slow things down, but it should be more or less consistent anyhow. Smiley  I guess my point is that if you use that combination graph/chart/fps display in the jME tests to gather your frame counts, you're introducing stats collection, chart management and RTT overhead on the jME side so it's less than an even comparison.  You could add a line to the Test's constructor to remove all that.  Here's how you would do it on the included Obj test:
1  
2  
3  
4  
    TestObjJmeWrite() {
        super();
        System.clearProperty("jme.stats");
    }

   

Renanse  (ruh-NON-say)
Offline EgonOlsen
« Reply #26 - Posted 2009-04-10 13:25:06 »

Ok, i'll try that later and post the results.

Offline DzzD
« Reply #27 - Posted 2009-04-10 14:05:20 »

I would like to also test straight from LWJGL, as a speedy VBO object.

I would like to test the tank, DzzD can you please post the view settings?

like perspective if any, or is it Ortho?
intial scale, position from camera and such.

and how do I view the FPS in the tank applet demo?

I will post camera settings tonight, it is a standard 3ds max camera, the framerate is printed in the java console  ( or do you mean "first person shooter" or "frame per second" ? )

EDIT :
for the framerate : 3000 mean 30 (it is fps*100)

EDIT 2:
ok for the camera seems that I ve used one that is in the 3ds file named "Camera01"

the properties are :

in 3d studio max (z is up):
FOV 45°
position : 1338,-3691,1231
rotation : 72,0,20.5

in 3dzzd (y is up) :
FOV 45°
position : 1338,1231,-3691
rotation : 72,20.5,0

depending on your axis (right hand/left hand) you have to use one of this settings

Offline EgonOlsen
« Reply #28 - Posted 2009-04-10 18:07:35 »

Ok, i'll try that later and post the results.
And here are the results: It doesn't matter performance wise on my machine if i disable the stats or not when not drawing them. The performance measured with FRAPS stays the same (but is still lower than without FRAPS). When drawing the stats, there's a slight drop of 5-10 fps, i.e. maybe we can add another 5-10fps to the score of 520 for JME, which leads to 525-530 with stats disabled.

Offline EgonOlsen
« Reply #29 - Posted 2009-04-10 18:11:31 »

Ok, i've tried that...it doesn't do any good. Performance stays at 120fps.

Pages: [1] 2
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

TehJavaDev (18 views)
2014-08-28 18:26:30

CopyableCougar4 (26 views)
2014-08-22 19:31:30

atombrot (39 views)
2014-08-19 09:29:53

Tekkerue (36 views)
2014-08-16 06:45:27

Tekkerue (33 views)
2014-08-16 06:22:17

Tekkerue (22 views)
2014-08-16 06:20:21

Tekkerue (33 views)
2014-08-16 06:12:11

Rayexar (70 views)
2014-08-11 02:49:23

BurntPizza (47 views)
2014-08-09 21:09:32

BurntPizza (37 views)
2014-08-08 02:01:56
List of Learning Resources
by Longor1996
2014-08-16 10:40:00

List of Learning Resources
by SilverTiger
2014-08-05 19:33:27

Resources for WIP games
by CogWheelz
2014-08-01 16:20:17

Resources for WIP games
by CogWheelz
2014-08-01 16:19:50

List of Learning Resources
by SilverTiger
2014-07-31 16:29:50

List of Learning Resources
by SilverTiger
2014-07-31 16:26:06

List of Learning Resources
by SilverTiger
2014-07-31 11:54:12

HotSpot Options
by dleskov
2014-07-08 01:59:08
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!