Quake3 level loader - Benchmarks

[Amos Wenger]

Um, so then we'd have to report your new stuff to J3D and jME to be able to do a valid cross test?

Sure Qudus stuff should be ported again.
But still wait until he implement Shape3DList and RenderBinCache optimizations... 

[renanse]

Loading up the new BSPLoaderTest I see issues with the transparent textures... is that a known issue in general (thought the transparent issue was just an abberation with the old test)?

[Marvin Fröhlich]

I deleted the com.xithtk packages. Your version now rules, Marvin.

  Thanks.

Sure Qudus stuff should be ported again.
But still wait until he implement Shape3DList and RenderBinCache optimizations... 

No please don't wait until then. The RenderBinCache thing won't hit the loader implementation and Shape3DList is a feature, the others don't even have (please correct me in case), so it won't be necessary (possible) to be ported. And Shale3DList won't hit the loader implementation too deeply (I guess about one or two additional lines). And at least I'm interested in the test results before these optimization .

Loading up the new BSPLoaderTest I see issues with the transparent textures... is that a known issue in general (thought the transparent issue was just an abberation with the old test)?

Well, I thought the textures can't be transparent, since they're loaded from jpegs, which don't support alpha channels. Isn't this the cause?

Marvin

Games published by our own members! Check 'em out!

[renanse]

If the flame textures are jpgs (never looked all that close) then I guess it is using a color based transparency (ie. black = transparent).  It's transparent in the other two implementations.  Probably just an invalid improper texture setting in your loading code.

[renanse]

Looking closer, you have the right values set.  Perhaps the flame quads need to be placed in some kind of transparent queue so they are rendered after the walls to get the blending working?

[Marvin Fröhlich]

Looking closer, you have the right values set.  Perhaps the flame quads need to be placed in some kind of transparent queue so they are rendered after the walls to get the blending working?

Let's just ignore this thing until I've optimized the render code. Maybe it is fixed with it .

[Marvin Fröhlich]

Solved the problem with the CameraFlight interpolation. I miscalculated the interpolation factor. Now it is absolutely smooth. And I've switched from Matrix4f as an interpolation point to a pair of Quat4f and Vector3f, which makes the camflight files smaller .

Marvin

[renanse]

One request -- can you remove all the UI elements (fps counter and reticle) from the test, I'm afraid that might artificially lower your  frame rate as compared to other engines.

[Marvin Fröhlich]

One request -- can you remove all the UI elements (fps counter and reticle) from the test, I'm afraid that might artificially lower your  frame rate as compared to other engines.

That's only a matter of changing one line. So no problem .

Just comment out the line

1	createHUD(env, new RenderPassConfig(RenderPassConfig.PARALLEL_PROJECTION), canvas, demoFolder);

And the HUD stuff will disappear (and won't be known of by the renderer).

Marvin

[renanse]

Cool, was pretty easy to port the CameraFlight class to work with jME and run your pre-recorded flight.  I've also ported the class to my copy of the Q3 Java3D test and for kicks I've also ported it to the old version of the Xith3D test as a means of measuring any difference between old and new xith tests.

One thing I noticed is the texture filtering and the torch transparencies are much much nicer on the older version of the xith q3 test.  The first thing I was able to correct in BSPConverter.convertTextures(String[]) like so:

1   2   3   4   5   6   7

if ((textures[i] == TextureLoader.getInstance().getDefectTexture())/* && (Xith3DDefaults.getLocalDebug(BSPLoader.DEBUG))*/)             {                 System.err.println( "texture not found: " + textureNames[i] + " (.jpg / .tga)" );             } else { //added                textures[i].setMagFilter(Texture.BASE_LEVEL_LINEAR);  // added                textures[i].setMinFilter(Texture.MULTI_LEVEL_LINEAR);  // added                            }

For the transparency issue, I could not find anything different between how the old xith and new xith tests set up transparency, so I'm guessing it's due to a change in the engine itself.  Have you changed how render bins and sorting work by any chance?

FYI: One other thing I noticed is that playback of the camera flight has several of intermittent flashes on xith...  I saw something similar in jME and switched from Quats to Matrix3f and that fixed it.

Btw, my machine at home is spec'd as follows:  P4 3.2 GHz with 2.75 GB of RAM and a Radeon X850 XT Platinum running WinXP and Java 1.5.0_07

In any case, without further ado, the flight numbers and memory usage.  I've let it run through 6 fly throughs and taken the readings at the end of the each (memory reading were done on a seperate run using yourkit and they were taken after the 3rd lap had finished and printed out the average time.)  Note also that no command line args were passed to the java process, so default memory caps, etc.

Xith3D old version:
    flight average times: 112.6, 113.8, 114.2, 113.4, 111.4, 113.1
    heap (allocated/used): 52.4 MB / 35MB (used more memory than the newer version, but used memory stays nice and flat.)
    non-heap (allocated/used): 11.2 MB / 11.1 MB

Xith3D new version (HUD off): 
    flight average times: 163.2, 163.5, 143.3, 121.2, 112.0, 98.9
    heap (allocated/used): 27.4 MB / btwn. 17MB and full usage (used memory was all over the place causing lots of small gcs, as it progressed, gc caused the framerate to drop as you can see.)
    non-heap (allocated/used): 12.1 MB / 12MB

jMonkeyEngine: 
    flight average times: 224.4, 226.8, 226.3, 226.5, 226.4, 224.4
    heap (allocated/used): 30.1MB / 17.5MB
    non-heap (allocated/used): 9.4MB / 8MB

Java3d:
    flight average times:  198.8, 199.1, 195.5, 194.8, 194.7, 194.3
    heap (allocated/used): 44.8 MB / 36 MB
    non-heap (allocated/used): 11MB / 10.8MB


So, on one hand, kudos on getting initial frame rates up and memory usage down.  On the flip side, object creation is so bad it really eats frame rate as time progresses and the flight plays, especially around the 3rd flight and beyond.  Also, it would be nice if the transparency was working on the new version so we could definitely be comparing apples to apples.

I'll wait to do webstarts and all of that until you do whatever optimizations you were talking about.  (Man I am so sick of seeing that same quake level!  )

edit: fyi, in case it wasn't apparent above, "flight average times" is the average FPS following a prerecorded flight plan.

edit: as requested, I've attached the flight path used (it's just what I found in CVS) I had to rename it from .cflt to .txt

Games published by our own members! Check 'em out!

[Marvin Fröhlich]

I would really like to see your ported versions of the CameraFlight class. Could you maybe attach them?

[renanse]

The port of the class itself was simply changing what is passed into update.  I'm at work now (code is at home), but iirc, J3D I passed in a Transform3D, old Xith I passed in a View and jME I passed in a TransformMatrix.  The newer xith is straight from cvs.  The rest of the work is in the individual test classes to make use of the CameraFlight class - pretty straightforward, just applying the passed in object to the camera and the eye point for the occlusion stuff.  Most of the time I spent last night was babysitting the running of the tests!  When you finalize your new xith3d implementation (or if you take longer than this weekend) I'll post all the sources up when I make webstarts.

[Amos Wenger]

I'm gonna investigate deeper with YourKit and : find the hotspots, find the garbage creation places.

[Amos Wenger]

Flame textures are black-background jpegs, the old texture loader probably had a way to handle that.

[renanse]

No, all of the implementations use color blending ONE-ONE to blend the black into the background.  It's just being rendered out of order.

edit:  Please take note that ALL of the implementations use the black background texture AS-IS. 

[Amos Wenger]

No, all of the implementations use color blending ONE-ONE to blend the black into the background.  It's just being rendered out of order.

edit:  Please take note that ALL of the implementations use the black background texture AS-IS. 

Yeah, yeah that's what I just figured out right after I posted.

[Marvin Fröhlich]

btw. Great job for the benchmarks

[renanse]

Thanks.

[Marvin Fröhlich]

I switched to Matrix3f instead of Quad4f in the CameraFlight classes, too and the hicks are gone

Please commit this change to you ports.

Marvin

[Marvin Fröhlich]

@renanse: Please attach the CameraFlight file, you were using, to the test results post.

[renanse]

Attached, although it's just the flight you had checked into cvs.

[renanse]

FYI, today at work I discovered a couple things we were doing more often then needed in our scenegraph related to static objects and also in our texture state system which, when corrected, had a fairly dramatic effect on jme's showing in this test.  The average fps I'm getting on the flythrough is now around 310 or so!

[renanse]

Btw, with the gc fixes to the new xith test I talked about making in the other thread, the new xith is getting about 150 on the new path (the latest in xith-tk), vs 146 before the gc fixes.  Before you panic, the 310 I mentioned in the previous post is using the old flight path, which was less intense.

[Amos Wenger]

I discovered a couple things we were doing more often then needed in our scenegraph related to static objects and also in our texture state system which, when corrected, had a fairly dramatic effect on jme's showing in this test

What were these ? All optimizations interest me.

[Amos Wenger]

hum... Marvin, did you change the flight path ?

[Marvin Fröhlich]

hum... Marvin, did you change the flight path ?

Yes, sorry. I switched from Quad4f to Matrix3f to get rid of these lazy hicks. You can download the old path from remanses test result post and convert the quad4f lines to Matrix3f ones.

Read a line like this:

x y z w   <--- this is the first line of each block and contains a Quat4f
x y z        <---- this is the second line and contains a Vector3f
d           <--- the third line contains the delta time
            <------ blank line
x y z w   <--- begin of next block

read the three lines and convert the the Quad4f to a Matrix3f. Then write the block into the output file:

m00 m01 m02   <--- this is the first line of each block and contains the first row of a Matrix3f
m10 m11 m12   <--- this is the second line of each block and contains the second row of a Matrix3f
m20 m21 m22   <--- this is the third line of each block and contains the third row of a Matrix3f
x y z        <---- this is the second line and contains a Vector3f
d           <--- the third line contains the delta time
            <------ blank line
m00 m01 m02   <--- begin of next block

This way you can convert the old path to the new file format. I would do that for you, if I could compile the source, which I cannot do at the moment, since I'm currently doing the optimizations .

Marvin

[Amos Wenger]

OK ok I'm not gonna run the old path one more time but phew I think "optimizations" had made the FPS go down..

Apparently the vecmath_kh library isn't any better.. I don't get perf boost when I use it.

[Marvin Fröhlich]

OK ok I'm not gonna run the old path one more time but phew I think "optimizations" had made the FPS go down..

Apparently the vecmath_kh library isn't any better.. I don't get perf boost when I use it.

What about the GC concerning Iterators? Has it become better? I converted several advanced-loops to classic ones. This should at least have effected this kind of GC.

[Marvin Fröhlich]

btw. When you've converted the "old" flight path please give it a good name like "benchmark-2006-09-29.cflt" and commit it to CVS. Additionally please PM remanse, so that he can reattach it to the test results post. OK?

Marvin

[c_lilian]

OK ok I'm not gonna run the old path one more time but phew I think "optimizations" had made the FPS go down..

Apparently the vecmath_kh library isn't any better.. I don't get perf boost when I use it.

Perf boost isn't important in that context... what matter is reducing full GC pauses as they are visible (and annoying) for the player...

Profiling and cleaning up vecmaths was my first task when I selected Xith for my game. now I have full gcs only on level loading, which isn't annoying at all

Well, however, I wouldn't mind gaining 50% frame rate : that would give much time for everything else (game logic and music especially)

Lilian