Amos Wenger
|
 |
«
Posted
2006-09-29 16:51:09 » |
|
Here's an interesting report made with YourKit profiler : http://www.xith.org/profiling/GC-Q3-Report.htmlIt shows the most garbage-producing methods. Some like AbstractArray.iterator() are unavoidable but these are just signs of other problems : too much array creation or somewhat. Some other methods that worry me are the getScale() and invert() methods of Matrix4f : sooo much garbage ? How is it just possible. BSPClusterManager.setVisibility(Vector3f) could surely be improved. I'll now look into sourcecode and profile CPU times. Note : If you want an open-source YourKit license, just mail me.
|
"Once you start working on something, don't be afraid of failure and don't abandon it. People who work sincerely are the happiest"
|
|
|
Marvin Fröhlich
|
 |
«
Reply #1 - Posted
2006-09-29 18:07:23 » |
|
...Some like AbstractArray.iterator() are unavoidable...
Well, I guess most of them are. I've never thought into it and realized, that each loop over a List is a creation of an Iterator, which needs to be garbage collected. We could just loop over a List by a regular for (i = 0; i < list.size(); i++) loop. This can also be done very efficiently, and doesn't produce any garbage. Some other methods that worry me are the getScale() and invert() methods of Matrix4f : sooo much garbage ? How is it just possible.
Strange  . No idea. BSPClusterManager.setVisibility(Vector3f) could surely be improved.
Yes, for sure. Marvin
|
|
|
|
|
Games published by our own members! Check 'em out!
|
|
Marvin Fröhlich
|
 |
«
Reply #3 - Posted
2006-09-29 18:36:15 » |
|
Hi,
As of VecMath and GC, this topic was discuseed 1-1.5 years ago (I think it should be possible to find it in forum archives).
Yes, standard version of vecmath from Sun produces a lot of garbage, but there is another one version of vecmath originally developed (published) by Kenji Hiranabe (I hope I spelled his name right from my head), and that library works pretty fine and produces nearly no garbage (or even no garbage at all). Artur Besiadovski (aka abies) pointed me to this library long time ago.
This lib is 100% compatible with javax.vecmath, (even from the naming) so I only made few modifications with it and just use another jar file instead of original vecmath.jar.
You should consider using this library I think. (actually, last time I pointed to some licensing issues, but to say truth I did not dig deep enough to this topic).
Yuri
Does anyone know, where to find this library? I'll continue to search for it, but so far with no success. Marvin
|
|
|
|
cylab
|
 |
«
Reply #4 - Posted
2006-09-29 18:57:28 » |
|
http://www.objectclub.jp/download/vecmath_eRegarding the Licence: there is an overview on the vecmath page. It seems that an implementation must be able to pass suns test suite to be allowed to be redistributed. If I understand correctly, the implementation above complies to the Java3D 1.2 specification and should therefor fullfill this requirement.
|
Mathias - I Know What [you] Did Last Summer!
|
|
|
Marvin Fröhlich
|
 |
«
Reply #5 - Posted
2006-09-29 19:02:46 » |
|
So would anyone object using (or first trying) this lib?
|
|
|
|
cylab
|
 |
«
Reply #6 - Posted
2006-09-29 19:06:51 » |
|
Please try it. Since it is an inplace replacement, we could support both alternatives.
|
Mathias - I Know What [you] Did Last Summer!
|
|
|
Marvin Fröhlich
|
 |
«
Reply #7 - Posted
2006-09-29 21:22:22 » |
|
I included the alternative vecmath implementation (vecmath-kh.jar) into xith, but the GC is still bad. Amos please remeasure GC. Hopefully the vecmath GC and iterator GC should have dropped to a minimum.
Marvin
|
|
|
|
Marvin Fröhlich
|
 |
«
Reply #8 - Posted
2006-09-29 22:33:40 » |
|
Which vecmath lib do the JMonkey guys use?
|
|
|
|
renanse
Junior Devvie   Exp: 14 years
Intelligence is light to a dark world.
|
 |
«
Reply #9 - Posted
2006-09-30 03:06:12 » |
|
We wrote our own math lib from the ground up.
|
Renanse (ruh-NON-say)
|
|
|
Games published by our own members! Check 'em out!
|
|
renanse
Junior Devvie   Exp: 14 years
Intelligence is light to a dark world.
|
 |
«
Reply #10 - Posted
2006-09-30 03:40:59 » |
|
Attached is a modified BSPClusterManager that does not create garbage during updates. Rename it to .java, of course.
Also, after profiling your code I've discovered that most of your garbage is being created by the recreating of the ShapeAtom in your Shape3D objects everytime it is readded to the scenegraph (as you move around the map and the portal view system does it's work.) Simply commenting out line 299 in Shape3D where it sets the atom to null in the method setLive(boolean) gets rid of almost all of the extra object creation, makes things very stable and gives you a nice little speed boost (a few frames on average). I'm not saying that's the solution, but it might give you an idea of how the BSP portal management might need to change.
Hope that is useful!
|
Renanse (ruh-NON-say)
|
|
|
Amos Wenger
|
 |
«
Reply #11 - Posted
2006-09-30 12:04:12 » |
|
|
"Once you start working on something, don't be afraid of failure and don't abandon it. People who work sincerely are the happiest"
|
|
|
Marvin Fröhlich
|
 |
«
Reply #12 - Posted
2006-09-30 12:22:30 » |
|
Yeah, I already found your code comment about that  . Now I've commented the line renanse suggested in Shape3D and I'll see with the different additional improvements how fast is it, and report it to you, of course.  Well, I'm not really sure if just commenting out this line is correct. I'll investigate it. But it will certainly do the trick for this test  . Marvin
|
|
|
|
Amos Wenger
|
 |
«
Reply #13 - Posted
2006-09-30 12:35:24 » |
|
Now I've commented the line renanse suggested in Shape3D and I'll see with the different additional improvements how fast is it, and report it to you, of course.  Well, I'm not really sure if just commenting out this line is correct. I'll investigate it. But it will certainly do the trick for this test  . In fact, we should provide a softcache with RenderAtoms.. when the memory usage is > 60-70% the older atoms are trashed. I t hink it would be nice this way.
|
"Once you start working on something, don't be afraid of failure and don't abandon it. People who work sincerely are the happiest"
|
|
|
Marvin Fröhlich
|
 |
«
Reply #14 - Posted
2006-09-30 12:42:23 » |
|
In fact, we should provide a softcache with RenderAtoms.. when the memory usage is > 60-70% the older atoms are trashed. I t hink it would be nice this way.
Well, the atoms will never be "old". They can always be recycled.
|
|
|
|
Amos Wenger
|
 |
«
Reply #15 - Posted
2006-09-30 13:16:02 » |
|
In fact, we should provide a softcache with RenderAtoms.. when the memory usage is > 60-70% the older atoms are trashed. I t hink it would be nice this way.
Well, the atoms will never be "old". They can always be recycled. Why ? They do contain shape-specific info so if the shape is removed from the scenegraph or deleted the atom is of no use, if the shape is not re-added later... ?
|
"Once you start working on something, don't be afraid of failure and don't abandon it. People who work sincerely are the happiest"
|
|
|
Marvin Fröhlich
|
 |
«
Reply #16 - Posted
2006-09-30 13:19:47 » |
|
Why ? They do contain shape-specific info so if the shape is removed from the scenegraph or deleted the atom is of no use, if the shape is not re-added later... ?
That's true. And this is exactly the cause, why I said: Well, I'm not really sure if just commenting out this line is correct. I'll investigate it. But it will certainly do the trick for this test  . But for scenegraph persistent shapes the atom will never be "old". Marvin
|
|
|
|
Amos Wenger
|
 |
«
Reply #17 - Posted
2006-09-30 13:26:03 » |
|
Yeah, so we should provide a way to tell that nodes are persistent (thus, their atom should not be deleted). maybe the ones which are static ? (display lists)  wouldn't that be nice ?
|
"Once you start working on something, don't be afraid of failure and don't abandon it. People who work sincerely are the happiest"
|
|
|
Marvin Fröhlich
|
 |
«
Reply #18 - Posted
2006-09-30 13:29:29 » |
|
Yeah, so we should provide a way to tell that nodes are persistent (thus, their atom should not be deleted). maybe the ones which are static ? (display lists)  wouldn't that be nice ? No. When you remove a Node from the scenegraph the only reference to the Atom is in the Shape itself. So unless you waste the pointer to the shape the Atom is held by the shape and can be reused. When you decide to never reuse the shape and waste the last pointer to it, the atom is wasted, too. Correct me if I'm wrong, but I think this is possibly the most efficient and convenient way to handle atom wasting, isn't it. Marvin
|
|
|
|
Amos Wenger
|
 |
«
Reply #19 - Posted
2006-09-30 14:56:53 » |
|
Yeah, so we should provide a way to tell that nodes are persistent (thus, their atom should not be deleted). maybe the ones which are static ? (display lists)  wouldn't that be nice ? No. When you remove a Node from the scenegraph the only reference to the Atom is in the Shape itself. So unless you waste the pointer to the shape the Atom is held by the shape and can be reused. When you decide to never reuse the shape and waste the last pointer to it, the atom is wasted, too. Correct me if I'm wrong, but I think this is possibly the most efficient and convenient way to handle atom wasting, isn't it. Marvin oh, yeah, sure.
|
"Once you start working on something, don't be afraid of failure and don't abandon it. People who work sincerely are the happiest"
|
|
|
Riven
|
 |
«
Reply #20 - Posted
2006-09-30 15:46:45 » |
|
I know I've created that other thread about GC and Object allocation of tiny objects, but I really think you guys are working on the wrong problem, it doesn't seem to be the bottleneck!
Run your app with: -verbose:gc and calculate how much time per second actually is spent cleaning up old objects. Most likely it is in the range of 0.0-0.2ms per second. Running a game for a minute, and find out 360.000 were disposed, means 6 objects per millisecond were collected. That's just noise-level, performance-wise.
I'm not saying it's NOT a bottleneck in Xith, I'm saying you should investigate it, before spending hours on something that might only gain you a few FPS, while working on something else would double the framerate.
My $0.02!
|
Hi, appreciate more people! Σ ♥ = ¾ Learn how to award medals... and work your way up the social rankings!
|
|
|
renanse
Junior Devvie   Exp: 14 years
Intelligence is light to a dark world.
|
 |
«
Reply #21 - Posted
2006-09-30 15:58:23 » |
|
I think you are missing the gc issue here. It's not just gc, it's complex object recreation... It also only happens as you fly around due to shapes being added and removed from the scene. (try hitting F10 and letting it fly around while you look at garbage collection.) And the goal of the gc cleanup they are doing is not to increase frame rate really, it's to preserve frame rate over time in situations like bsp where occlusion and such mandates altering the scenegraph often.
|
Renanse (ruh-NON-say)
|
|
|
Riven
|
 |
«
Reply #22 - Posted
2006-09-30 16:04:24 » |
|
Try hitting F10 where and let fly what around?
Link to app, please.
|
Hi, appreciate more people! Σ ♥ = ¾ Learn how to award medals... and work your way up the social rankings!
|
|
|
Marvin Fröhlich
|
 |
«
Reply #23 - Posted
2006-09-30 16:12:29 » |
|
Try hitting F10 where and let fly what around?
Link to app, please.
org.xith3d.test.loader.BSPLoaderTest
|
|
|
|
Marvin Fröhlich
|
 |
«
Reply #24 - Posted
2006-10-01 16:49:30 » |
|
I really don't know why, but the GC overhaed seems to have gone (or been reduced to an acceptable level). It must have anything to do with my recent changes. But I don't know what did the trick. Well... anyway... the Q3 flight won't hick anymore.
@Amos: Please remeasure.
Marvin
|
|
|
|
Amos Wenger
|
 |
«
Reply #25 - Posted
2006-10-02 18:33:24 » |
|
Can you send the sources of your modified vecmath ?
|
"Once you start working on something, don't be afraid of failure and don't abandon it. People who work sincerely are the happiest"
|
|
|
Marvin Fröhlich
|
 |
«
Reply #26 - Posted
2006-10-02 18:36:55 » |
|
I really think our GC problems are gone. Have you already remeasured? The Q3 flight runs at stable 135 FPS on my system.
Marvin
|
|
|
|
|