Java-Gaming.org    
Featured games (79)
games approved by the League of Dukes
Games in Showcase (477)
Games in Android Showcase (106)
games submitted by our members
Games in WIP (533)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  Reducing GC overhead - 2nd edition  (Read 3913 times)
0 Members and 1 Guest are viewing this topic.
Offline Amos Wenger

Senior Member




Everything's possible, but not everything's fun...


« Posted 2006-09-29 18:51:09 »

Here's an interesting report made with YourKit profiler : http://www.xith.org/profiling/GC-Q3-Report.html
It shows the most garbage-producing methods. Some like AbstractArray.iterator() are unavoidable but these are just signs of other problems : too much array creation or somewhat.

Some other methods that worry me are the getScale() and invert() methods of Matrix4f : sooo much garbage ? How is it just possible.

BSPClusterManager.setVisibility(Vector3f) could surely be improved.

I'll now look into sourcecode and profile CPU times.

Note : If you want an open-source YourKit license, just mail me.

"Once you start working on something, don't be afraid of failure and don't abandon it. People who work sincerely are the happiest"
Offline Marvin Fröhlich

Senior Member




May the 4th, be with you...


« Reply #1 - Posted 2006-09-29 20:07:23 »

...Some like AbstractArray.iterator() are unavoidable...

Well, I guess most of them are. I've never thought into it and realized, that each loop over a List is a creation of an Iterator, which needs to be garbage collected. We could just loop over a List by a regular for (i = 0; i < list.size(); i++) loop. This can also be done very efficiently, and doesn't produce any garbage.

Some other methods that worry me are the getScale() and invert() methods of Matrix4f : sooo much garbage ? How is it just possible.

Strange Huh. No idea.

BSPClusterManager.setVisibility(Vector3f) could surely be improved.

Yes, for sure.

Marvin
Offline c_lilian

Senior Member


Projects: 1


Java games will probably rock someday...


« Reply #2 - Posted 2006-09-29 20:18:55 »

Matrix4f problems can be solved, has specified here http://www.java-gaming.org/forums/index.php?topic=11811.0

I've done it for my game, and reduced GC to nearly nothing

Lilian Smiley

Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Marvin Fröhlich

Senior Member




May the 4th, be with you...


« Reply #3 - Posted 2006-09-29 20:36:15 »

Hi,

As of VecMath and GC, this topic was discuseed 1-1.5 years ago (I think it should be possible to find it in forum archives).

Yes, standard version of vecmath from Sun produces a lot of garbage, but there is another one version of vecmath originally developed (published) by Kenji Hiranabe (I hope I spelled his name right from my head), and that library works pretty fine and produces nearly no garbage (or even no garbage at all). Artur Besiadovski (aka abies) pointed me to this library long time ago.

This lib is 100% compatible with javax.vecmath, (even from the naming) so I only made few modifications with it and just use another jar file instead of original vecmath.jar.

You should consider using this library I think. (actually, last time I pointed to some licensing issues, but to say truth I did not dig deep enough to this topic).

Yuri

Does anyone know, where to find this library? I'll continue to search for it, but so far with no success.

Marvin
Offline cylab

JGO Ninja


Medals: 38



« Reply #4 - Posted 2006-09-29 20:57:28 »

http://www.objectclub.jp/download/vecmath_e

Regarding the Licence: there is an overview on the vecmath page. It seems that an implementation must be able to pass suns test suite to be allowed to be redistributed. If I understand correctly, the implementation above complies to the Java3D 1.2 specification and should therefor fullfill this requirement.

Mathias - I Know What [you] Did Last Summer!
Offline Marvin Fröhlich

Senior Member




May the 4th, be with you...


« Reply #5 - Posted 2006-09-29 21:02:46 »

So would anyone object using (or first trying) this lib?
Offline cylab

JGO Ninja


Medals: 38



« Reply #6 - Posted 2006-09-29 21:06:51 »

Please try it. Since it is an inplace replacement, we could support both alternatives.

Mathias - I Know What [you] Did Last Summer!
Offline Marvin Fröhlich

Senior Member




May the 4th, be with you...


« Reply #7 - Posted 2006-09-29 23:22:22 »

I included the alternative vecmath implementation (vecmath-kh.jar) into xith, but the GC is still bad. Amos please remeasure GC. Hopefully the vecmath GC and iterator GC should have dropped to a minimum.

Marvin
Offline Marvin Fröhlich

Senior Member




May the 4th, be with you...


« Reply #8 - Posted 2006-09-30 00:33:40 »

Which vecmath lib do the JMonkey guys use?
Offline renanse

Junior Member




Intelligence is light to a dark world.


« Reply #9 - Posted 2006-09-30 05:06:12 »

We wrote our own math lib from the ground up.

Renanse  (ruh-NON-say)
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline renanse

Junior Member




Intelligence is light to a dark world.


« Reply #10 - Posted 2006-09-30 05:40:59 »

Attached is a modified BSPClusterManager that does not create garbage during updates.  Rename it to .java, of course.

Also, after profiling your code I've discovered that most of your garbage is being created by the recreating of the ShapeAtom in your Shape3D objects everytime it is readded to the scenegraph (as you move around the map and the portal view system does it's work.)  Simply commenting out line 299 in Shape3D where it sets the atom to null in the method setLive(boolean) gets rid of almost all of the extra object creation, makes things very stable and gives you a nice little speed boost (a few frames on average).  I'm not saying that's the solution, but it might give you an idea of how the BSP portal management might need to change.

Hope that is useful!

Renanse  (ruh-NON-say)
Offline Amos Wenger

Senior Member




Everything's possible, but not everything's fun...


« Reply #11 - Posted 2006-09-30 14:04:12 »

I already converted (before you mentioned it, Qudus) some enhanced-for loops (Shape3D shape : shapes) to classic loops (int i = 0; i< shapes.size(); i++) which gave me a HUUUUge performance boost  Grin Grin Grin Grin From 16.88FPS avg. I have gone up to 16.95FPS Yeah !! great.

Now I've commented the line renanse suggested in Shape3D and I'll see with the different additional improvements how fast is it, and report it to you, of course.  Cheesy

"Once you start working on something, don't be afraid of failure and don't abandon it. People who work sincerely are the happiest"
Offline Marvin Fröhlich

Senior Member




May the 4th, be with you...


« Reply #12 - Posted 2006-09-30 14:22:30 »

I already converted (before you mentioned it, Qudus) some enhanced-for loops (Shape3D shape : shapes) to classic loops (int i = 0; i< shapes.size(); i++) which gave me a HUUUUge performance boost  Grin Grin Grin Grin From 16.88FPS avg. I have gone up to 16.95FPS Yeah !! great.

Yeah, I already found your code comment about that Wink.

Now I've commented the line renanse suggested in Shape3D and I'll see with the different additional improvements how fast is it, and report it to you, of course.  Cheesy

Well, I'm not really sure if just commenting out this line is correct. I'll investigate it. But it will certainly do the trick for this test Wink.

Marvin
Offline Amos Wenger

Senior Member




Everything's possible, but not everything's fun...


« Reply #13 - Posted 2006-09-30 14:35:24 »

Now I've commented the line renanse suggested in Shape3D and I'll see with the different additional improvements how fast is it, and report it to you, of course.  Cheesy

Well, I'm not really sure if just commenting out this line is correct. I'll investigate it. But it will certainly do the trick for this test Wink.
In fact, we should provide a softcache with RenderAtoms.. when the memory usage is > 60-70% the older atoms are trashed. I t hink it would be nice this way.

"Once you start working on something, don't be afraid of failure and don't abandon it. People who work sincerely are the happiest"
Offline Marvin Fröhlich

Senior Member




May the 4th, be with you...


« Reply #14 - Posted 2006-09-30 14:42:23 »

In fact, we should provide a softcache with RenderAtoms.. when the memory usage is > 60-70% the older atoms are trashed. I t hink it would be nice this way.

Well, the atoms will never be "old". They can always be recycled.
Offline Amos Wenger

Senior Member




Everything's possible, but not everything's fun...


« Reply #15 - Posted 2006-09-30 15:16:02 »

In fact, we should provide a softcache with RenderAtoms.. when the memory usage is > 60-70% the older atoms are trashed. I t hink it would be nice this way.

Well, the atoms will never be "old". They can always be recycled.
Why ? They do contain shape-specific info so if the shape is removed from the scenegraph or deleted the atom is of no use, if the shape is not re-added later... ?

"Once you start working on something, don't be afraid of failure and don't abandon it. People who work sincerely are the happiest"
Offline Marvin Fröhlich

Senior Member




May the 4th, be with you...


« Reply #16 - Posted 2006-09-30 15:19:47 »

Why ? They do contain shape-specific info so if the shape is removed from the scenegraph or deleted the atom is of no use, if the shape is not re-added later... ?

That's true. And this is exactly the cause, why I said:
Well, I'm not really sure if just commenting out this line is correct. I'll investigate it. But it will certainly do the trick for this test Wink.

But for scenegraph persistent shapes the atom will never be "old".

Marvin
Offline Amos Wenger

Senior Member




Everything's possible, but not everything's fun...


« Reply #17 - Posted 2006-09-30 15:26:03 »

Yeah, so we should provide a way to tell that nodes are persistent (thus, their atom should not be deleted). maybe the ones which are static ? (display lists)  Cheesy wouldn't that be nice ?

"Once you start working on something, don't be afraid of failure and don't abandon it. People who work sincerely are the happiest"
Offline Marvin Fröhlich

Senior Member




May the 4th, be with you...


« Reply #18 - Posted 2006-09-30 15:29:29 »

Yeah, so we should provide a way to tell that nodes are persistent (thus, their atom should not be deleted). maybe the ones which are static ? (display lists)  Cheesy wouldn't that be nice ?

No. When you remove a Node from the scenegraph the only reference to the Atom is in the Shape itself. So unless you waste the pointer to the shape the Atom is held by the shape and can be reused. When you decide to never reuse the shape and waste the last pointer to it, the atom is wasted, too.

Correct me if I'm wrong, but I think this is possibly the most efficient and convenient way to handle atom wasting, isn't it.

Marvin
Offline Amos Wenger

Senior Member




Everything's possible, but not everything's fun...


« Reply #19 - Posted 2006-09-30 16:56:53 »

Yeah, so we should provide a way to tell that nodes are persistent (thus, their atom should not be deleted). maybe the ones which are static ? (display lists)  Cheesy wouldn't that be nice ?

No. When you remove a Node from the scenegraph the only reference to the Atom is in the Shape itself. So unless you waste the pointer to the shape the Atom is held by the shape and can be reused. When you decide to never reuse the shape and waste the last pointer to it, the atom is wasted, too.

Correct me if I'm wrong, but I think this is possibly the most efficient and convenient way to handle atom wasting, isn't it.

Marvin
oh, yeah, sure.

"Once you start working on something, don't be afraid of failure and don't abandon it. People who work sincerely are the happiest"
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 743
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #20 - Posted 2006-09-30 17:46:45 »

I know I've created that other thread about GC and Object allocation of tiny objects, but I really think you guys are working on the wrong problem, it doesn't seem to be the bottleneck!

Run your app with: -verbose:gc and calculate how much time per second actually is spent cleaning up old objects. Most likely it is in the range of 0.0-0.2ms per second. Running a game for a minute, and find out 360.000 were disposed, means 6 objects per millisecond were collected. That's just noise-level, performance-wise.

I'm not saying it's NOT a bottleneck in Xith, I'm saying you should investigate it, before spending hours on something that might only gain you a few FPS, while working on something else would double the framerate.

My $0.02!

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline renanse

Junior Member




Intelligence is light to a dark world.


« Reply #21 - Posted 2006-09-30 17:58:23 »

I think you are missing the gc issue here.  It's not just gc, it's complex object recreation...  It also only happens as you fly around due to shapes being added and removed from the scene. (try hitting F10 and letting it fly around while you look at garbage collection.)  And the goal of the gc cleanup they are doing is not to increase frame rate really, it's to preserve frame rate over time in situations like bsp where occlusion and such mandates altering the scenegraph often.

Renanse  (ruh-NON-say)
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 743
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #22 - Posted 2006-09-30 18:04:24 »

Try hitting F10 where and let fly what around?

Link to app, please.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Marvin Fröhlich

Senior Member




May the 4th, be with you...


« Reply #23 - Posted 2006-09-30 18:12:29 »

Try hitting F10 where and let fly what around?

Link to app, please.

org.xith3d.test.loader.BSPLoaderTest
Offline Marvin Fröhlich

Senior Member




May the 4th, be with you...


« Reply #24 - Posted 2006-10-01 18:49:30 »

I really don't know why, but the GC overhaed seems to have gone (or been reduced to an acceptable level). It must have anything to do with my recent changes. But I don't know what did the trick. Well... anyway... the Q3 flight won't hick anymore.

@Amos: Please remeasure.

Marvin
Offline Amos Wenger

Senior Member




Everything's possible, but not everything's fun...


« Reply #25 - Posted 2006-10-02 20:33:24 »

Matrix4f problems can be solved, has specified here http://www.java-gaming.org/forums/index.php?topic=11811.0

I've done it for my game, and reduced GC to nearly nothing
Can you send the sources of your modified vecmath ?

"Once you start working on something, don't be afraid of failure and don't abandon it. People who work sincerely are the happiest"
Offline Marvin Fröhlich

Senior Member




May the 4th, be with you...


« Reply #26 - Posted 2006-10-02 20:36:55 »

I really think our GC problems are gone. Have you already remeasured? The Q3 flight runs at stable 135 FPS on my system.

Marvin
Pages: [1]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

pw (25 views)
2014-07-24 01:59:36

Riven (25 views)
2014-07-23 21:16:32

Riven (19 views)
2014-07-23 21:07:15

Riven (22 views)
2014-07-23 20:56:16

ctomni231 (51 views)
2014-07-18 06:55:21

Zero Volt (46 views)
2014-07-17 23:47:54

danieldean (37 views)
2014-07-17 23:41:23

MustardPeter (40 views)
2014-07-16 23:30:00

Cero (56 views)
2014-07-16 00:42:17

Riven (55 views)
2014-07-14 18:02:53
HotSpot Options
by dleskov
2014-07-08 03:59:08

Java and Game Development Tutorials
by SwordsMiner
2014-06-14 00:58:24

Java and Game Development Tutorials
by SwordsMiner
2014-06-14 00:47:22

How do I start Java Game Development?
by ra4king
2014-05-17 11:13:37

HotSpot Options
by Roquen
2014-05-15 09:59:54

HotSpot Options
by Roquen
2014-05-06 15:03:10

Escape Analysis
by Roquen
2014-04-29 22:16:43

Experimental Toys
by Roquen
2014-04-28 13:24:22
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!