Java-Gaming.org Hi !
Featured games (83)
games approved by the League of Dukes
Games in Showcase (538)
Games in Android Showcase (132)
games submitted by our members
Games in WIP (601)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  Profiling Xith memory usage  (Read 2832 times)
0 Members and 1 Guest are viewing this topic.
Offline Jani Laakso

Junior Devvie




Do it with Java!


« Posted 2004-03-14 19:26:17 »

I did some profiling of Xith3d and decided to list some parts of Xith methods and classes that create most work for the garbage collector. I do not know if any of these object creations can be avoided, e.g. by reusing existing objects but I decided to list the methods anyway as I got the report easily while evaluating the profiler.

My test application consisted of 200 objects that are constantly rotated and positioned. The instance count list below consists of Xith objects only. The counts are a result from 2000 view.renderOnce() calls only, so you can double the counts below after each 2000 more frames etc. GC was disabled.

And here is the top instance list:

5719196 double[]
1623184 Object[]
273137 java.util.ArrayList
271142 long[]
271137 com.xith3d.render.StateSortableMap
271137 com.xith3d.render.ColoringShader
271137 com.xith3d.render.ShapeAtom
271137 com.xith3d.render.TextureShader
271137 com.xith3d.render.TransformShader
70943 com.xith3d.render.MaterialShader
64944 float[]
15992 javax.vecmath.Vector3f
8000 com.xith3d.render.RenderingShader
8000 com.xith3d.render.PolygonAttrShader
..

The most interesting part is hefty amount of doubles, 6 million after 2000 frame iterations.
Have no clue what creates these doubles but if I add following code to Xith instead, then renderOnce() (actually getScale method) does no longer create doubles excessively .
com.xith3d.scenegraph.Transform3D
public float getScale() {
 //return matrix.getScale(); OLD LINE
 return matrix.getScaleCustom(); // new line
}
com.xith3d.datatypes.MatrixExt4f
// new method
public final float getScaleCustom() {
 return (float) Math.sqrt((m00 * m00 + m10 * m10 + m20 * m20 + m01
   * m01 + m11 * m11 + m21 * m21 + m02 * m02 + m12 * m12 + m22
   * m22) / 3.0);
}

And here are the "hotspots" of object creations..

com.xith3d.render.TransformShader
makeTransformShader()
-new TransformShader

com.xith3d.render.RenderAtomBase
-new StateSortableMap
-new ArrayList

com.xith3d.render.StateSortableMap
-new StateSortable array
-new long array

com.xith3d.render.ColoringShader
makeColoringShader()
-new ColoringShader

com.xith3d.render.TextureShader
makeTextureShader
-new TextureShader

com.xith3d.scenegraph.View
getAtom
-new ShapeAtom

com.xith3d.scenegraph.Node
updateBounds
-new Bounds array


Hope this info was any good, Jani!
Offline Bombadil

Senior Devvie





« Reply #1 - Posted 2004-03-15 05:50:35 »

Yes, it's helpful. Because it also confirms what I experienced some days ago. Please have a short look at my thread here: the interesting article is the second one named "Question B) jerky movement".

In that thread Yuri replied:
Quote
To minimize problems caused by full GC, we should try to minimize memory allocations inside Xith3D rendering engine. abies already did a lot to reduce memory allocations, but there are still some things to do in that area.


Interesting though, that aside the Xith3d developers and helpers (Abies?) just you and me noticed these "full GC delays". Like my recent poll indicates, the Xith3d user base is still very small... as are Java interested game programmers in general. :-(
Offline Jani Laakso

Junior Devvie




Do it with Java!


« Reply #2 - Posted 2004-03-15 06:49:13 »

Quote
Yes, it's helpful. Because it also confirms what I experienced some days ago. Please have a short look at my thread here: the interesting article is the second one named "Question B) jerky movement".


This is the reason:
TransformGroup.setTransform(Transform3D)
It trashes memory quite a lot. Every transform means lot's of different tasks for the renderer to do.

It's extremely easy to see cpu and memory intensive blocks with a profiler like OptimizeIT. You can download 10 day evaluation version from:
http://www.borland.com/products/downloads/download_optimizeit.html

I dont know if all object creation can be reasonably avoided but some optimizations could be done over there. Because now you have to tweak GC a bit in order to avoid
small freezes from happen, e.g. use incgc to avoid freezes but all solutions lower general performance somewhat as long as GC has a lot of work to do.

Note that if you have any tight loops e.g. iterators that go through every visible object and set new coordinates and orientations to your application, make sure you are not unneccessarily creating e.g. Vector3f and Matrix3f objects per each step. This strains GC even more. Just create your transform temporary objects once and re-use them, this helps some.
[/quote]

Quote

Interesting though, that aside the Xith3d developers and helpers (Abies?) just you and me noticed these "full GC delays".


Put several hundred objects moving to your scene and you are bound to see freezes with standard JVM parameters, GC has a lot to collect..

Summary, Xith might have more important tasks first before optimizing excessive object creation. Then again, maybe not.

PS. Do you have clue what makes those doubles, is it vecmath that has a bug? I got around this by creating my own setScale method.
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline abies

Senior Devvie





« Reply #3 - Posted 2004-03-15 16:12:00 »

As for the vecmath, use Kenji Hiranabe version - it is a lot better, as it creates no temporary objects (except one method somewhere, I have corrected it at some point and forgotten now). I have not even started profiling without this change, because data was skewed too much by java3d vecmath garbage. Funny thing is that java3d itself managed to avoid using this heavy methods - it was generating next to none garbage (something like 2 objects per frame for non-trivial scene).

As far as I remember (I have checked it some time ago), one of major garbage generators left in xith3d is ShapeAtom. AFAIK, it is created from scratch for any change in any state of shape3d. I have changed it some time ago to reuse the object and reuse all shaders if they have not changed - this reduced amount of garbage considerably. Unfortunately, I was not able to code it in way that looks elegant (spagetti code is real bad) and I was a bit afraid about possible side effects on state sorting routines, as I'm not fully aware of the way they work.

Artur Biesiadowski
Offline Bombadil

Senior Devvie





« Reply #4 - Posted 2004-03-16 03:35:50 »

@Jani: Yes, I always use tempory Transfrom3D() objects which aren't being destroyed and re-created when a model is being translated/rotated.

@Abies: Ah, so you suggest to use Kenji Hiranabe's Vecmath. Good hint, I'll try that.

In a thread here in the forum you quoted an URL to a slightly modified version of this improved Japanese Vecmath. (Inside the Zip there's a Readme file but the URL to the original Japanese Web site doesn't work anymore...)

Since you and Yuri say that the slightly modified Japanese Vecmath is good to use with Xith3d: maybe William should replace the Java3d Vecmath on the Xith3d download site with the improved one?
Offline abies

Senior Devvie





« Reply #5 - Posted 2004-03-16 05:30:19 »

In zip on my page I have also added few methods which are not really needed (I was not aware that opengl has transpose-matrix extension, so I have added methods to vecmath). Tonight I'll try to prepare version with just bugfixes/missing Tex4f.

Artur Biesiadowski
Offline Bombadil

Senior Devvie





« Reply #6 - Posted 2004-03-16 05:55:59 »

Quote
In zip on my page I have also added few methods which are not really needed (I was not aware that opengl has transpose-matrix extension, so I have added methods to vecmath). Tonight I'll try to prepare version with just bugfixes/missing Tex4f.

Thanks a lot! :-)

With your current Vecmath version the full GC does occur a little rarer on my example jnlp with 100 rotated objects (mentioned in my first post). However a full GC still occurs every ~7 seconds when all 100 objects are visible.
Well, that's an improvement compared to the full GC occuring every ~3 seconds with the original J3d Vecmath.


{Edit}: Inspired by the thread named "RFE: SERVER jvm into JRE" I've run the same test program with 100 rotating objects with the -server VM switch on Windoze, and voila: when the program runs for 2-3 seconds and is "warm", the full GC just occurs every ~3 minutes or so, which is a lot better. So it's not a real problem for my test program anymore. (I'm using Abies' japanese Vecmath.)
Offline princec

« JGO Spiffy Duke »


Medals: 433
Projects: 3
Exp: 16 years


Eh? Who? What? ... Me?


« Reply #7 - Posted 2004-03-16 06:37:55 »

Is there any way that floats could be used instead of doubles, or is Xith intended to be a general purpose scenegraph rather than a games scenegraph?

Cas Smiley

Offline Bombadil

Senior Devvie





« Reply #8 - Posted 2004-03-16 06:48:21 »

Quote
Is there any way that floats could be used instead of doubles, or is Xith intended to be a general purpose scenegraph rather than a games scenegraph?

From a user's point of view Xith uses just floats. Internally mostly, too.

{Edit}  Xith FAQ #8
Quote
9. Why do none of the methods accept doubles like Java3D did?
Since the underlying graphics hardware only supports floats - a design decision was made to only use floats for all method calls in Xith3D.



PS: However there's some talk in the forums that today on modern machines the difference between using doubles and floats (all done on the co-processor) doesn't mean a big performance problem anymore... isn't it?
Offline abies

Senior Devvie





« Reply #9 - Posted 2004-03-16 11:21:55 »

There is a noticeable difference if you start to think about more complicated operations - especially square root. Not that java has float sqrt anyway....

BTW, have you noticed that there is sqrt(x^2+y^2) operation hardcoded in 1.5 Math class ? This can probably map directly to some SSE/etc opcodes...

Artur Biesiadowski
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline abies

Senior Devvie





« Reply #10 - Posted 2004-03-16 17:58:27 »

It is available at
http://nwn-j3d.sourceforge.net/xith3d/vecmath.zip

It turned out that after removing my row/column major changes, only difference is one small optimalization in GMatrix and added Tex4x classes.

Artur Biesiadowski
Offline Bombadil

Senior Devvie





« Reply #11 - Posted 2004-03-17 06:47:19 »

Quote
It is available at
http://nwn-j3d.sourceforge.net/xith3d/vecmath.zip

It turned out that after removing my row/column major changes, only difference is one small optimalization in GMatrix and added Tex4x classes.

Thanks.

One small problem still: the alternative Vecmath reports an an error message when you try to use the clone() method. It says:
clone() has protected access in java.lang.Object

So when you have got a Point3f object namend thepoint and used
Point3f new = (Point3f) thepoint.clone();
this doesn't work anymore.

OK, you could use
Point3f new =new Point3f(thepoint);
... but for the sake of full portability I think it would be nice to have the clone method working with the new Vecmath, too. What do you think?
Offline abies

Senior Devvie





« Reply #12 - Posted 2004-03-17 07:01:20 »

I think it is just a problem of adding Cloneable interface to TupleXY. I'll do it tonight.

Artur Biesiadowski
Offline Bombadil

Senior Devvie





« Reply #13 - Posted 2004-03-17 09:38:27 »

Quote
I think it is just a problem of adding Cloneable interface to TupleXY. I'll do it tonight.

Yes. Additionally we would have to overwrite the protected clone() method with a public one I think. For example something like this seems to work...:

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
public abstract class Tuple3f implements Serializable, Cloneable
{
  ...

  /**
   * @return  Binary copy of object
   */

  public Object clone()
  {
    try {
      Object copy = super.clone();
      return copy;
    }
    catch (CloneNotSupportedException ex) {
      System.err.println(ex);
      throw new RuntimeException(ex);
    }
  }
}
Offline Jani Laakso

Junior Devvie




Do it with Java!


« Reply #14 - Posted 2004-03-24 16:51:43 »

Considering the coming Java gaming contest, I'd like to ask again about the state of Xith memory usage.

Does anyone have ideas if Xith's memory management can be fixed in a month or so? Theres plenty of classes that use the dreaded new clause quite excessively and this strains garbage collector quite a lot. It slows down the performance or causes minor jerking on bigger scenes that have hundreds of objects constantly being transformed.

Again, the "hotspot" that causes excessive object creation is TransformGroup.setTransform(Transform3D), it calls various classes and does several functions, various classes create a lot of new objects and discard old values.

I tried to help a bit by giving a summary of the classes that do excessive object creation (see first mail on this thread), perhaps I might help out some more but Xith internals are new to me.

I assume someone already fixed vecmaths excessive double creation that I stated earlier?

Well, I understand that Xith is under development, anyway this is promising technology have to say!

Cheers, Jani!
Offline abies

Senior Devvie





« Reply #15 - Posted 2004-03-24 18:42:06 »

With exception of collision routines, it CAN be corrected withing a month - it is really just few places where considerable amount of garbage is generated. Question is, if somebody will do it...

Artur Biesiadowski
Offline blahblahblahh

JGO Coder


Medals: 1


http://t-machine.org


« Reply #16 - Posted 2004-03-24 20:27:44 »

Quote
Y
Interesting though, that aside the Xith3d developers and helpers (Abies?) just you and me noticed these "full GC delays". Like my recent poll indicates, the Xith3d user base is still very small... as are Java interested game programmers in general. :-(


This perhaps is the cause of the jerkiness I see in *almost* every one of the Xith demos on the website? About one jerk per second (nb: I don't have much RAM, only 256 and desktop linux is an evil memory hogger). The silly thing is that apart from that its smooth, it just has a regular (very small) jerk - especially noticeable on the terrain / heightfield demo.

Seeing the simple official demos run poorly on a gefroce2 @ 1ghz is enough to give *anyone* serious 2nd thoughts I'd think...

malloc will be first against the wall when the revolution comes...
Offline Jani Laakso

Junior Devvie




Do it with Java!


« Reply #17 - Posted 2004-03-25 03:52:07 »

Quote
With exception of collision routines, it CAN be corrected withing a month - it is really just few places where considerable amount of garbage is generated. Question is, if somebody will do it...


Can you tell me the people that can affect to this issue in any way? Sure the people are watching these threads also, but I'm considering to send annoying but polite Smiley query for some people and ask experts opinions about this issue (how complex is it, is there plans to fix it in the near future, time estimates).  

This problem definately is not on JOGL.
Offline Bombadil

Senior Devvie





« Reply #18 - Posted 2004-03-25 05:19:18 »

Quote

This perhaps is the cause of the jerkiness I see in *almost* every one of the Xith demos on the website? About one jerk per second. (nb: I don't have much RAM, only 256 and desktop linux is an evil memory hogger). The silly thing is that apart from that its smooth, it just has a regular (very small) jerk - especially noticeable on the terrain / heightfield demo.

{Edit} Now I see what you mean: small jerks, yes, they're visible with the demos on my Winblows box, too. Maybe it's something with their renderloops.
In a "real" application, with timed simulation & render loop, I just see jerks when many objects (100+) with transformations are inside the view. Then every few seconds I see jerks. This is due to a full GC which occurs every few seconds in the Xith transformation code. I think Jani's memory observation explains it.

Generally these GC related jerks can be examined by tring the following: start the demos or/and your app with:
java -verbose:gc ...
However with the Xith demos I can't get it to work right now.

The Japanese Vecmath patched by Abies improves the memory consumation slightly for me.

However using the server VM solves most of my full GC problems with Xith (the jerks appears every few minutes only). However it isn't a solution when it comes to deployment via Webstart or similar. So, like Jani, I'm too interested in this being fixed within Xith.
Offline princec

« JGO Spiffy Duke »


Medals: 433
Projects: 3
Exp: 16 years


Eh? Who? What? ... Me?


« Reply #19 - Posted 2004-03-25 06:32:47 »

The new 1.5 collectors rock. The -XX:MaxGCPauseMillis=nnn will tune the garbage collector automatically for you. A bit like staring at that graphical profily thing and fiddling with the old tuning parameters in 1.4 until it doesn't jerk.

Cas Smiley

Offline blahblahblahh

JGO Coder


Medals: 1


http://t-machine.org


« Reply #20 - Posted 2004-03-25 06:42:28 »

Quote

Please try the following: start the demos or/and your app with:


OK, I'll try sometime (I've only been running the JWS demos so far, and I'm too busy right now to sit down for more than about 1 minute!)

I'll have a go with 1.5beta as well, see if the same thing happens.

malloc will be first against the wall when the revolution comes...
Offline Jani Laakso

Junior Devvie




Do it with Java!


« Reply #21 - Posted 2004-03-25 18:16:26 »

Quote

OK, I'll try sometime (I've only been running the JWS demos so far, and I'm too busy right now to sit down for more than about 1 minute!)


I sincerily believe the only right solution is to get rid of these excessive object creations. I'm sure JOGL, LWJGL or jME (scenegraph also) do not act like this.

Sure you can set your -Xms -Xmx into 1gigabyte and "buy" big timewindow before first gc bang occurs, or use -incgc and tweak your garbage collector to run all the time on the background, but the overall performance suffers from this. I haven't done benchmarks but I believe fps takes a noticeable hit because of this. I am now talking about scenes that have hundreds of objects moving.

Again, the "guilty" part is TransformGroup.setTransform() call. It does several functions and trashes memory bigtime.

Quote

I'll have a go with 1.5beta as well, see if the same thing happens.


I am positive that it happens, or at least in the expense of performance. Nothing changes the facts that there's lots of work for GC to do in any case.

Last words and reasons why I started this thread:
Ok, perhaps we should remember that Xith is heavily under development and just accept the facts, I'm sure this will be corrected before v1.0 is out, I just wanted to prioritize this issue a bit more on the development queue :-)


Cheers, Jani!
Offline Yuri Vl. Gushchin

Senior Devvie




Speak Java!


« Reply #22 - Posted 2004-03-26 09:39:30 »

OK, I will try to take a deeper look at setTransform(...) nearest week. This can be non-trivial task, because of transform shader and state management is involved... but we will see.

Yuri

Yuri Vl. Gushchin
JProof Group
Offline Yuri Vl. Gushchin

Senior Devvie




Speak Java!


« Reply #23 - Posted 2004-04-25 15:40:42 »

Hi,

Now setTransform(...) on TransformGroup should produce no garbage. Please test if your apps run OK with current CVS version.

Yuri

Yuri Vl. Gushchin
JProof Group
Pages: [1]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

rwatson462 (30 views)
2014-12-15 09:26:44

Mr.CodeIt (20 views)
2014-12-14 19:50:38

BurntPizza (42 views)
2014-12-09 22:41:13

BurntPizza (76 views)
2014-12-08 04:46:31

JscottyBieshaar (37 views)
2014-12-05 12:39:02

SHC (51 views)
2014-12-03 16:27:13

CopyableCougar4 (48 views)
2014-11-29 21:32:03

toopeicgaming1999 (115 views)
2014-11-26 15:22:04

toopeicgaming1999 (103 views)
2014-11-26 15:20:36

toopeicgaming1999 (30 views)
2014-11-26 15:20:08
Resources for WIP games
by kpars
2014-12-18 10:26:14

Understanding relations between setOrigin, setScale and setPosition in libGdx
by mbabuskov
2014-10-09 22:35:00

Definite guide to supporting multiple device resolutions on Android (2014)
by mbabuskov
2014-10-02 22:36:02

List of Learning Resources
by Longor1996
2014-08-16 10:40:00

List of Learning Resources
by SilverTiger
2014-08-05 19:33:27

Resources for WIP games
by CogWheelz
2014-08-01 16:20:17

Resources for WIP games
by CogWheelz
2014-08-01 16:19:50

List of Learning Resources
by SilverTiger
2014-07-31 16:29:50
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!