Java-Gaming.org Hi !
Featured games (90)
games approved by the League of Dukes
Games in Showcase (753)
Games in Android Showcase (228)
games submitted by our members
Games in WIP (842)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  Long collection times (Perm/Static heap)  (Read 2812 times)
0 Members and 1 Guest are viewing this topic.
Offline rreyelts

Junior Devvie




There is nothing Nu under the sun


« Posted 2004-10-28 17:18:09 »

Is there anything that can be done about the long collection times caused by having large numbers of objects in the VM?

Basically, if I have 10 million live objects in the VM, the garbage collection takes forever, because it's scanning all of those objects. I know that these 10 million objects are going to live a very long time - they're essentially static data - so if there were some way for me to indicate that they should go directly into some special heap (static heap?) that only gets examined very rarely (perhaps a scan can be specifically requested), then the garbage collection times would go back down to nothing.

As it is now, the only way to work around this is to represent those objects as primitive values, on or off the Java heap.

Azeem, can you respond to this?

God bless,
-Toby Reyelts

About me: http://jroller.com/page/rreyelts
Jace - Easier JNI: http://jace.reyelts.com/jace
Retroweaver - Compile on JDK1.5, and deploy on 1.4: http://retroweaver.sf.net.
Offline Azeem Jiva

Junior Devvie




Java VM Engineer, Sun Microsystems


« Reply #1 - Posted 2004-10-28 18:14:33 »

I'm not a GC expert at all, but I asked a GC guy, and one thing you could do is speed up the moving of objects from new space to old space.  Use the flags -XX:InitialTenuringThreshold=1 -XX:MaxTenuringThreshold=1.  Of course you'll have other issues because you'll be putting other objects that would have stayed in new space over to old space.  There are pause times to worry about, and other issues.  

Do you really need to keep 10 million objects live?  Can't you create them when you need them, even if they live for a long time, just don't create them all at once?
Offline rreyelts

Junior Devvie




There is nothing Nu under the sun


« Reply #2 - Posted 2004-10-28 19:44:07 »

Quote
I'm not a GC expert at all, but I asked a GC guy

Thanks.

Quote
and one thing you could do is speed up the moving of objects from new space to old space.  Use the flags -XX:InitialTenuringThreshold=1 -XX:MaxTenuringThreshold=1.  Of course you'll have other issues because you'll be putting other objects that would have stayed in new space over to old space.  There are pause times to worry about, and other issues.

The problem isn't the objects being in new space. They all end up in tenured space pretty fast. When a full collection runs (which happens fairly often), the collector scans all the objects in the tenured space, which takes forever. 99.9% of those objects in tenured space won't ever need to be collected.

Quote
Do you really need to keep 10 million objects live?  Can't you create them when you need them, even if they live for a long time, just don't create them all at once?

Yes, I need all of those objects. They belong to large data sets like road networks, spatial indices, etc... Those data structures get created at load time and are never released. What I want is an additional memory space, beyond tenured, that those objects can be placed in, where they will rarely or never be scanned.

God bless,
-Toby Reyelts

About me: http://jroller.com/page/rreyelts
Jace - Easier JNI: http://jace.reyelts.com/jace
Retroweaver - Compile on JDK1.5, and deploy on 1.4: http://retroweaver.sf.net.
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Azeem Jiva

Junior Devvie




Java VM Engineer, Sun Microsystems


« Reply #3 - Posted 2004-10-28 19:48:01 »

Hmmm well from what I've been told, unfortunetly there isn't much we can do.  You might try one of the other collectors, but I think they're all the same in this respect.   Like I said before, definitly not a GC expert. Sorry
Offline princec

« JGO Spiffy Duke »


Medals: 1012
Projects: 3
Exp: 20 years


Eh? Who? What? ... Me?


« Reply #4 - Posted 2004-10-28 19:57:05 »

Sounds like what you need is either Structs (ie. lightweight objects), or a way to utilise different heaps with different collection characteristics, along the lines of

HugeRoadStructureThing x = new HugeRoadStructureThing() in UncollectedHeap;

Cas Smiley

Offline rreyelts

Junior Devvie




There is nothing Nu under the sun


« Reply #5 - Posted 2004-10-28 20:45:44 »

Quote

HugeRoadStructureThing x = new HugeRoadStructureThing() in UncollectedHeap;

Right - like Real Time Java.

Node node = ImmortalMemory.instance().newInstance( Node.class );

God bless,
-Toby Reyelts

About me: http://jroller.com/page/rreyelts
Jace - Easier JNI: http://jace.reyelts.com/jace
Retroweaver - Compile on JDK1.5, and deploy on 1.4: http://retroweaver.sf.net.
Offline Mark Thornton

Senior Devvie





« Reply #6 - Posted 2004-10-29 07:43:01 »

My solution to this was to put all these big structures in a separate process that was started with a suitable initial size for the heap. Then it is just a matter of keeping the amount of gc subsequently needed by this process reasonably low.
Offline rreyelts

Junior Devvie




There is nothing Nu under the sun


« Reply #7 - Posted 2004-10-29 12:44:25 »

Quote
Then it is just a matter of keeping the amount of gc subsequently needed by this process reasonably low.

Sure, if you can avoid running any algorithms on those data structures that would generate garbage in the tenured heap. That's not been my experience.

God bless,
-Toby Reyelts

About me: http://jroller.com/page/rreyelts
Jace - Easier JNI: http://jace.reyelts.com/jace
Retroweaver - Compile on JDK1.5, and deploy on 1.4: http://retroweaver.sf.net.
Offline Mark Thornton

Senior Devvie





« Reply #8 - Posted 2004-10-29 13:33:07 »

It doesn't avoid the problem altogether, but does reduce it by not having to contend with garbage generated by unrelated activities. Also many of the algorithms I use on such structures either generate very little garbage or generate short lived garbage which doesn't trouble us. What causes trouble is creating large volumes of objects which live long enough to migrate out of the nursery area(s).
Pages: [1]
  ignore  |  Print  
 
 

 
ivj94 (586 views)
2018-03-24 14:47:39

ivj94 (49 views)
2018-03-24 14:46:31

ivj94 (383 views)
2018-03-24 14:43:53

Solater (63 views)
2018-03-17 05:04:08

nelsongames (110 views)
2018-03-05 17:56:34

Gornova (159 views)
2018-03-02 22:15:33

buddyBro (704 views)
2018-02-28 16:59:18

buddyBro (93 views)
2018-02-28 16:45:17

xxMrPHDxx (494 views)
2017-12-31 17:17:51

xxMrPHDxx (734 views)
2017-12-31 17:15:51
Java Gaming Resources
by philfrei
2017-12-05 19:38:37

Java Gaming Resources
by philfrei
2017-12-05 19:37:39

Java Gaming Resources
by philfrei
2017-12-05 19:36:10

Java Gaming Resources
by philfrei
2017-12-05 19:33:10

List of Learning Resources
by elect
2017-03-13 14:05:44

List of Learning Resources
by elect
2017-03-13 14:04:45

SF/X Libraries
by philfrei
2017-03-02 08:45:19

SF/X Libraries
by philfrei
2017-03-02 08:44:05
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!