serialization - woe is me

[markuskidd]

For my multiplayer strategy game I'm writing, I eventually intend to be able to support around a hundred players on a Pentium-II class machine. I anticipate my game will require about 1kB/second, at most 2kB.

Since this is my first game, I thought it would be easier to use Serialization to send data to my clients while writing (and then refactoring, and then writing, and then refactoring) the rest of the game -- just so I would have one less thing to worry about as I get started, you see. The game is really close to getting done, and testing it out with just my client connecting to the server works just fine. I've also worked out a Web Start deployment and people have been able to play from remote locations with no problem. However, when two or even three players are playing simultaneously, the server starts acting nondeterminently and, well, slow as hell.

By this point, I have been able to determine that, since the Serializiation bloats my transfers a bit, the bandwith needed is around 4.5kB/second, but that's no big deal. And on the client end, where I was most worried about objects being spawned out of control by the deserialization, I haven't really had any performance problems to speak of yet. However, I downloaded the trial version of  OptomizeIt, and to my dismay, writeObject is the bad guy! With two clients connected to my server, sending Objects which are generally very simple, writeObject is spawning thousands upon thousands of Objects all over the place! AHHHH!!!

I thought that this would be a good way for me and for other people to take advantage of one of Java's niceties and ease into multiplayer game development, since that is an area I think really has a lot of potential right now. Unfortunately, all of my hopes are dashed in that direction. In the end I was going to have to convert to sending byte buffers anyway, so it's no big loss to me, but on the other hand it's going to be another week or two before I can see my game running  

/me weeps

[bmyers]

Yes, the default serialization strategy can get big really quick...

writeObject recursively writes all the Serializable objects in the tree, so in the worst case it has the potential to write out almost the entire heap, if your Serializable objects are all cross-referencing each other!  

For starters, you should make sure you mark as transient any attributes/members that you do not want to serialize, or that should not be serialized, in your Serializable objects.

You might want to investigate writeReplace() and readResolve(), to use light-weight "proxy" serializations.  Those two methods, under the Serializable interface, let you change the objects that actually get serialized/deserialized.

We are doing that for our MMORTS game, which results in 1) much-much faster serialization, and 2) much-much smaller packet size, without sacrificing the advantages of Serialization.

It took a few days to get it set up right, but the end result is pretty good!  We use a tag inside the "proxy" object to specify the "actual" class, and a lightweight HashMap to store the attributes.  Plus a version number to handle backwards compatibility issues.  We use the same code to do serialize/deserialize mapping, database mapping and XML mapping.  Whenever we add/remove an attribute to a class, we just update the mapping info and we're done - no additional code needed!

One thing to look out for - especially if you are using JMS, which can do serialization under the covers - is to make sure that your serialized "proxy" objects can get -re-serialized without corruption.  And vice-versa on re-deserialization.  

As always, the default Java implementation gets you pretty far, but the next-better version requires some more work!

I hope that helps!

--Brad

[markuskidd]

I have just double-checked for any Object "leaks" from fields that should be transient but aren't, and unfortunately I think I already have everything marked correctly. I will see what I can dig up on writeReplace() and readResolve(), but I've already tinkered a lot with Serialization and I'm wary of crossing the point of diminishing returns. On the other hand, a MMORTS written *with* serialization sounds very interesting. Do you have a link?

Games published by our own members! Check 'em out!

[bmyers]

Sure:  http://www.galactic-village.com

This is a work-in-progress.  We're still in development, and still working on getting publishing/hosting arrangements worked out.

The web site is a little stale...we're working on an upgraded version with screen shots, etc. and hope to have it up in a few weeks...

There's no downloadable or applet-based demo available yet, but we plan to have one with the upgraded web site.

And "RTS" might be a misnomer   -- it's really "paced strategic simulation"... maybe I should call it a MMOSIM?

--Brad

[mill]

just curios, how do you measure the packet size? is there support for this in Java or do I have to fire up good old Ethereal?

(i'm writing a customizable highscore (with gui and all) at the moment and i'm using Serialization at the moment)

[bmyers]

If you want a quick-n-dirty method of seeing how big a serialized object is using the default serialization, just call writeObject() on one of your serializable objects, and write it to a file.  Then look at the file to see what's in it.  

Or use a handy-dandy debugger and examine the stream contents.

I'm not measuring the network packet size itself, if that's what you mean.

[mill]

clever, thanks.

[coilcore]

Serialization is a beast for performance and should never be thought of as a means to do something like this rapidly.  

One relatively simple solution is to convert to Externalizable instead of Serializable. Of course then you need to do the fine grained grunt work of knowing whats being put in and in what order.  But its still relatively easy (albeit tedious work).

Secondly - IMHO an even simpler way I to find the size of a data chunk is to use put a convert it into byte[] (using ByteArrayOutputStream and ByteArrayInputStream) and look at then System.err the lengths.  Its alot easier to watch a bunch of packets that way then writing to a file (and a lot faster and no mess to clean up too).  You can also add sequence and millis and a few other things to your output and do some more indepth profiling.

[Jeff]

I'd caution against the "Externalizable" hack.

Almost anything that can be done with Externalizable can be done with readObject/writeObject.

What Externalizable does in addition is to short circuit the serialization code of an parent classes.  

IMO

Either (1) You know know the internals of your parent classes and have already tamed or otherwise dealt with them in the right palce (their code) OR
(2) You shouldn't be turning off transmission of data you don't know.

In general this is a serious defeat of the encapsulation provided by a class inheritance structure. Use with great caution if you don't want to be searchign for hard to find bugs 6 months from now.  The only really legitimate use I know for Externalizable is when you have no choice but to inherit from a parent over which whose source you have no control.

[GergisKhan]

Jeff,

An alternative viewpoint.

http://www.churchillobjects.com/c/11009i.html