I would recommend not blindly using char[]'s everywhere in place of Strings. But a char[] can very much outperform a String... in some situations.  This usually is related to when you might need to perform multiple mutations on a String or when you might need to recursively go through a String, where the char[] loop can handle many things simultaneously.  You can hope that Hotspot will catch and clean this for you, but the more complex the code the less likely it will happen.

For example the following snippet would be a nightmare to do any other way and is insanely fast.  

   

Doing this same thing in Strings is really yuckky and doing this with java.util.regex is really slow.  Microbenchmarking this shows a String version to be about 20x slower and the regex version to be about 100x slower.

While I would not advocate always using char[]'s, it does make good sense to use char[]'s where you understand the difference and the performance is important.  

When you serialize an object every member within that object is also serailized, and each of any members objects are serialized, and so on.  The way to prevent this is to mark a member as 'transient' this means it will not be serialized and therefore will not be in your data you send across the wire.  e.g.

 public ArrayList a = new ArrayList();
 public transient ArrayList b = new ArrayList();

The deserialized object ArrayList a will have all the contents from the original object, but b will be empty even if the serialized class had data there.

Clearly you are trying to deserialize something thats not the same between each JVM (either by your own classes or JVM version).  I'd suggest starting near the top and narrowing as you go making objects transient to try to isolate the problem.  

I did report a linking problem, however previously it was a dependancy in liblwjgl.so so the java binary would crash right off.  Its now no longer a dependancy (ldd confirms this), so the exception confused me.  

Odd exception for that kind of problem, also strange linking that all three filenames are required (.so, .so.0, and .so.0.0.0).  However that does indeed solve the problem.  After that everything seems to work fine.

A few other wierdness - Changing the control settings seems to have nuked all control settings.  I changed the left button to 'shoot' and right to 'thrust', that seems to have made all controls except the crosshair movement unusable.

Ok - I'm using Redhat 8 with the Linux .so files that Elias put in here.  Now I get a different Exception than before - related to sound.  The system is farily current but the sound card is not highend (read that as "cheap sound card") - I don't know the brand off hand, but it uses the standard es1371 driver and ac97_codec.  This has never had any problems before, so I suspect its something in the linux native code.  

Heres the exception:
org.lwjgl.openal.OpenALException: Unable to load function pointers to openal.
       at org.lwjgl.openal.BaseAL.nCreate(Native Method)
       at org.lwjgl.openal.BaseAL.create(BaseAL.java:100)
       at xap.Game.init(Game.java:411)
       at xap.Game.main(Game.java:246)
*** Alert ***
Alien Flux
Can't do any sound! Time for a new sound card.

Paypal like services, where the purchaser has to make an account with the third party first are certainly easy to use.  But I tend to shy away from them because it turns a fair number of people away, lets people know you are a very tiny shop, and can seem very amaturish for selling software.

Handling the credit cards yourself is the most time consuming and has a ton of headaches waiting in the wings. Most have monthly fee and eat about 2.5% of the transaction.  I've dealt with Cardservice International (http://www.cardservice.com) for this in the past and they're nothing special but not bad either, but there are a ton of these vendors to choose from.  Almost all of these have a Java API that you can integrate with.

For a happy medium between the time and the most professional looking there are third party companies that handle the order and transaction - basically you link or frame to their ssl site and they handle the shopping cart, processing transactions, et al.  Most of the credit card machine vendors above will do this, but there are comapnies that only do this -Verisign, Verotel, etc.  They will often absorb a hefty percent (5-15%), but usually have no monthly fees and offer addtional methods of payment - not just credit card or debit, but check, cash, and most can do recurring payments.  Again there are a number of these kinds of companies out there, but I tend to prefer Share*It ( http://www.shareit.com ) or Element5 (http://www.element5.com )  because they are all about handling software  (I think Share*It and Element5 are the same company with a different face - but I could be wrong).  If my memory serves they not only handle the transaction and order, but they handle the download itself using a unique-key based path.  Since its their bandwidth and they have the headaches involved with managing the order, transaction, and fulfillment the 5-15% seems like not that bad.

A few days late but still valid - Welcome back Jeff

Before I download that zip file is it a .jar or a .exe?  Last time I downloaded AlienFlux it was a .exe hidden inside a .zip.

Blahbalhbalhh -- Linux support is there and apparently has been there for some time, but the documentation wasn't changed to reflect this until about 12 hours ago.  So I thought the same thing as you, but in the last 12 hours I've been having a fine time tinkering with LWJGL.

Of course the real problem with TCP comes in from packet loss (and the subsequent retransmits and sequencing), which is when you start having to get into really interesting experiments with what works best for your particullar game.    

Its really so trivial that its more of a bug tracking issue than a forum issue.  But heres the link to the sourceforge bug
http://sourceforge.net/tracker/index.php?func=detail&aid=712258&group_id=58488&atid=487940

Also do you guys not use any kind of bug tracking?  Just keep it in your heads?  That doesn't seem like it can scale well.

Great now that I know this - I dug around a little found the rh8 stuff, and have those two targets in JOSRTS building on my system.  Though there is one tiny bug that I had to work around that I will report through sourceforge.

Heres what I know -

A while ago I looked that the LWJGL site on sourceforge and it said explicitly support for Win32.  

I followed the link Cas put out for the AlienFlux alpha and found a exe file wrapped in a zip.  So no dice, I figured it was because it used LWJGL and therefore no reason to not make a jar.

Yesterday I grabbed the source for the JOSRTS and the raw java compiles, but the 'test client' and 'map editor' targets fail because they are trying to include a bunch of LWJGL dlls into java.system.library and Linux does know what to do with them.  So I look again at the LWJGL site and it still said support for Win32 only.  

If LWJGL works under linux that is great - but it looks like the people who are using LWJGL still only do whats needed for Win32 and the site turned away people like me looking to make things work on my OS of choice.  

Probably the most import is turning off the Nagles algorithm, which is designed to gain bandwidth at the expense of latency

 Socket.setTcpNoDelay(true);    // TCP_NODELAY

There are other things to work on, but the time difference you were seeing doesn't make sense considering that you should get almost no packet loss since your on that switch.   Of course your packets will still be a bit bigger with TCP (more header info), but with no packet loss you should get nearly the same latency from TCP and UDP.  If you are not within a millisecond or two something is very wrong.

Does LWJGL support Linux now?  I thought it was limited to Win32 environs, which is the reason I'm working with Java3D rather than LWJGL.

In fact I remember reading something in the LWJGL docs that said something to the effect of  it should be for Win32 first and foremost, but that at some random point off in the future other platforms might be supported.  This seems to work against one of the really great advantages to doing games in Java and means if you use Java3D  what you write today will run on many platforms.  

Did you say its faster for you to spend "a little over a day" making your own than finding out what options you can use to tune TCP?   Hey by that same logic you should write your own database, configuring and using a canned database like MySQL can be diffcult.   I don't know about you but I can find dozens of articles that talk about optimizing TCP performance by spending 5 minutes googling.

Also expect to spend a bunch more time when you use that thing in a real situation (like inside your app across the internet cloud).

Generally I live by the philosophy of 'not reinventing the wheel', if you really understand TCP, have done all your proper tuning to how you are using TCP, and realy *REALLY* understand what your doing then by all means make a UDP implementation.  Its no walk in the park and you *WILL* spend a great amount of time trying to debug and make it work properly and you could still wind up with something that is effective as well tuned TCP.



This is quite simply not true.  There are congestion control mechanisms in most TCP stacks such as the Nagle algorithm , it *IS* the default behavior to use this because it reduces overall bandwidth utilization.  Those algorithms are easily turned off.

Let me say straight forward and bluntly your statements are a clear indication that you do not understand TCP enough and you would likely spend a inordinate amount of time reinventing the wheel and would likely still not have any better performance.

Some name dropping here (please forgive, I feel I must make sure those who made it happen get proper credit) --  

The previous posting stating that  'vonJacobson' who is really "Van Jacobson" did enable PPP link to use TCP header compression.   I worked at LBNL shortly after I worked in the game biz, not in the network technologies group but I did get to know them reasonably well (http://www-nrg.ee.lbl.gov/).  They were incredibly influential in the network world, the folks who made 'traceroute', 'tcpdump', and 'libpcap', and hundreds of low level optimizations to IP stacks and tools against TCP in the 90's.  Let me begin by saying that Van Jacobson is an incredibly brilliant man and did a lot of amazing and unimaginable things, he lead that group (I cannot give him the credit he deserves), however the TCP header compression was intended for very slow point-to-point links mostly which are disappearing every day and those that are not are out-optimizing his compression (e.g everyones compressing streams these days).

Back into the UDP v. TCP argument -- In the late 80's (I'll call this the pre-quake era) everything was IPX(Novell) , but there was some effort in trying to figure out what worked being converted to TCP/IP.  I was fortunate enough to work with a very brave, ingenious, and adventurous lot that were trying to make games playable on the Internet -- everything was TCP then.  Back in those days when one person lost the connection even in a game of backgammon, the whole game crashed (lockstep).  

Later, a few innovative and ingenious souls started to realize that the problem was not nature of the network but how they were looking at the network.  I personally think of Bill Lipa who was the Architect at TEN, but Quake came out a few weeks after TEN did Duke Nukem in UDP, so I'm not sure if it was Bill Lipa or John Carmack who originally thought of it first (I know I'm biased since I worked at TEN in those days, but Bill Lipa was a network proramming guru while John Carmack was a god, but mostly in graphics and non-networked game stuff).

I have to simultaneuously agree and disagree with ' blahblahblahh' .  I've not read the X-wing-versus-tie-fighter article, but I have to state quite bluntly, use TCP until you understand whether to use UDP in place of TCP.   There is quite simply no reason to blindly assume you will get better performance out of UDP than TCP, and using UDP will make things much more complicated.    

1.  "Guaranteed Delivery"

Agreed,  it is generally *NOT* worth doing on your own - if you don't understand this please accept this as fact!   Many, many really smart people have spent half of their lives on this and only gotten tiny advancements!  If its critical your packet gets there then use TCP, and focus on your game design.  

2.  "Connection negotiation and maintenance is not trivial."  

Agreed.  Again people have spent half of their lives on this and gotten tiny advancements.   Save yourself the grief and accept this as fact!  Unless you want to dedicate your life to a network protocol stack, again a reason to use TCP.

3.  "TCP is normally as fast or faster than UDP. "  

Heres where I disagree,  this depends largely on whether you are measuring as 'fast', latency or bandwidth.

"Normally, UDP is very inefficient and wastes bandwidth",

True UDP wastes bandwidth, but its in trade for latency.  If the raw packet delivery is more important than the sequence or state, and the game itself can deal with these factors on a frame-by-frame basis then what a great place we've gotten to.  The fact is in some games its important, in others, its not.

4.  "The vast majority of games developers only have one problem with TCP (but often mistakenly believe they need more!). They need to remove the "in-order arrival". "Whenever you hear about "TCP is sloooow" or "TCP has high latencies", you are listening to someone who is 99% likely to have bitten by this problem but not understood it.  The problem is that if you send 30 packets, and the fifth one gets dropped, but packets 6-10 arrive OK, your network stack will NOT allow your application to see those last 5 packets UNTIL it has received a re-transmitted packet 5."

Agreed, a single retransmitted packet can seem tragic for a developer trying to make their game playable across the internet.  Which often makes developers inappropriately reach for UDP, thinking it will fix their problem and usually it will not.  But there is something beautiful to the idea of being able to drop packet '5' and proceed that seems appealing.

5.  "Lastly, there ARE alternatives to TCP and UDP. Not surprisingly, since almost every game finds that neither is really good enough (the games that just go TCP only suffer weird stuttering, the ones that are UDP only often get players freezing, or their guns not firing because of lost packets). The last time I looked, ENet seems to be the best widely/freely available implementation around, but people have suggested several others to me, including: RAKnet (sp?), RDP (covered by an official internet RFC)
"

Ok here, fully disagree.   Random replacements for to on top of of IP seem to invoke the same arguments you were posing earlier - working against your initial premise.  I would say quite simply use TCP for reliable delivery, use UDP where latency is at a premium.

Cas -

I'm not really sure that the market share question really belongs in the 'Performance Tuning' section of the forum, but I do have one comment on your logic.  

You base a basic a general assumption on making your product available globally as opposed to a North American and European crowd.  This leads to more assumptions on system requirements, download size requirements, payment methods (a side note - the phone bill is another great medium as seen in Korea).  This should also open questions about the effectiveness of advertising and issues about what is reasonable pricing in a 'local' sense.    

So how does your equation change by going after a higher end market?  Of course there are no restrictions exactly, anyone with enough bandwidth to adequately download the game and the CPU  horsepower can play the game.  But what if it is targeted at the mid point of the typical North American and European user?

Working against those same statistics provided on internet population (http://www.nua.ie/surveys/how_many_online/), 62% of Internet users are in North America and Europe.   Lets work on the assumption that a mere 60% of the consumers meet the hardware requirement.

On all other points - these percentages might fluctuate *UPWARDS* based on the demographics (better ability to pay, more likely to download big packages, etc), but lets leave these the same for the sake of discussion.

Using your same calculations with the appropriate modifications the net results show that you've more than doubled the number of sales!  Of course that doesn't even account for the handful of sales outside of the market area.  

Of course all of this is based on broad assumptions and working aganst an equation that no one in the world can make accurate.  Then of course I always like that lovely quote from Mark Twain -- "There are three kinds of lies: lies, damned lies, and statistics."

As far as class libraries usually the best thing to do is to make your on jar manifest file (META-INF/MANIFEST.MF) where in you can specify classpaths to other jar files. The manifest also lets you control things like the base class to use and signed classes if your into that kind of thing.

Once its all done.  Pack all the libraries and your own code into the single jar.  Then like automagic you type 'java -jar myjar.jar' and everything just falls into place.

Look at
http://java.sun.com/j2se/1.3/docs/guide/jar/jar.html

Now a more complex question and one that has been bothering me is the question of the JRE itself.  Currently you either need to ship out your package and hope they have a JRE installed or make several distributables for each OS you want to "support", which include your package plus the JRE.  I've been thinking about how to sidestep this without going the InstallAnywhere route.

I've used java.util.zip.GZIPOutputStream to compress HTML data which was a totally off topic project.  Since many browsers can decompress that data it was a significant difference on throughput for slower connections.  Though it got tricky when exposed to various browsers and content-types.  But again thats off topic.

In terms of game data I'm not sure you would see the same level of results because it tends to be bytes with much less reoccurance than Strings.  Still an interesting idea and one I'm looking forward to trying.

If performance is at a premium its definitely worth writing your own.  Externalizable does still include the class header which typically adds about 20-30 bytes, the rest can be easily done as pure raw bytes.   You could write your own data handling methods, using a coded version of the packet type and save some of those bytes per packet over the Externalized packets.  

However, converting from Serializable to Externalizable is usually a fairly easy middle point before going to your own byte handlers and often gets the performance needed with a fraction of the work.  

In my experience Externalizable usually nets a significant improvement in processing time (in the range of 300-400% improvement in decomposing and rebuilding) and significant reduction in raw data (approximately 20-30% of the Serializable size) being pushed through the wire as compared to Serialization.

Umm, sorry...  As I said, "If you are only concerned about the server side", I figured you were concerned about the efficency of your code, not the efficency of your hardware.

I can give you no insight into the efficiency of your hardware and would specullate that it would require some very indepth understanding of the particullar hardware (like details of how the drivers are buffering data, etc) - which Java generally avoids due to cross-platform questions.

Is there a reason you are avoiding Externalizable objects as a lightweight protocol?  Its essentially Serializable, where you have to define the raw byte order used to disassemble and reassemble it.  Its much much lighter and faster than Serializable and built right in to the Java language.

Ah the old UDP verses TCP debate.   I don't currently work in the game biz, but did circa the early/mid 90's.  It was the dawn of making online games and where I worked making games Internet enabled was our thing.  We did a lot of research on using TCP verse UDP for games.  The hardware infrastructure is much better now but the same problems exist.

The simplest way to sum it all up is UDP will beat TCP for raw throughput performance but TCP will usually beat UDP for reliability.    In the old days the rules were fairly simple.  You used TCP if game state was important (lockstep) and you used UDP if latency was a premium and the game was wiggly enough to have things maybe be inconsistant at times.   The rules are significantly more complicated now, but the basic premise is still there.

The best stuff I've seen is some very interesting layers on top of UDP that essentially replicated some TCP functionality but effectivlely did 'read ahead' and 'gap filling' that normal TCP is not designed for - in those cases UDP was nearly as reliable as TCP but much better performing.  I'm sure things are much more developed now.

Serialization is a beast for performance and should never be thought of as a means to do something like this rapidly.  

One relatively simple solution is to convert to Externalizable instead of Serializable. Of course then you need to do the fine grained grunt work of knowing whats being put in and in what order.  But its still relatively easy (albeit tedious work).

Secondly - IMHO an even simpler way I to find the size of a data chunk is to use put a convert it into byte[] (using ByteArrayOutputStream and ByteArrayInputStream) and look at then System.err the lengths.  Its alot easier to watch a bunch of packets that way then writing to a file (and a lot faster and no mess to clean up too).  You can also add sequence and millis and a few other things to your output and do some more indepth profiling.

If you are only concerned about the server side its as simple as

long start = System.currentTimeMillis();
//do whatever
long duration = System.currentTimeMillis() - start;

Of course this can get into issues with the the granularity of the real time clock when you are measuring really small numbers.

Functionally zip and jar are the same but there are some minor differences to note.  

1.  Both use zlib and make more or less the same kind of files.  However if you want to retain cross platform portability you need to take care in making permutations to the compression.  Not all platforms come with the same ability to decompress more dense compression formats.  Jar uses fairly conservative compression to maintain cross platform portability.

2.  Minor point but jar by default adds a /META-INF and manifest file.  

As a purely hypothetical situation this would work (assuming myObject was intialized), but could create very strange situations.

The monitor will be on the original object, not the new object, therefore locks can occur on the new object just as if there was no lock.  One would assume the reason for the lock was to prevent concurrent mutation of the object but that is not accomplished.  Its conceivable that the object is locked by a second thread while a first thread is still in the synchronized block, which creates a race condition I've never really considered.

But beyond the theoretical question, obfuscating code, and the potential problems, there doesn't seem to be any particullar efficency gained by taking this approach.  

System.currentTimeMillis() most likely uses the Real Time Clock (RTC) which is often not handled by the CPU itself but a daughter component.  This is NOT the same as the systems internal clock cycles which are susceptible to skew based on load.  

Intel platforms typically use a RTC that has a 19ms interval, so things which look use this will not show any difference between System.currentTimeMillis() calls within 19ms.  That is to say that if you time two events that take less than 19ms, it is likely you will show zero time used.