Java-Gaming.org    
Featured games (79)
games approved by the League of Dukes
Games in Showcase (477)
Games in Android Showcase (106)
games submitted by our members
Games in WIP (533)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  how to get these bytes over there  (Read 3611 times)
0 Members and 1 Guest are viewing this topic.
Offline markuskidd

Junior Member


Medals: 1



« Posted 2003-01-25 00:04:44 »

I'm preparing to make the conversion from Serialization to my own lightweight "protocol," which will initially be on top of TCP (may move to datagrams at some point in the future if there seems to be a need). In order to facilitate this changeover, I am going to have all of my classes which currently implement the Serializible interface to implement this:

1  
2  
3  
4  
5  
public interface ActiveSerializable {

      public void fillByteBuffer(ByteBuffer in);
      public void readByteBuffer(ByteBuffer in);
}


Basically, the class uses fillByteBuffer to call in.putXXX() for each of its members and conversely calls in.getXXX() in readByteBuffer(). Assuming that I have an active TCP connection to my client, what's the most efficient way to get this data through the Socket? I'm assuming that I should just call ByteBuffer.array() and then pop that through my OutputStream, but how do I delimit the messages? Anything else I need to watch out for?
Offline leknor

Junior Member




ROCK!!!


« Reply #1 - Posted 2003-01-25 05:02:38 »

Quote
how do I delimit the messages?

I'm fond of the TLV format which I first read at the end of this page: http://aimdoc.sourceforge.net/OSCARdoc/section2.html
Quote
TLVs are a very convenient and efficient method of putting data into an organized format, esp variable length strings, etc. TLV litterally stands for "Type, Length, Value". And that's exactly what it is: a 16bit Type code, a 16bit value for the length of the Value field, and then the actual data in the Value field (variable length).

It's simple, flexable, nestable, and forward compatiable.
Offline elias

Senior Member





« Reply #2 - Posted 2003-01-25 13:06:24 »

If you're already working with ByteBuffers, I'd recommend to go all the way and use channels (java.nio) for your tcp conncetions. That way you avoid buffer.array() and the like. nio code might even save you a buffer copy.

It all depends on your performance needs of course.

Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline markuskidd

Junior Member


Medals: 1



« Reply #3 - Posted 2003-01-25 18:49:13 »

Say that my intended design is this: each incoming Socket from my ServerSocket is encapsulated in a Session class, which has a process() method that is invoked during every run loop of my networking thread. This process function checks to see if there is anything available() in the stream and, if so, categorizes it, checks out a GameEvent instance from my event pool, and puts it in a queue for the game logic thread.

What advantages would I gain from moving this to nio/channels?
Offline elias

Senior Member





« Reply #4 - Posted 2003-01-25 20:55:57 »

Other than performance and working with the most natural network API when you are already dealing with ByteBuffers, you get to drop your explicit network thread. By using non blocking networking you can check for incoming bytes in your gameloop without worrying about blocking.

Having used both the old stream interface and the new nio stuff, I actually think nio is easier to work with.

Offline markuskidd

Junior Member


Medals: 1



« Reply #5 - Posted 2003-01-26 17:43:42 »

I can't find that much material on working with NIO (maybe because it's still relatively new), but I did pull this prototype of non-blocking socket IO with Select from http://www.onjava.com/pub/a/onjava/2002/10/02/javanio.html?page=4


1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
ServerSocketChannel serverChannel = ServerSocketChannel.open();
Selector selector = Selector.open();

serverChannel.socket().bind (new InetSocketAddress (port));
serverChannel.configureBlocking (false);
serverChannel.register (selector, SelectionKey.OP_ACCEPT);

while (true) {
   selector.select();

   Iterator it = selector.selectedKeys().iterator();

   while (it.hasNext()) {
      SelectionKey key = (SelectionKey) it.next();

      if (key.isAcceptable()) {
         ServerSocketChannel server = (ServerSocketChannel) key.channel();
         SocketChannel channel = server.accept();

         channel.configureBlocking (false);
         channel.register (selector, SelectionKey.OP_READ);
      }

      if (key.isReadable()) {
         readDataFromSocket (key);
      }

      it.remove();
   }
}


Before I instigate any major refactoring to try out NIO, is this safe to work with as my baseline? Am I going to run into trouble by instantiating an iterator during each loop like that?
Offline elias

Senior Member





« Reply #6 - Posted 2003-01-26 18:02:03 »

You won't run into trouble, because you won't be iterating on the client, only on the server side. Servers are much less troubled by small gc hitches that clients are.

- elias

Offline coilcore

Senior Newbie




Java games rock!


« Reply #7 - Posted 2003-03-21 03:54:52 »

Is there a reason you are avoiding Externalizable objects as a lightweight protocol?  Its essentially Serializable, where you have to define the raw byte order used to disassemble and reassemble it.  Its much much lighter and faster than Serializable and built right in to the Java language.
Offline shawnkendall

Senior Member





« Reply #8 - Posted 2003-03-21 11:08:46 »

Externalizable is lighter than the regular Serializable but I do believe it still has:
* Class of the object
* Class signature
in the stream.
In many cases that can be bigger than you object data, making the overhead still annoyingly high.  Often a message or command passing system is a faster/cleaner/more-efficient way to get "events" around the network than object serialization.

Object serialization is great for "saving" objects or object state, but might not be the best way to communicate game state changes in a networked game. (IMHO)

Shawn Kendall
Cosmic Interactive, LLC
http://www.facebook.com/BermudaDash
Offline coilcore

Senior Newbie




Java games rock!


« Reply #9 - Posted 2003-03-21 18:39:00 »

If performance is at a premium its definitely worth writing your own.  Externalizable does still include the class header which typically adds about 20-30 bytes, the rest can be easily done as pure raw bytes.   You could write your own data handling methods, using a coded version of the packet type and save some of those bytes per packet over the Externalized packets.  

However, converting from Serializable to Externalizable is usually a fairly easy middle point before going to your own byte handlers and often gets the performance needed with a fraction of the work.  

In my experience Externalizable usually nets a significant improvement in processing time (in the range of 300-400% improvement in decomposing and rebuilding) and significant reduction in raw data (approximately 20-30% of the Serializable size) being pushed through the wire as compared to Serialization.
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline leknor

Junior Member




ROCK!!!


« Reply #10 - Posted 2003-03-21 20:14:11 »

Quote
In my experience Externalizable usually nets a significant improvement in processing time (in the range of 300-400% improvement in decomposing and rebuilding) and significant reduction in raw data (approximately 20-30% of the Serializable size) being pushed through the wire as compared to Serialization.
Have you used java.util.zip.GZIPInputStream and java.util.zip.GZIPOutputStream in any of your networking? I'm curious what effect it has on performance.
Offline coilcore

Senior Newbie




Java games rock!


« Reply #11 - Posted 2003-03-22 15:55:16 »

I've used java.util.zip.GZIPOutputStream to compress HTML data which was a totally off topic project.  Since many browsers can decompress that data it was a significant difference on throughput for slower connections.  Though it got tricky when exposed to various browsers and content-types.  But again thats off topic.

In terms of game data I'm not sure you would see the same level of results because it tends to be bytes with much less reoccurance than Strings.  Still an interesting idea and one I'm looking forward to trying.
Offline gregorypierce

Senior Member




I come upon thee like the blue screen of death....


« Reply #12 - Posted 2003-03-23 01:26:15 »

It also depends entirely on what your performance shows. Compression of packets also means that the server has to decompress packets (unless its just a traffic cop routing messages blindly to everyone else). If you have a very chatty protocol you will quickly discover that you will spend an obscene amount of time compressing traffic on the client and decompressing traffic on the server so you can generate a response and compress it and send it to your clients who much then decompress it.

I've found that compression didn't help me much because I had gone out of my way to keep my packets relatively small in order to avoid them being split into multiple packets and potentially lost or damaged due to UDP transmissions or waiting for TCP to get the larger packets back together again.

http://www.gregorypierce.com

She builds, she builds oh man
When she links, she links I go crazy
Cause she looks like good code but she's really a hack
I think I'll run upstairs and grab a snack!
Offline jbanes

JGO Coder


Projects: 1


"Java Games? Incredible! Mr. Incredible, that is!"


« Reply #13 - Posted 2003-03-23 03:28:03 »

Latency would probably be a big problem as well. GZip compression works by scanning blocks of data for similar data chunks. The compression algo is most likely to hold onto your messages until it has enough to create an entire block of data to transmit. Even if it does transmit immediately (via poor compression methods), the receiving end will still need to wait for the translation table to be written out in order to begin decompression.

That being said, compression can possibly help with relatively high-bandwidth data such as real-time voice communications. The one gotcha there is that you'll need as much control over the compression process as possible. Since you generally have between one and two milliseconds for each frame of animation, you'll most likely need a progressive compressor that can be preempted at any time. (A concession to the real-time nature of video games)

Edit: Sorry, that should read 10-20 ms.

Java Game Console Project
Last Journal Entry: 12/17/04
Offline leknor

Junior Member




ROCK!!!


« Reply #14 - Posted 2003-03-23 03:58:38 »

Quote
The compression algo is most likely to hold onto your messages until it has enough to create an entire block of data to transmit.
Yea, I assumed that too but then I saw the GZIPOutputSream.finish() method in addition to the OutputStream.flush() method which I though may solve the buffering problems. Maybe I should just use them and see what happens.
Offline jbanes

JGO Coder


Projects: 1


"Java Games? Incredible! Mr. Incredible, that is!"


« Reply #15 - Posted 2003-03-23 04:19:24 »

Quote

Yea, I assumed that too but then I saw the GZIPOutputSream.finish() method in addition to the OutputStream.flush() method which I though may solve the buffering problems. Maybe I should just use them and see what happens.


Unless you have higher-bandwidth data, you're probably going to *increase* the size of the packets from having to send the translation table. Of course, if your game works by sending complete game-data snapshots every half-second, you might get a decent bang. (Apparently, this method does work for RTS type games. Never tried it myself tho.) The ptoblem of course, is that you're probably already tuning your data to be as small as possible. Given that, there's most likely little that the compressor can do.

Java Game Console Project
Last Journal Entry: 12/17/04
Pages: [1]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

pw (26 views)
2014-07-24 01:59:36

Riven (25 views)
2014-07-23 21:16:32

Riven (20 views)
2014-07-23 21:07:15

Riven (22 views)
2014-07-23 20:56:16

ctomni231 (51 views)
2014-07-18 06:55:21

Zero Volt (46 views)
2014-07-17 23:47:54

danieldean (37 views)
2014-07-17 23:41:23

MustardPeter (40 views)
2014-07-16 23:30:00

Cero (57 views)
2014-07-16 00:42:17

Riven (55 views)
2014-07-14 18:02:53
HotSpot Options
by dleskov
2014-07-08 03:59:08

Java and Game Development Tutorials
by SwordsMiner
2014-06-14 00:58:24

Java and Game Development Tutorials
by SwordsMiner
2014-06-14 00:47:22

How do I start Java Game Development?
by ra4king
2014-05-17 11:13:37

HotSpot Options
by Roquen
2014-05-15 09:59:54

HotSpot Options
by Roquen
2014-05-06 15:03:10

Escape Analysis
by Roquen
2014-04-29 22:16:43

Experimental Toys
by Roquen
2014-04-28 13:24:22
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!