Java-Gaming.org    
Featured games (79)
games approved by the League of Dukes
Games in Showcase (477)
Games in Android Showcase (107)
games submitted by our members
Games in WIP (534)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  Idea for a packet system  (Read 3181 times)
0 Members and 1 Guest are viewing this topic.
Offline CyanPrime
« Posted 2011-05-11 08:00:27 »

Now what I was thinking of doing was having a byte[] buffer, and then filling it with ints floats, and booleans somehow.

I figure I could read the next variable by saying "buffer << sizeOf(int)" or something. would this work well? and does anyone have any idea how I could get the data to go IN the buffer?
Offline counterp

Senior Member


Medals: 11



« Reply #1 - Posted 2011-05-11 08:40:44 »

that's what buffers do already.

is this what you're looking for? (very simple, just added the methods you mentioned)

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  
42  
43  
44  
45  
46  
47  
48  
49  
50  
51  
52  
53  
54  
55  
56  
57  
58  
59  
60  
61  
62  
63  
64  
65  
66  
67  
68  
69  
70  
71  
72  
73  
74  
75  
76  
77  
78  
79  
80  
81  
82  
83  
public class Buffer {

   public Buffer() {
      this(128);
   }

   public Buffer(int size) {
      this.size = size;
      buffer = new byte[size];
   }

   private final int size;

   private byte[] buffer;
   private int pos, limit;

   public boolean getBoolean() {
      return buffer[pos++] == 1;
   }

   public float getFloat() {
      return Float.intBitsToFloat(getInt());
   }

   public int getInt() {
      return (buffer[pos++] & 0xFF) << 24 | (buffer[pos++] & 0xFF) << 16 | (buffer[pos++] & 0xFF) << 8 | (buffer[pos++] & 0xFF);
   }

   public int limit() {
      return limit;
   }

   public int position() {
      return pos;
   }

   public int remaining() {
      return limit - pos;
   }

   public Buffer putBoolean(boolean bool) {
      ensure(1);
      buffer[pos++] = (byte) (bool ? 1 : 0);
      return this;
   }

   public Buffer putFloat(float f) {
      return putInt(Float.floatToRawIntBits(f));
   }

   public Buffer putInt(int i) {
      ensure(4);
      buffer[pos++] = (byte) (i >> 24);
      buffer[pos++] = (byte) (i >> 16);
      buffer[pos++] = (byte) (i >> 8);
      buffer[pos++] = (byte) i;
      return this;
   }

   public void clear() {
      pos = limit = 0;
      buffer = new byte[size];
   }

   private void ensure(int size) {
      int required = pos + size;
      if (buffer.length <= required) {
         byte[] temp = new byte[required + 64];
         System.arraycopy(buffer, 0, temp, 0, buffer.length);
         buffer = temp;
      }
   }

   public void flip() {
      limit = pos;
      pos = 0;
   }

   public void rewind() {
      pos = 0;
   }

}
Offline loom_weaver

JGO Coder


Medals: 17



« Reply #2 - Posted 2011-05-11 13:33:50 »

Why re-invent the wheel?  See DataInputStream and DataOutputStream.
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline counterp

Senior Member


Medals: 11



« Reply #3 - Posted 2011-05-11 14:45:51 »

Why re-invent the wheel?  See DataInputStream and DataOutputStream.

There's a huge difference between writing to/reading from a stream and writing/reading buffered data.

You should have compared this to a ByteBuffer not DataInputStream/DataOutputStream.

Even then, this still has speed optimizations.

So it's not reinventing the wheel.
Offline ra4king

JGO Kernel


Medals: 337
Projects: 2
Exp: 5 years


I'm the King!


« Reply #4 - Posted 2011-05-11 17:59:03 »

What's the point of the variables "size" and "limit" in your code? They are never used....?

Offline DzzD
« Reply #5 - Posted 2011-05-11 18:46:13 »

easier to use and/or more easy to manage

1  
2  
3  
4  
5  
6  
7  
ByteArrayOutputStream bos=new ByteArrayOutputStream();
DataOutputStream dos=new DataOutputStream(bos);
dos.writeSomething();
bytes=bos.toByteArray()
>> send packet id
>> send packet size
>>send size bytes over network


and

1  
2  
3  
4  
5  
6  
>> read paket id
>> read packet size
>>receive size bytes from network
ByteArrayInputStream bais=new ByteArrayInputStream(bytes);
DataInputStream dis =new DataInputStream(bais);
dis.readSomething();




Also in a base/superclass Packet you should have at least
1  
2  
int packetId 
int packetSize


and some kind of packets identification :
1  
2  
3  
public final static int PACKET_PING = 0x1;
public final static int PACKET_START = 0x2;
etc...


your stream then should look something  like :
PACKET_ID,PACKET_SIZE,PACKET_CONTENT,PACKET_ID,PACKET_SIZE,PACKET_CONTENT,PACKET_ID,PACKET_SIZE,PACKET_CONTENT, etc..

Offline counterp

Senior Member


Medals: 11



« Reply #6 - Posted 2011-05-12 01:36:59 »

What's the point of the variables "size" and "limit" in your code? They are never used....?

size is used to reset the internal byte array to the original size after clearing (check the clear() method).

and limit is important, it indicates at what position the buffer was at after it was flipped. it let's you know how much data is available to be read (check the remaining() method)
Offline ra4king

JGO Kernel


Medals: 337
Projects: 2
Exp: 5 years


I'm the King!


« Reply #7 - Posted 2011-05-12 02:41:26 »

I have seen the flip() also used in the NIO package and I could never understand its use. All I see here is the same as rewind except for setting limit to pos. That's it or am I missing something...?

Offline counterp

Senior Member


Medals: 11



« Reply #8 - Posted 2011-05-12 03:31:55 »

Yes the same thing happened to me when I first saw the flip() method, I had no idea really what it did. But I eventually figured it out after asking people Wink

Basically, it flips from writing to reading.

So you have this example code:

1  
2  
3  
Buffer buffer = new Buffer();
buffer.putInt(100); // Integer consists of 4 bytes, therefore the position of the internal byte array is now at 4
System.out.println(buffer.getInt()); // Begin reading from the 4th byte onward, because the position is still at 4 after writing. You are not reading at the correct position this is INCORRECT!


This will not print 100. Why? Because the buffer's position is not set to the beginning.

This is what it should be:

1  
2  
3  
4  
Buffer buffer = new Buffer();
buffer.putInt(100); // Integer consists of 4 bytes, therefore the position of the internal byte array is now at 4
buffer.flip(); // Set the position back to 0. Set limit to 4 (old position)
System.out.println(buffer.getInt()); // Before this is read, the remainder() is 4. After this is read the remainder is 0, because the position is increased by 4 since you are reading an integer (4 bytes). You begin reading from 0 because the position was reset to 0 (which is where you wrote the integer 100, the first 4 bytes in the byte array)


Once you flip it, the position is set back to 0 (so you can start reading from the beginning instead of where the position was left off when you wrote the integer 100) and the limit becomes the position.

Then you can use the remainder() method to see how many more bytes can be read. Without flipping, there is no limit, and therefore the remainder will not be accurate.
Offline ra4king

JGO Kernel


Medals: 337
Projects: 2
Exp: 5 years


I'm the King!


« Reply #9 - Posted 2011-05-12 03:35:57 »

Oh so with these buffers, you can only read after you have written everything you needed? If you call putInt(100) after you call flip(), it will overwrite the first 4 values of the array? Why can't the buffer just be a FIFO queue?

Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline counterp

Senior Member


Medals: 11



« Reply #10 - Posted 2011-05-12 04:17:56 »

It's not realistic to use any such implementations like a queue, one of the main priorities of a buffer is to be fast as possible. And it would be much more complicated...
Offline DzzD
« Reply #11 - Posted 2011-05-12 10:49:49 »

It's not realistic to use any such implementations like a queue, one of the main priorities of a buffer is to be fast as possible. And it would be much more complicated...
it wont be slower, you can use things like two "rotative index", one fore reading and another for writing, but in this case care must be taken and you have to ensure flow control.

pseudo code :
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
byte readByte()
{
 return bytes[readIndex++%bytes.length];
}

void writeByte(byte ) throw OverflowException
{
 if(writeIndex==readIndex)
  throw new OverflowException();
 bytes[writeIndex++%bytes.length]=byte;
}


once you can read/write one byte you can implement all other method writeFloat/writeInt etc...

Offline ra4king

JGO Kernel


Medals: 337
Projects: 2
Exp: 5 years


I'm the King!


« Reply #12 - Posted 2011-05-12 12:47:07 »

@DzzD
My thoughts exactly Smiley

Offline counterp

Senior Member


Medals: 11



« Reply #13 - Posted 2011-05-12 17:19:01 »

You should probably stay away from wrapping around and overwriting data (unless you're positive you will never read over a certain amount like you said)

And I'm not sure what kind of design you will be using where you need to read and write at the same time..
Offline ra4king

JGO Kernel


Medals: 337
Projects: 2
Exp: 5 years


I'm the King!


« Reply #14 - Posted 2011-05-12 17:25:39 »

I have created my own "Packet" system that uses ObjectInputStream and ObjectOutputStream. I'd say it's quite efficient and easy Smiley

Offline DzzD
« Reply #15 - Posted 2011-05-12 19:19:31 »


And I'm not sure what kind of design you will be using where you need to read and write at the same time..
I can think of three possibilities :
 1 - hang on read until data are available
 2 - throw NoDataAvailableException
 3 - return a special value indicating that no data is available (this one could be weird to manage)

Offline counterp

Senior Member


Medals: 11



« Reply #16 - Posted 2011-05-12 21:10:47 »


And I'm not sure what kind of design you will be using where you need to read and write at the same time..
I can think of three possibilities :
 1 - hang on read until data are available
 2 - throw NoDataAvailableException
 3 - return a special value indicating that no data is available (this one could be weird to manage)

Well 2 and 3 can be done with any standard Buffer (after all, you know how much is remaining).

As for 1, it's true that with this implementation that can be done... But I really don't see a purpose for not reading fully and writing additional data :p

For the games I've written, I process all data when it's available, I don't wait for more things to be written.
Offline DzzD
« Reply #17 - Posted 2011-05-12 21:32:39 »

Quote
But I really don't see a purpose for not reading fully and writing additional data :p

having a thread that do the network job asynchronously and give you only fullyfilled packets is very practice & efficient and hanging on no data avaialble is the best for such architecture (inded you have to make it thread safe), in your mainloop you can do things like :

1  
2  
3  
4  
5  
6  
7  
8  
if(networkManager.isPacketAvailable())
{
 Packets packets=networkManager.popPackets();
 foreach(packet in packets)
 {
  processPacket(packet)
 }
}


it is very important to process network asynchronously especially for low connection, if you get first byte of a packet (wich is for example 50 bytes long) and try to read it fully synchronously you may hang up to 6.25ms (50/8000) on a 80Kb connection (best case only if everything goes well), also by using a separated thread the requiered buffer dont have to be that big because you can read ASAP new arriving byte on network an push them into the right packet (wich is very good to avoid network conjonction ), finally on your mainloop you only have to ask all fullfilled packet and threat them, no risk of uncomplete packet and for "simplicity of your project" packet validity/discarding/flowcontrol can be done in the network thread

Offline counterp

Senior Member


Medals: 11



« Reply #18 - Posted 2011-05-12 23:51:59 »

Ehh, I stay away from IO, too much grief, I use NIO. No wait time for reading <3 data buffered up before it's sent to my custom buffer so no fragmented packets either.

I've noticed a lot of people here tend to stay away from NIO, I wonder why? It's very elegant..
Offline ra4king

JGO Kernel


Medals: 337
Projects: 2
Exp: 5 years


I'm the King!


« Reply #19 - Posted 2011-05-13 01:20:21 »

Hehe I tend to stay away from NIO, IO is easier Cheesy

Offline counterp

Senior Member


Medals: 11



« Reply #20 - Posted 2011-05-13 01:58:09 »

Yeah, that's true Tongue

It is easier, but it's really only suitable for smaller servers
Offline DzzD
« Reply #21 - Posted 2011-05-13 07:50:56 »

I ve manage to have more than 100 players playing realtime game together on the same server without optimizing anything and without loading it that much, could theorically handle a lot more, I use standard IO because It was the one who look the most familiar to me, the most similar to the one I used in C in the past, but probably the only reason is I just  never realy ask myself about that Smiley

Offline counterp

Senior Member


Medals: 11



« Reply #22 - Posted 2011-05-13 19:32:15 »

Ehh, I'm familiar with IO, but I rarely use except when I'm testing things networking related since it's quicker to code using IO.. My game server should theoretically hold over 2000 players (must have a decent internet though :p, like on a dedicated server or vps)

For 100 players a thread-per-client model with IO works well Smiley
Offline DzzD
« Reply #23 - Posted 2011-05-13 20:48:55 »

...2000 players (must have a decent internet though :p, like on a dedicated server or vps)

For 100 players a thread-per-client model with IO works well Smiley
100 players is just the tiny part of targeted audience (it is only a real test case we was able to measure on), number of attended visitor is lot bigger (can be up to thousands of user, the leader in what we target had up to 300000 players on official races and we got a better product and we begin to have official races, so we cross fingers), we have built a very extensible architecture cpu/network/ram, "theorically" we can handle those number of connection, principe is pretty simple and have shown to work very well on all tests we was able to do/simulate.

in sumary we got a big server that do all logic using multithread on component based system :
 1 - perform task 1 on n threads
 2 - perform task 2 on n threads
 3 - perform task 3 on n threads
etc....

this enable to switch to 4/8/16/32/etc... cores depending on server charge

and for the network part, clients does not connect directly to the root game server, rather we used fronts server that act as proxies :
 - discard invalid packets,
 - respond to some of them (not impling any logic)
 - improve security / act as firewall, etc...
 
those front server are cheap and can be rent/installed and connected to the root one within arround one hour (and inded removing pretty quickly),
this way we can make our bandwith and number of possible connection grow/shrink depending on the charge, also interresting economicaly as we pay only the requiered bandwith at a given time.

we can handle arround 500 players on one front server and this front server only make one connection to the root server (multiplexing/demultiplexing packets),
if we want to be able to have 500 connections more we juste have to add another front server (nb connection depend on the server spec) and etc...

this can be done until the root server cannot handle anymore front server / connection, and alternativly we can branch biggers fronts server


for example with only 10 fronts servers that would each enable 500 connections (once again 500 really depend on servers spec)
we will enable 5000 players in a race and the root server will still only see 10 connections.

for now with a maximum of 100 players connected at once and arround 600/700 boats ingame we got only one small front server with a medium root server ( and both are just doing nothing, server load is very low :/ )

"the icing on the cake"  : client <=> frontal work over HTTP/1.1, yes that dont look serious hé ? but it enable a lot of player to play from their work even with strict proxy, basically we had redo some kind of stream/TCP over HTTP ..., note that even throught proxy/front an due to asynchronous threatment of HTTP stream, ping is usually arround 100 ms (wich is really enought for the kind of gameplay we target : 1s is still ok)

...all that to say that I love this architecture, yes Smiley ... but also that I keep in mind using NIO, I will investigate more "if" in the futur standard IO start to show some issues

Offline Icecore

Senior Member


Medals: 5



« Reply #24 - Posted 2011-05-19 22:34:07 »

This look like previus post so sorry if this don’t help;
Here is how I send and packed data maybe it looks weird but I do so:)
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
ByteArrayOutputStream=baos
       Pi=(int) 45645646;
// Float.floatToIntBits(Pi);
      a1=Pi>>24;
       a2=(Pi<<8)>>24;
       a3=(Pi<<16)>>24;
       a4=(Pi<<24)>>24;
       baos.write(a1);
       baos.write(a2);
       baos.write(a3);
       baos.write(a4);


byte ar[]=baos.toByteArray();
iterat=0;
a1=(char)(ar[iterat]&0xFF);
iterat++;
a2=(char)(ar[iterat]&0xFF);
iterat++;
a3=(char)(ar[iterat]&0xFF);
iterat++;
a4=(char)(ar[iterat]&0xFF);
iterat++;
int Pi=(a1<<24)+(a2<<16)+(a3<<8)+a4;
//Care it will be signed
// I use char for unsigned byte ;)
// Float.intBitsToFloat(Pi);
//long
                a1=  timee>>56;//send
                a2= (timee<<8)>>56;
                 a3= (timee<<16)>>56;
                 a4= (timee<<24)>>56;
                 a5= (timee<<32)>>56;
                 a6= (timee<<40)>>56;
                 a7= (timee<<48)>>56;
                 a8= (timee<<56)>>56;
         
timee=((long)a1<<56)+((long)a2<<48)+((long)a3<<40)+((long)a4<<32);//read
timee +=((a5<<24)+(a6<<16)+(a7<<8)+a8)& 0xFFFFFFFFL;
Pages: [1]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

pw (35 views)
2014-07-24 01:59:36

Riven (34 views)
2014-07-23 21:16:32

Riven (22 views)
2014-07-23 21:07:15

Riven (25 views)
2014-07-23 20:56:16

ctomni231 (56 views)
2014-07-18 06:55:21

Zero Volt (48 views)
2014-07-17 23:47:54

danieldean (39 views)
2014-07-17 23:41:23

MustardPeter (43 views)
2014-07-16 23:30:00

Cero (59 views)
2014-07-16 00:42:17

Riven (56 views)
2014-07-14 18:02:53
HotSpot Options
by dleskov
2014-07-08 03:59:08

Java and Game Development Tutorials
by SwordsMiner
2014-06-14 00:58:24

Java and Game Development Tutorials
by SwordsMiner
2014-06-14 00:47:22

How do I start Java Game Development?
by ra4king
2014-05-17 11:13:37

HotSpot Options
by Roquen
2014-05-15 09:59:54

HotSpot Options
by Roquen
2014-05-06 15:03:10

Escape Analysis
by Roquen
2014-04-29 22:16:43

Experimental Toys
by Roquen
2014-04-28 13:24:22
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!