Java-Gaming.org Hi !
Featured games (83)
games approved by the League of Dukes
Games in Showcase (523)
Games in Android Showcase (127)
games submitted by our members
Games in WIP (592)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  NIO problem sending data when multiple requests in the queue  (Read 3001 times)
0 Members and 1 Guest are viewing this topic.
Offline zidsal

Senior Newbie





« Posted 2014-03-15 12:24:29 »

I have my networking code set up so if we wish to send data we simply add it to the queue and mark the selectionkey for OP_WRITE

1  
2  
3  
4  
5  
6  
7  
8  
   public void send( byte[] message) throws IOException{
      if(this.socketChannel == null){
         throw new SocketException("Socket connection has closed!");
      }
     
      messages.add(message);
      selectionKey = selectionKey.interestOps(SelectionKey.OP_WRITE);
   }


in my handleWrite() method I simply loop over all the data we find in the queue and send it

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
   public void handleWrite(SelectionKey key) throws IOException
   {
      this.socketChannel = (SocketChannel) key.channel();
     
                //error checking removed to conserve space
     
      while (!messages.isEmpty()) {
         writeBuffer = ByteBuffer.wrap(messages.peek());
         
         if(socketChannel.write(writeBuffer) == 0){
            break;
         }
           
         if (writeBuffer.remaining() > 0) {
            break;
         }
         writeBuffer.compact();
         messages.remove(); //everything went well so remove head
      }
     
      selectionKey.interestOps(SelectionKey.OP_READ);      
   }


in my read method I simply take the bytes I get and transform it into a packet object

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
   public Packet handleRead(SelectionKey key, PacketFactory factory) throws IOException{
     
      this.socketChannel = (SocketChannel) key.channel();
     
           //error checking removed to conserve space
     
      int bytesRead = 0;
      readBuffer.clear();
     
      try {
         bytesRead = this.socketChannel.read(readBuffer);
         readBuffer.flip();
      } catch (IOException e) {
         log.message("connection closed at: " + this.socketChannel.getRemoteAddress(), Log.INFO);
         key.cancel();
         close();
      }
     
      //error checking removed to conserve space
      byte[] data = readBuffer.array();
   
      return factory.buildPacket(transformer.getType(data), transformer.getData(data));
   }


my problem arises from the fact of when there are mutliple messages in the queue. So lets say I have requested to send a packet 3 times [3]:FOO:[3]:BAR, [3]:FOO:[3]:BAR, [3]:FOO:[3]:BAR. I would expect my read to be called 3 times and return 3 FOO packets with each one having the arguments BAR. Instead it returns 1 big packet FOO with the arguments BAR,FOO,BAR,FOO,BAR.

whats the best way to split this up? Have I done something wrong with my reading and writing or is this behavour to be expected? should I instead change my protocol so packets have a termination character so we know when a new packet begins?
Offline TeamworkGuy2

Junior Devvie


Medals: 10



« Reply #1 - Posted 2014-03-15 14:32:12 »

Looks like you've got a good understanding of how socket channels work.  Took me weeks to understand them Wink
Yes, if you use TCP (Socket, SocketChannel) the underlying OS/socket protocol stack will arbitrarily decide when to send a packet, you can somewhat affect this by calling flush if you are using a Socket, but I'd have to do some research.
The JavaDoc for OutputStream's flush() method mentions:
Quote
If the intended destination of this stream is an abstraction provided by the underlying operating system, for example a file, then flushing the stream guarantees only that bytes previously written to the stream are passed to the operating system for writing; it does not guarantee that they are actually written to a physical device such as a disk drive.

However, this doesn't really apply to SocketChannel's read() or write() method.  So I'm not sure that there are any guarantees about the behavior of either method.

If you use a DatagramSocket or DatagramChannel and keep the data you write under the maximum UDP transmission size, then the write methods will almost always send one packet per each write() method call, although it may vary from OS to OS.

So yes, the behavior you are describing is expected.  One way around it is to use a special marker byte/character between packets.
However, if the data in your packets might contain that special marker, then you will need to use an escape marker for your special marker to differentiate between the end of a packet and a marker that happens to be data in the packet.

Another solution is to include the length of the packet at the beginning of each packet, this way you don't need any special markers or escape markers.  Just read the length whenever you receive data and keep reading until number-of-bytes-read == length, then read the next packet's length and so on.
Offline Riven
« League of Dukes »

« JGO Overlord »


Medals: 832
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #2 - Posted 2014-03-15 17:13:20 »

Additionally, no matter how packets leave a computer, the routers and switches in between are at liberty to split/merge packets as they see fit. Never make any assumptions on packet boundaries.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline zidsal

Senior Newbie





« Reply #3 - Posted 2014-03-15 17:59:50 »

ok thanks for the details guys. I've decided to add a termination character to the end of each packet so I can detect if I have multiple packets being sent to the read.
Offline jonjava
« Reply #4 - Posted 2014-03-15 19:26:20 »

You could also additionally or instead put the length of the packet at the start so you know when the packet has been read fully and prepare for the next packet. This has the benefit of you knowing when to stop reading and e.g. check if the packet length is something ridiculously large and skip it.

Offline Riven
« League of Dukes »

« JGO Overlord »


Medals: 832
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #5 - Posted 2014-03-15 21:06:05 »

you can't skip packets without reading them...

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline jonjava
« Reply #6 - Posted 2014-03-15 21:33:49 »

Yes, you're right. But you can avoid allocating memory for erroneous packets. I guess it makes more sense if you have some sort of packet identifier.

Offline Riven
« League of Dukes »

« JGO Overlord »


Medals: 832
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #7 - Posted 2014-03-15 22:23:46 »

Upon my recommendation (claim to fame!), Nate used chucked encoding in Kryo. This allows streaming of data without knowing the packet length ahead of time, and equally reading back the stream without holding it in memory.

With chunked encoding you basically say that the current message has N more bytes, and after N bytes you send the size of the next chunck. A chuck size of 0 signals the end of the message.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Pages: [1]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

SHC (24 views)
2014-11-25 12:00:59

SHC (24 views)
2014-11-25 11:53:45

Norakomi (22 views)
2014-11-25 11:26:43

Gibbo3771 (22 views)
2014-11-24 19:59:16

trollwarrior1 (36 views)
2014-11-22 12:13:56

xFryIx (74 views)
2014-11-13 12:34:49

digdugdiggy (52 views)
2014-11-12 21:11:50

digdugdiggy (46 views)
2014-11-12 21:10:15

digdugdiggy (41 views)
2014-11-12 21:09:33

kovacsa (69 views)
2014-11-07 19:57:14
Understanding relations between setOrigin, setScale and setPosition in libGdx
by mbabuskov
2014-10-09 22:35:00

Definite guide to supporting multiple device resolutions on Android (2014)
by mbabuskov
2014-10-02 22:36:02

List of Learning Resources
by Longor1996
2014-08-16 10:40:00

List of Learning Resources
by SilverTiger
2014-08-05 19:33:27

Resources for WIP games
by CogWheelz
2014-08-01 16:20:17

Resources for WIP games
by CogWheelz
2014-08-01 16:19:50

List of Learning Resources
by SilverTiger
2014-07-31 16:29:50

List of Learning Resources
by SilverTiger
2014-07-31 16:26:06
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!