Java-Gaming.org    
Featured games (81)
games approved by the League of Dukes
Games in Showcase (487)
Games in Android Showcase (112)
games submitted by our members
Games in WIP (553)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  server with NIO - sending data  (Read 5073 times)
0 Members and 1 Guest are viewing this topic.
Offline Kova

Senior Member





« Posted 2006-03-08 22:35:42 »

Hello.
So I've learned how to accept connections and read data when it comes (registered operations for accepting and reading). Now I'm a little bit confused about sending data. I want my server to send packet every 50 ms (or whatever is best). How do I do that? Huh
Problem is I don't understand how registring OP_WRITE works (didn't even tried it yet). Selector then notifies you every time channel is ready for writing? Dosen't make much sense since it is ready all the time from opening a channel. My inital thought is new thread for sending data, who would sleep for 50ms and then send data across the channel to every client connected. I would like it if it could be implemented without creating new thread, in the same thread for listening.
Thanks.
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 783
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #1 - Posted 2006-03-08 22:53:10 »

Yes, OP_WRITE works just like OP_READ, it tells you when the channel is ready.

It will be ready *very* often, unless the OS buffers are full.

If you have nothing to send, but the channel is writable, you can sleep(1) to prevent the thread consuming too much cpu-cycles.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline swpalmer

JGO Coder




Where's the Kaboom?


« Reply #2 - Posted 2006-03-09 00:27:31 »

If you have nothing to send the thread should probably block on a mutex that is protecting your list of outgoing packets or something like that.

Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 783
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #3 - Posted 2006-03-09 00:46:54 »

That would also block the OP_READ, as that's (probably) in the same thread.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline swpalmer

JGO Coder




Where's the Kaboom?


« Reply #4 - Posted 2006-03-09 03:35:40 »

Ah... well I tend to prefer proper blocking as opposed to spinning in a loop with nothing to do... sleep(1) is more of hack ot make that tolerable than a properly designed solution to the problem.

But I'll shut up now, since I haven't used any of the NIO networking stuff  yet Smiley

Offline Kova

Senior Member





« Reply #5 - Posted 2006-03-09 17:45:36 »

So what do I do? Sending and reading data is extreamly fast, so I could just try to send the data, not testing readiness and if exception occurs I catch it, sleep 1ms and try again? It's a simple game with less then 50 bytes per packet, but that isn't a solution also, not what I had in mind. In all tutorials out there this isn't mentioned, just sending data after you read it, like as response, and that certanly dosen't satisfy my needs for a server.
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 783
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #6 - Posted 2006-03-09 18:41:13 »

Do I/O in a seperate thread if you're using selectors (in a loop).

Otherwise just write to the channel (and check how many bytes were actually sent)
and just read from the channel (and be prepared to read 0 bytes very often)
this can be done in your main-loop, in asynchronous mode, that is.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Kova

Senior Member





« Reply #7 - Posted 2006-03-10 00:20:41 »

i know the basics... read the first post. What I don't know is how to send data properly like described in first post.
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 783
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #8 - Posted 2006-03-10 11:54:35 »

I answered your question in the last 3 lines. Roll Eyes

And yes, they are the basics, but you were asking for them. Roll Eyes

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Kova

Senior Member





« Reply #9 - Posted 2006-03-10 14:11:28 »

Do I/O in a seperate thread if you're using selectors (in a loop).

Otherwise just write to the channel (and check how many bytes were actually sent)
and just read from the channel (and be prepared to read 0 bytes very often)
this can be done in your main-loop, in asynchronous mode, that is.

I am using a selector, so all stuff after otherwise word, when reading, is considered not usable to me (or maybe my english is bad, but othrewise = in another case => if I'm not using selectors Undecided ).

To summerize, I am using select(), which blocks, for accepting and reading data in separate thread for network IO. I don't know how to implement writing in the same thread, couse if I register OP_WRITE, select() will activate all the time telling me channel is ready for writing... I just want to send data every 50ms, without having 1 mil useless selects in between.
So with my knowlage I can only think of one more thread for sending the data and sleeping for 50ms. Only potentional problem is can one thread send data at the same time other thread reads data from same channel? Could happen...
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 783
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #10 - Posted 2006-03-10 14:15:26 »

Quote
I am using a selector, so all stuff after otherwise word, when reading, is considered not usable to me

In your case you shouldn't.


Either you use selectors, and let the OS do the timing, or do you own timing, without selectors.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline whome

Junior Member




Carte Noir Java


« Reply #11 - Posted 2006-03-13 18:45:48 »

I think you can just write to ClientChannels without OP_WRITE opcode registrations. Some NIO servers use
* ThreadA to read selector for incoming packets. Packets are put to pending incoming actions queue.
* ThreadB removes actions from queue and process it. Outgoing packets are put to outgoing actions queue.
* ThreadC is run at constant rate to remove from outgoing queue and write packets to ClientChannels.

I think NIO SocketChannels are multithread safe. You can read packets in one thread and write in another. They sure synchronize stuff somewhere underneat but its not our side.

Or use single thread to handle OP_READ and OP_WRITE. Once you have outgoing packet you register SocketChannel with OP_WRITE code. Use your normal selector while loop to handle accept/read/write events. Soon after you have written a complete packet you can unregister OP_WRITE code. It should then not create uncecessary wakeups in a selector while loop.

To check for complete packet written, you must cumulate writtenBytes or use buffer.available() method to see when the position of write (byte)buffer is at the end.
Offline Kova

Senior Member





« Reply #12 - Posted 2006-03-14 00:15:59 »

Actually I had already written what I said I'll do... and so far it works.
I have one thread to accept connections and read data. When connection is accepted I put that channel in client list. Second thread is for sending only and it waits until client list has some connections, then it loops through them and send data through channels every second. So at the end I don't even need to bother with OP_WRITE as it seems, but It would be nice if someone confirms that thread that reads the data and thread that sends the data through same channel are synchronized in that, and not just the fact operations are fast so bug didn't happened to me yet.

I'm now having problems with client list (a HashSet) and synchronizing adding to the list in accept thread, with send thread that iterates through it and gets channels to send data through them. I just started with this so I'll post after some thinking and experimenting if I can't solve it.

One thing I didn't implement is queueing of actions, so far I just calculate buffer and write it directly. Is this important?
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 783
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #13 - Posted 2006-03-14 09:56:35 »

Yes, queueing is important as not all bytes in the ByteBuffer may have been written to the channel. So in case it was a partial send, you want that data into some kind of queue to be sent later when the OS buffers have space available again to write.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline whome

Junior Member




Carte Noir Java


« Reply #14 - Posted 2006-03-14 20:47:13 »

Kova: how do you handle incoming packets to determine the end of packet sent. Do you have a terminator byte or each packet has a lengthOfData value at teh start?

What I do, I have ByteArrayOutputStream attached to each SocketChannel as an attachment object. I write incoming bytes to a stream and check for terminator byte (I use NULL byte). When I have read null, I take bytes from a stream and create Packet object to be put on pending packet queue. Byte stream is then emptied. But this is tricky and must handle few special cases.

Test case:
# = terminator byte, ABC = msg1, CDE = msg2, EFG = msg3

Reading a SocketChannel may give me bytes belonging to 0,1 or more packets. It is possible to receive "ABC#CD" bytes in one go, so I must put leftside bytes to a byte array stream, create a completed Packet instance, clear stream and put rightside bytes to a stream because they belong to a next packet. Or it may give me "ABC#CDE#E" where I read _two_ completed packets in one go and a fragment of third packet.

All this is easy if you create a synchronized FIFO queue for incoming completed Packets. Action handler thread takes packets and put outgoing packets to another FIFO queue.

You won't see such special case if clients don't write data at high rate, but it is a pure luck not see it happen. I had a Flash client who did it constantly and I was puzzled few days why server lost or broke packets randomly.
Offline Kova

Senior Member





« Reply #15 - Posted 2006-03-14 22:59:51 »

Kova: how do you handle incoming packets to determine the end of packet sent. Do you have a terminator byte or each packet has a lengthOfData value at teh start?

I don't Smiley ... I just wanted to see if it can be done my way (2 threads, listener and sender) since nobody anwsered how to build a server logic. I read somewhere about null byte terminator, I'll try to implement that.

Test case:
# = terminator byte, ABC = msg1, CDE = msg2, EFG = msg3

Reading a SocketChannel may give me bytes belonging to 0,1 or more packets. It is possible to receive "ABC#CD" bytes in one go, so I must put leftside bytes to a byte array stream, create a completed Packet instance, clear stream and put rightside bytes to a stream because they belong to a next packet. Or it may give me "ABC#CDE#E" where I read _two_ completed packets in one go and a fragment of third packet.
...<cut>

Exactly what I had in mind, I better implement queue fast. Tell me, why separate thread for action handler? Can't queueing be arranged in thread you receive data? Like you read 1.5 packets, 1 packet goes to queue and half of other goes to temp buffer. Then next time you put temp + rest of it to queue as a whole packet.
Offline whome

Junior Member




Carte Noir Java


« Reply #16 - Posted 2006-03-15 15:54:18 »

Queing is meant to synchronize two separate threads to work along, thats how I understand it. If you use a thread who received a data as an action handler then you may block incoming op_read and op_accept operations if handler takes too much time. I use two threads in my server and FIFO queue is used to transfer packets between threads.
ThreadA: handle op_accept, op_read and op_write
ThreadB: handle completed packets and do what need to be done

ThreadB may run at constant rate to write data to remote clients constantly (like isAlive query or similar). You just take a packet from fifo queue each "while(isRunning)" step if one exists and handle it. But in my case I dont need to do anything if packets are not been sent, so ThreadB just idles until fifo queue notifies for new data.

Offline Kova

Senior Member





« Reply #17 - Posted 2006-03-15 16:41:22 »

Queing is meant to synchronize two separate threads to work along, thats how I understand it. If you use a thread who received a data as an action handler then you may block incoming op_read and op_accept operations if handler takes too much time. I use two threads in my server and FIFO queue is used to transfer packets between threads.

Yeah, handeling received data in same thread might delay reading operation, but since my game is small and I'll send packets every 50ms or something like that, I don't worry.
Offline blahblahblahh

JGO Coder


Medals: 1


http://t-machine.org


« Reply #18 - Posted 2006-03-15 17:36:03 »

Yes, OP_WRITE works just like OP_READ, it tells you when the channel is ready.

It will be ready *very* often, unless the OS buffers are full.

If you have nothing to send, but the channel is writable, you can sleep(1) to prevent the thread consuming too much cpu-cycles.

FYI that's completely the wrong thing to do, it's around 100 times slower than doing things properly (blocking on a mutex or, best of all, send what you need, then deregister).

If you want to send every X milliseconds, you should:

1. wait till the next 50'th ms
2. register for WRITE
3. send
4. repeat until all sent
5. de-register for WRITE
6. ... go and do other logic, then come back to 1. at the appropriate moment

malloc will be first against the wall when the revolution comes...
Offline Kova

Senior Member





« Reply #19 - Posted 2006-03-15 17:44:59 »

1. wait till the next 50'th ms
2. register for WRITE
3. send
4. repeat until all sent
5. de-register for WRITE
6. ... go and do other logic, then come back to 1. at the appropriate moment

How this solves puting send in same thread as read? You still have to sleep 50 ms in step 1 and miss incoming data. ...or I'm the one who missed something? Smiley
Any why registering for write? Isn't that a selector thing? Why not just write if channel is open?
Offline blahblahblahh

JGO Coder


Medals: 1


http://t-machine.org


« Reply #20 - Posted 2006-03-15 18:27:43 »

1. wait till the next 50'th ms
2. register for WRITE
3. send
4. repeat until all sent
5. de-register for WRITE
6. ... go and do other logic, then come back to 1. at the appropriate moment

How this solves puting send in same thread as read? You still have to sleep 50 ms in step 1 and miss incoming data. ...or I'm the one who missed something? Smiley

a. get time.
b. ...do reading stuff...
c. check how many millis left before need to do next write
d. sleep (that long)
e. go to step 2

Quote
Any why registering for write? Isn't that a selector thing? Why not just write if channel is open?

OP_WRITE is the only way of knowing that the channel is ready for being written to, that is the definition of it. If you write when its not ready, then either you'll get an exception (if you're lucky), or you'll write 0 bytes, or you'll overflow your bytes and think you've written them but not actually managed to. Your network card and OS only have a limited capacity of outgoing bytes Smiley.

malloc will be first against the wall when the revolution comes...
Offline Kova

Senior Member





« Reply #21 - Posted 2006-03-16 00:42:45 »

a. get time.
b. ...do reading stuff...
c. check how many millis left before need to do next write
d. sleep (that long)
e. go to step 2

That means that data can arrive when I'm sleeping and max response time of reading the data can be 50ms. Kind of don't like this approach... what if won't need send data so often in future, like it could be 500ms. Then reading would be unnecesary slow. I'll stick to multiple threads thing.

OP_WRITE is the only way of knowing that the channel is ready for being written to, that is the definition of it. If you write when its not ready, then either you'll get an exception (if you're lucky), or you'll write 0 bytes, or you'll overflow your bytes and think you've written them but not actually managed to. Your network card and OS only have a limited capacity of outgoing bytes Smiley.

tnx, good to know... although I kind of expected that OS or VM queues requests if it can't fit it all. Like TCP, it makes sure you get the packet, but it takes time.
Offline Kova

Senior Member





« Reply #22 - Posted 2006-03-17 00:20:34 »

I finally deceided not to bother with the thing right now and focus on getting that single connection that works usable. I'll come to this later when game will be in more completed stage.
Offline Kova

Senior Member





« Reply #23 - Posted 2006-03-18 18:26:49 »

more usefull stuff about writing to channel, I found this in www.codecomments.com:

Quote from: Thomas Hawtin, 2006-01-28, 3:57 am
Keep your key registered with the selector throughout. However you need
not be interested in OP_WRITE until you find that you can't write any
more. Just write your buffer out, and see if it fails to be copied out.
When the channel wont accept any more writes, only then do you need to
be notified when it is in a state for further writes.

Unless you have data that you can't written at the moment, then there is
no need to be notified of when you can write data. Selecting with
OP_WRITE will immediately return until you fill the relevant system buffers.

Similarly if you couldn't deal with reading any more data (because your
buffers are full, say), then you should not be interested in OP_READ. Or
if all your worker threads are tied up and you can't take another
connection, don't be interested in OP_ACCEPT.

As a rough approximation, you could write your code not to use OP_WRITE
at all (although it will obviously fail nastily when the relevant system
buffers fill up).
Offline blahblahblahh

JGO Coder


Medals: 1


http://t-machine.org


« Reply #24 - Posted 2006-03-18 20:08:47 »

That means that data can arrive when I'm sleeping and max response time of reading the data can be 50ms. Kind of don't like this approach... what if won't need send data so often in future, like it could be 500ms. Then reading would be unnecesary slow. I'll stick to multiple threads thing.

Shrug. I don't know what you're doing, you haven't really described it, so I'll stop giving advice Smiley.

malloc will be first against the wall when the revolution comes...
Offline Kova

Senior Member





« Reply #25 - Posted 2006-03-19 00:49:56 »

nvm, I was just creating a thread that would send packets every x ms (not every christmas, but every x milisecs Cheesy ). Anyway it took longer but I got it myself. All left is to handle unusual behavior and exceptions. Tnx everyone for their little something.
Pages: [1]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

TehJavaDev (15 views)
2014-08-28 18:26:30

CopyableCougar4 (25 views)
2014-08-22 19:31:30

atombrot (38 views)
2014-08-19 09:29:53

Tekkerue (34 views)
2014-08-16 06:45:27

Tekkerue (32 views)
2014-08-16 06:22:17

Tekkerue (20 views)
2014-08-16 06:20:21

Tekkerue (30 views)
2014-08-16 06:12:11

Rayexar (66 views)
2014-08-11 02:49:23

BurntPizza (43 views)
2014-08-09 21:09:32

BurntPizza (34 views)
2014-08-08 02:01:56
List of Learning Resources
by Longor1996
2014-08-16 10:40:00

List of Learning Resources
by SilverTiger
2014-08-05 19:33:27

Resources for WIP games
by CogWheelz
2014-08-01 16:20:17

Resources for WIP games
by CogWheelz
2014-08-01 16:19:50

List of Learning Resources
by SilverTiger
2014-07-31 16:29:50

List of Learning Resources
by SilverTiger
2014-07-31 16:26:06

List of Learning Resources
by SilverTiger
2014-07-31 11:54:12

HotSpot Options
by dleskov
2014-07-08 01:59:08
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!