Java-Gaming.org    
Featured games (81)
games approved by the League of Dukes
Games in Showcase (495)
Games in Android Showcase (114)
games submitted by our members
Games in WIP (563)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  Learn NIO  (Read 2388 times)
0 Members and 1 Guest are viewing this topic.
Offline CaptainJester

JGO Knight


Medals: 12
Projects: 2
Exp: 14 years


Make it work; make it better.


« Posted 2010-03-16 03:09:17 »

I finally took the plunge and tried to learn NIO.  I decided to write an FTP server to make it a real world application.  I have attached the code here in hopes that others will find it useful.  Also if anyone has any comments, corrections, improvements etc.. they would most certainly be welcome.  As this is my first attempt, my implementation may be naive and I appreciate anyone pointing out the flaws.

As stated, the point was to learn NIO, so the server meets the minimum standard set out by RFC 959 section 5.1.  MINIMUM IMPLEMENTATION, except record structures for file transfer.  I also only allows anonymous access.  There is also a configuration file to set some variable options like server welcome message and maximum concurrent connections.

Also I released the code under the BSD license so if you want use, go ahead.

Offline Eli Delventhal

JGO Kernel


Medals: 42
Projects: 11
Exp: 10 years


Game Engineer


« Reply #1 - Posted 2010-03-16 03:53:37 »

Thanks for sharing. I really need to buck up and learn NIO also.  Undecided

See my work:
OTC Software
Online Riven
« League of Dukes »

JGO Overlord


Medals: 798
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #2 - Posted 2010-03-16 07:03:51 »

Now try to make it single-threaded. Wink

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Nate

JGO Kernel


Medals: 147
Projects: 4
Exp: 14 years


Esoteric Software


« Reply #3 - Posted 2010-03-16 07:50:50 »

I took a peek, though while I've written some NIO stuff, I don't pretend to be an expert. Smiley

* You allocate 8KB for every read. You might want to allocate one buffer per client. Also you may want to use a direct buffer, but I haven't personally tested that it is better.

* Client#run has @Override but doesn't, so doesn't compile for me. Same with Main#run().

* In Client#read()... Not sure you need the "channel.isOpen()"? There is no guarantee the channel is still open inside the IF, since it can go bad at any time.

* In Client#read() you use "new String(byte[])", I'm guessing you probably want to explicitly specify a charset?

* "messageBufferInUse" looks suspicious. You synchronize on "lock" to set it true and false, but in Client#run you read its value without the lock. It looks like you are trying to avoid the synchronize and sleep instead, but it would be better to just synchronize and your thread will wake up as soon as the monitor is free.

* In Client... You copy the new string into a StringBuffer which is thread safe, and you also do the locking yourself. You probably want to use StringBuilder and lock yourself, of StringBuffer and let it do the locking.

* Not sure why you start a new Thread in the Client constructor? It seems you were going well with one thread, then fell back to the good old ways when it came to the DataConnection stuff. Wink

* With NIO, after connect, generally you should only have OP_READ registered. When you want to write something, you should just try to write it. This almost always succeeds, and avoids the overhead of OP_WRITE. If not all the data could be written, only then should you register both OP_READ and OP_WRITE to write the remainder. When all the data you have is written, you should go back to just OP_READ.

Currently you seem to have OP_WRITE registered all the time. This will introduce unnecessary delays because your data won't be written until the next select. For FTP this is probably negligible. However, selector.select() in Main#run will always return immediately since it thinks you have data to write, and Client#write will be called repeatedly when there is no data to write. You effectively have a "busy wait".

Offline Stranger

Senior Member


Medals: 6



« Reply #4 - Posted 2010-03-16 11:46:21 »

Hi,

I finally took the plunge and tried to learn NIO.  I decided to write an FTP server to make it a real world application.  I have attached the code here in hopes that others will find it useful.  Also if anyone has any comments, corrections, improvements etc.. they would most certainly be welcome.  As this is my first attempt, my implementation may be naive and I appreciate anyone pointing out the flaws.

As stated, the point was to learn NIO, so the server meets the minimum standard set out by RFC 959 section 5.1.  MINIMUM IMPLEMENTATION, except record structures for file transfer.  I also only allows anonymous access.  There is also a configuration file to set some variable options like server welcome message and maximum concurrent connections.

Also I released the code under the BSD license so if you want use, go ahead.

I hope this could be useful as an example:

http://mina.apache.org/ftpserver/

Anton
Offline CaptainJester

JGO Knight


Medals: 12
Projects: 2
Exp: 14 years


Make it work; make it better.


« Reply #5 - Posted 2010-03-16 11:56:15 »

Now try to make it single-threaded. Wink

I originally started a thread per client so I could deal with commands not being fully read.  But now that I think about I could probably do it all in a single thread.

Thanks.

Offline CaptainJester

JGO Knight


Medals: 12
Projects: 2
Exp: 14 years


Make it work; make it better.


« Reply #6 - Posted 2010-03-16 12:09:31 »

Thanks for all the great points Nate.  I will definitely make some changes based on your comments.
I took a peek, though while I've written some NIO stuff, I don't pretend to be an expert. Smiley

* You allocate 8KB for every read. You might want to allocate one buffer per client. Also you may want to use a direct buffer, but I haven't personally tested that it is better.
Each client gets and instance of Client.  Since I allocate 8KB inside the client, that is only 1 buffer per client.  It shouldn't hurt to use direct ByteBuffers so I can make that change.
* Client#run has @Override but doesn't, so doesn't compile for me. Same with Main#run().
Both Client and Main implement Runnable and therefore override public void run().  It was Eclipse that put those annotations in.  I usually don't remove them because it doesn't cause me a problem.
* In Client#read()... Not sure you need the "channel.isOpen()"? There is no guarantee the channel is still open inside the IF, since it can go bad at any time.
Most of the examples I found used that so I thought it was appropriate.
* In Client#read() you use "new String(byte[])", I'm guessing you probably want to explicitly specify a charset?
True.
* "messageBufferInUse" looks suspicious. You synchronize on "lock" to set it true and false, but in Client#run you read its value without the lock. It looks like you are trying to avoid the synchronize and sleep instead, but it would be better to just synchronize and your thread will wake up as soon as the monitor is free.

* In Client... You copy the new string into a StringBuffer which is thread safe, and you also do the locking yourself. You probably want to use StringBuilder and lock yourself, of StringBuffer and let it do the locking.
Yeah, I didn't quite get that right.  I will have to fix it.
* Not sure why you start a new Thread in the Client constructor? It seems you were going well with one thread, then fell back to the good old ways when it came to the DataConnection stuff. Wink
I was thinking that since a command may not come through completely in on read I would need to keep checking the message buffer.  However I thought of another way that will remove the thread from client and the whole thing can be in one thread.
* With NIO, after connect, generally you should only have OP_READ registered. When you want to write something, you should just try to write it. This almost always succeeds, and avoids the overhead of OP_WRITE. If not all the data could be written, only then should you register both OP_READ and OP_WRITE to write the remainder. When all the data you have is written, you should go back to just OP_READ.

Currently you seem to have OP_WRITE registered all the time. This will introduce unnecessary delays because your data won't be written until the next select. For FTP this is probably negligible. However, selector.select() in Main#run will always return immediately since it thinks you have data to write, and Client#write will be called repeatedly when there is no data to write. You effectively have a "busy wait".
I didn't know that.  Thanks.

Thanks again Nate.

Offline CaptainJester

JGO Knight


Medals: 12
Projects: 2
Exp: 14 years


Make it work; make it better.


« Reply #7 - Posted 2010-03-16 12:13:12 »

Hi,

I hope this could be useful as an example:

http://mina.apache.org/ftpserver/
Thanks.

Offline Mr. Gol

Senior Member


Medals: 1



« Reply #8 - Posted 2010-03-16 14:33:56 »

* Client#run has @Override but doesn't, so doesn't compile for me. Same with Main#run().

In 1.6 the @Override annotation can also be used on methods that are declared by an interface.
Offline Nate

JGO Kernel


Medals: 147
Projects: 4
Exp: 14 years


Esoteric Software


« Reply #9 - Posted 2010-03-16 22:49:39 »

Each client gets and instance of Client.  Since I allocate 8KB inside the client, that is only 1 buffer per client.
Main#run calls Client#read every time some data comes in for that client. Client#read allocates 8KB every time it is called.

Quote
I was thinking that since a command may not come through completely in on read I would need to keep checking the message buffer.
Just check it once after each read. If it has enough data to process, do so, otherwise let it be. I think the biggest changes you'll need to make it single threaded are around the DataConnection stuff. Treat them like you do Client (where Main#run calls methods to read/write/accept as needed) only with different logic (for sending/receiving files rather than processing FTP commands).

In 1.6 the @Override annotation can also be used on methods that are declared by an interface.
Ah, I must have had my compiler set to 1.5. Personally I hate that annotation, as it doesn't provide any useful information. My IDE already knows the method is implementing or overriding a super class method. It even shows me a little arrow in the gutter I can click to go to the super method. The annotation is just extraneous noise, says I!

I hope this could be useful as an example:
http://mina.apache.org/ftpserver/
But MINA completely hides NIO, so then he wouldn't get to deal with this horrible, magical API. Smiley

Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Online Riven
« League of Dukes »

JGO Overlord


Medals: 798
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #10 - Posted 2010-03-16 23:28:16 »

I originally started a thread per client so I could deal with commands not being fully read.  But now that I think about I could probably do it all in a single thread.

Single-threaded NIO is actually... hard. Especially if you want it to be performant and efficient.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline CaptainJester

JGO Knight


Medals: 12
Projects: 2
Exp: 14 years


Make it work; make it better.


« Reply #11 - Posted 2010-03-17 01:47:31 »

Single-threaded NIO is actually... hard. Especially if you want it to be performant and efficient.

Yeah, especially in this case, transferring large files cannot hold up the thread in case other requests need to be processed.

Online Riven
« League of Dukes »

JGO Overlord


Medals: 798
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #12 - Posted 2010-03-17 06:49:37 »

Yeah, especially in this case, transferring large files cannot hold up the thread in case other requests need to be processed.

You have that exact same problem transferring 1 byte.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Pages: [1]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

BurntPizza (13 views)
2014-09-19 03:14:18

Dwinin (31 views)
2014-09-12 09:08:26

Norakomi (57 views)
2014-09-10 13:57:51

TehJavaDev (79 views)
2014-09-10 06:39:09

Tekkerue (40 views)
2014-09-09 02:24:56

mitcheeb (62 views)
2014-09-08 06:06:29

BurntPizza (45 views)
2014-09-07 01:13:42

Longarmx (30 views)
2014-09-07 01:12:14

Longarmx (36 views)
2014-09-07 01:11:22

Longarmx (36 views)
2014-09-07 01:10:19
List of Learning Resources
by Longor1996
2014-08-16 10:40:00

List of Learning Resources
by SilverTiger
2014-08-05 19:33:27

Resources for WIP games
by CogWheelz
2014-08-01 16:20:17

Resources for WIP games
by CogWheelz
2014-08-01 16:19:50

List of Learning Resources
by SilverTiger
2014-07-31 16:29:50

List of Learning Resources
by SilverTiger
2014-07-31 16:26:06

List of Learning Resources
by SilverTiger
2014-07-31 11:54:12

HotSpot Options
by dleskov
2014-07-08 01:59:08
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!