Java-Gaming.org    
Featured games (81)
games approved by the League of Dukes
Games in Showcase (487)
Games in Android Showcase (112)
games submitted by our members
Games in WIP (553)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
   Home   Help   Search   Login   Register   
  Show Posts
Pages: [1]
1  Game Development / Networking & Multiplayer / Selector Deadlock on selector.close() ? on: 2003-09-19 20:38:43
What does this paragraph straight from http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/Selector.html javadocs on NIO mean regarding how to interrupt a select using close()?:
Quote
A thread blocked in one of the select() or select(long) methods may be interrupted by some other thread in one of three ways:

By invoking the selector's wakeup method,

By invoking the selector's close method, or

By invoking the blocked thread's interrupt method, in which case its interrupt status will be set and the selector's wakeup method will be invoked.

The close method synchronizes on the selector and all three key sets in the same order as in a selection operation.



The first part says that selector.close() will interupt the select(), but then it says that close() locks on the same keys as select(), meaning that it will wait forever until the select() is complete before closing.  So which should it be.  What I am seeing is that close() does not interrupt the select, but waits for select to finish.  Is this a bug, or by design?

Then down below in the close() method it states:
Quote
If a thread is currently blocked in one of this selector's selection methods then it is interrupted as if by invoking the selector's wakeup method.

Any uncancelled keys still associated with this selector are invalidated, their channels are deregistered, and any other resources associated with this selector are released.



However, this is not the case.  Simple example:
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  
42  
43  
44  
45  
46  
47  
48  
49  
50  
51  
52  
53  
54  
55  
56  
57  
58  
59  
60  
61  
62  
63  
64  
65  
66  
67  
68  
69  
70  
71  
72  
73  
74  
75  
76  
77  
78  
79  
80  
81  
82  
83  
84  
85  
86  
87  
import java.nio.channels.Selector;

public class SelectorDeadlock implements Runnable
{
     
      Thread _thisThread = null;
      Selector _selector = null;
     
      public SelectorDeadlock( )
      {
           
      }
     
     
      public static void main(String[] args)
      {
            SelectorDeadlock sd = null;
            try
            {
                  sd = new SelectorDeadlock( );
                  sd.start();
                  // wait 3 seconds
                 Thread.currentThread().sleep( 3000 );
                  // now try to close and see the deadlock
                 sd.closeSelector();
            }catch( Exception e )
            {
                  sd.stop();
                  e.printStackTrace();
            }
           
      }
     
      public void start( )throws Exception
      {
            if ( _selector == null )
            {
                  _selector = Selector.open();
            }
            if ( _thisThread == null )
            {
                  _thisThread = new Thread( this, "SelectorDeadlock" );
                  _thisThread.start();
            }
           
      }
     
      protected void stop()
      {
            if ( _thisThread != null )
            {
                  Thread threadTemp = _thisThread;
                  _thisThread = null;

                  threadTemp.interrupt(  );        
            }
      }
     
      public void closeSelector( )throws Exception
      {
            if ( _selector != null )
            {
                  System.out.println("Attempting to close selector");
                  _selector.close();
                  System.out.println("Selector has been successfully closed");
            }
      }
     
      /* (non-Javadoc)
       * @see java.lang.Runnable#run()
       */

      public void run()
      {
            while ( _thisThread != null )
            {
                  try
                  {
                        _selector.select();
                  }catch( Exception e )
                  {
                        e.printStackTrace();
                  }
            }

      }

}


This is on windows running 1.4.2beta and 1.4.2_01

Any feedback.  I just checked java bug report and didnt see this anywhere.  It seems too obvious to be wrong.
2  Game Development / Networking & Multiplayer / Re: Extremely poor performance with NIO... on: 2003-07-28 15:42:03
blah blah,

how has the workaround held up so far.  

I am thinking of putting a bug report in for the problem i was seeing with multiple clients, and so revisited your test.

What java version were you running?  I saw a VERY significant difference when running the exact same numbers between java 1.4.1_01 and java 1.4.2b.

Just wondering if something may have changed between the version.  It runs much faster with 1.4.1_01
3  Game Development / Networking & Multiplayer / Re: Should you risk using NIO for hard-core networ on: 2003-06-27 15:03:10
Im back,

I cleaned up the code to make it more legible and it now uses the select() method instead of sleeping.  I am still seeing slower-than-expected results with the sample code and would like more feedback if any.

the code is found at: http://users.adelphia.net/~dfellars1/NIOSelectorCode.html


Also, on a different NIO related topic, I have 2 servers communicating with eachother over a socket that remains open the entire time the servers are up.  Up until recently I was running the servers on the same network, but recently moved one of them to a different network and am getting the following situation:

The connection is getting dropped for whatever reason, which is to be expected.  However, the client that writes to the dropped connection does not throw any exception when writing.  It writes all the bytes to the channel and returns as if everything is ok.  Then maybe a minute later I will receive a read of -1 specifying the connection was dropped, but the writen message was never received on the server.  

What I would like to be able to do is determine that the connection is dropped before writing to it, so that I can reconnect and then write to the channel.  Is there an easy way to detect a dropped connection?  I am already using isOpen() and key.isValid() which are returning true each time.  I have also tried socket.setKeepAlive(true) with no success.

Should the SocketChannel.isConnected() tell whether the connection is available to write to, or does that just state that the connection has connected to the server?

I have implemented a pinging system to keep traffic going across the connection, but dont want to have to rely on this to keep the connection alive.

Thanks for an insight

4  Game Development / Networking & Multiplayer / Re: Should you risk using NIO for hard-core networ on: 2003-06-04 15:56:32
Please forgive me for my stupidity.  This is one of those "I could have sworn I changed that" moments.  I dont know if you remember one of my previous post where I mentioned that when I performed a sleep after a selectNow call it speed things up, which made you blahblahblah suggest an MT issue, so that is why i made it all one thread and *thought* it put it back to using the select() method by passing true to that method.  But I obviously didnt.  Sorry.  I have updated the code on my webserver and will re-run again with new code to see if it speeds things up.

Thanks , and again I am sorry for that.
5  Game Development / Networking & Multiplayer / Re: Should you risk using NIO for hard-core networ on: 2003-06-04 15:01:36
Quote


If I understand correctly, to summarise (and slightly over-simplify):

 1: lots of clients get connected to the server, and go into a wait() situation.
 2: once all are waiting, server does the equivalent of a notifyall, to get ALL of them *simultaneously* to do their transfers.
 3: ...server waits for all to get back into the wait, then starts again


That is correct.

Quote

 1. Are your clients and server separate machines, with fully switched connection (as opposed to hubs)?


Yes, I have ran them on seperate machines on different network, I have ran them on the same network, as well as on the same machine.

Quote


 2. What happens if you double the number of physical client machines, halving the number of client-apps running on each?



I have tried this with somewhat similar results.
Quote


 3. What's the LAN saturation like?
 4. Any collision-storms going on? (sounds like you have an admin who would have net-management tools to give you this info).


I will have to get together again with my admin.  They wont allow me to run any network monitoring myself so I have to bug them to get them to do it.

Quote


 5. Have you tried having two selectors on the server, one for read, one for write? Changing the interestOps for a key is a trivial method call hiding a non-trivial implementation (note that it has at least one blocking point just to change the interestOps!)...it could be that there is some delay added by the switching back and forth of all those interest sets.


I originally had it designed this way. but was worried that my issue was MT, so went to a single selector approach.  Since going to a single selector, I actually saw it speed up, thus implying there may have been a MT issue.  although I still cant see where.


Quote


However, I suspect the problem is something completely different and hope that someone else will reply with a "nah, this is the problem:" post instead ;).


That is what I am hoping for as well.  

Thanks

6  Game Development / Networking & Multiplayer / Re: Should you risk using NIO for hard-core networ on: 2003-06-04 14:48:55
Oops.  Too much code to post.  I did a quick ref for it at:

http://users.adelphia.net/~dfellars1/NIOSelectorCode.html

Sorry about that.  Please check it out there.  

Thanks again for any help.
7  Game Development / Networking & Multiplayer / Re: Should you risk using NIO for hard-core networ on: 2003-06-04 14:22:01
Why did  this thread die?

I essentially cleaned down my sample code I was working with so that there is only 1 thread with one selector for all 3 operations: OP_CONNECT, OP_READ, OP_WRITE.  I know this is not the best approach, but wanted to remove any chance of having MT issues.  The following code was adapted from a Sun developer example and modified to keep track of how long it takes for each operation.  

The general flow is the following:
1) The Server starts up and listens for connecting sockets.
2) Once the predetermined number of connections have connected to the server, the server then sends 1 byte to all the connections telling them to send their payload.  This is the initialization phase.  After sending the bytes, the connections selection key interest ops is set to read.
3) The clients upon reading this one byte from the server, then switch their interest ops to write, and send their payload to the server.  This is the clients write phase.  The clients then switch interest ops back to read to be ready for the response from server.
4) The server then reads the payload for all the clients.  This is the servers read phase.  
5) When all bytes for a client connection have been read, the server immediately switches that connections selection key interest ops to write and writes back all the bytes received(simple echo).  This is the servers write phase.
6) The client then reads all the bytes sent back from the server.  This is the clients read phase.
7)  On the server, once all bytes have been read for all client connections, the server repeats the initialization phase (#2 above ) all over again to repeat the process up to the number of trips pre-determined.

Each phase keeps track of how long it takes to perform its operation based on System.currentTimeMillis() taken before and after each operation.  Then after each trip is completed, a print out is done summarizing the results.

The Server Trip results contain:

Total time: The time between the servers first read operation and the servers last write operation for this trip.
Init time: The time spent actually writting the 1 initialization byte to the sockets.
Read time: The time sent actually reading the payload from the socket, not counting time between select() opeation.
Write time: The time sent actually writing the payload from the socket, not counting time between select() opeation.
Read Sel Time: The time between the first read operation on a key and the last read operation on a key.  This DOES include time spent inside the select() method as well as time spent inside the read()
Write Sel Time: The time between the first write operation on a key and the last write operation on a key.  This DOES include time spent inside the select() method as well as time spent inside the write()


The Client process is a group of connections and has the following trip summary:

Read Time: This is the sum of all clients time to finish reading in the payload.  This is misleading as there is the chance of time overlap involved here.
Write Time: This is the sum of all clients time to finish writing the payload to the socket.  This is misleading as there is the chance of time overlap involved here.
Round Trip Time(RTT): This is the sum of all clients time between their final write time and their first read time.  This is essentially the time take to go across the network, be processed by the server, then get back to the client.  This is misleading as there is the chance of time overlap involved here.
Total Time: This is the sum of all clients time between their first write time and their final read time.  This is essentially the time take to write all bytes, go across the network, be processed by the server, then get back to the client, and all bytes read in.  This is misleading as there is the chance of time overlap involved here.

The Avgs are then printed out.  This is a better understanding as it is the total values printed in the previous line and explained above.  This is the values above divided by the number of clients.

The next line gives more insight:
Just Reading: The time sent actually reading the payload from the socket, not counting time between select() opeation.
Just Writing: The time sent actually writing the payload from the socket, not counting time between select() opeation.
Total Read time: The time between the first read operation on a key and the last read operation on a key.  This DOES include time spent inside the select() method as well as time spent inside the read()
Total Write Time: The time between the first write operation on a key and the last write operation on a key.  This DOES include time spent inside the select() method as well as time spent inside the write()

REASON FOR THIS POST:
Now, after explaining all this, the purpose of my post is to get some insight into the numbers I am receiving from running this test.  I am using the sample code below to do some stress testing on how many connections an advanced gaming server built on NIO can handle.  The tests I have ran have been rather dissapointing, which leads me to think there is something going on that I can't see.  

For example:  When running the test with 2000 clients, sending 100 bytes each, sent 100 trips, I will see the server taking anywhere from 5 to 20 seconds to read in all the bytes, and write them all back to the clients.  It appears that there is alot of time spent inside the select() method waiting to be notified for the operation to take place.  The actual reads and writes aren't taking that much time.  Because there is only one thread, that rules out MT deadlocks.

I have ran this test on Windows, Linux(Red Hat), and Solaris, all with similar results.  All on my company's 100Mbs(I think ??) network.  I have had my Network Admin sniff the network to see if there is any lags in the network and he assures me there is nothing slow or fishy going on.

Any insight or improvements on the code is welcome.  Or any other code to test how many connections NIO can handle.  My objective is to see how many players can send up their scores to be processed and ranked within a 10 second window.

Thanks for any insight.  Sorry for the long post.

The code will be the following post





8  Game Development / Networking & Multiplayer / Re: Should you risk using NIO for hard-core networ on: 2003-05-15 20:57:05
Which select method have you guys used? select() vs. select(long) vs. selectNow()?  Have you noticed any implementation issues between them?  From my testing I have found that select() often blocks too long and so switched to doing selectNow() followed by a thread.sleep() to rest a bit and saw significant connection throughput.  Anybody else seen something familiar?
9  Game Development / Networking & Multiplayer / Re: Should you risk using NIO for hard-core networ on: 2003-05-13 16:30:53
I am new to this site looking in particular to more insight on this particular subject.  We are doing a very similar approach of having multiple selectors for read, one per thread.  

Have you done any stress testing to find an efficient number of connections to be handled per read selector?

What in particular about the Byte Buffers is so sensitive?  

I have seen some rather long latencies in the testing that I have done and am in the process of determining if those latencies are on the underlying network connection or somewhere inside the NIO code.

I have noticed rather long garbage collection periods in between the selector.select() methods when running the -gc:verbose command.  These may be a result of how I am managing my ByteBuffers. (??) My approach of handling incoming connections is rather simple.  Each SelectionKey.attachment object reads in the first 4 bytes which is the size of the message, then allocates a ByteBuffer of that size and does non-blocking read until buffer is full, then flips, reads and sends to be processed at which point the ByteBuffer.clear() is called (which doesnt really free up the memory, just resets the indexes) and does process all over again.

Has anyone done any load testing with simple NIO example?  What kinds of results are seen?

Thanks for any insight.
Pages: [1]
 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

TehJavaDev (15 views)
2014-08-28 18:26:30

CopyableCougar4 (25 views)
2014-08-22 19:31:30

atombrot (38 views)
2014-08-19 09:29:53

Tekkerue (34 views)
2014-08-16 06:45:27

Tekkerue (32 views)
2014-08-16 06:22:17

Tekkerue (20 views)
2014-08-16 06:20:21

Tekkerue (30 views)
2014-08-16 06:12:11

Rayexar (66 views)
2014-08-11 02:49:23

BurntPizza (43 views)
2014-08-09 21:09:32

BurntPizza (34 views)
2014-08-08 02:01:56
List of Learning Resources
by Longor1996
2014-08-16 10:40:00

List of Learning Resources
by SilverTiger
2014-08-05 19:33:27

Resources for WIP games
by CogWheelz
2014-08-01 16:20:17

Resources for WIP games
by CogWheelz
2014-08-01 16:19:50

List of Learning Resources
by SilverTiger
2014-07-31 16:29:50

List of Learning Resources
by SilverTiger
2014-07-31 16:26:06

List of Learning Resources
by SilverTiger
2014-07-31 11:54:12

HotSpot Options
by dleskov
2014-07-08 01:59:08
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!