Java-Gaming.org    
Featured games (81)
games approved by the League of Dukes
Games in Showcase (492)
Games in Android Showcase (112)
games submitted by our members
Games in WIP (556)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  Problem losing data  (Read 4185 times)
0 Members and 1 Guest are viewing this topic.
Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Posted 2008-03-04 14:35:09 »

Hi there,

I've been fighting a problem for a few days now and I'm about out of ideas.

In short, this is the story:
I'm writing a client for a server, which uses plain TCP sockets to connect. The receiving of data happens asynchronous, so it happens in a separate thread.
It works great, but every now and then (it happens *very* rarely), some data sent by the server to the client seems to get lost, leaving the client in a state where it just loses track of the protocol.

One possible way the data could be screwed if 2 threads try to read from the socket simultaneously, so I've been very careful that this can't happen and I'm pretty sure this doesn't happen.
I've asked the server guys to put a trace on what the're sending back to the client, and that seems to be okay, even if the client lost some data.

Has anyone of you experienced something like this? Any ideas?
Since I can't post source code (I'm not allowed to), I'm basically just trying to get some fresh ideas to be able to track down the problem.

Cheers,
Erik

Offline kevglass

JGO Kernel


Medals: 159
Projects: 23
Exp: 18 years


Coder, Trainee Pixel Artist, Game Reviewer


« Reply #1 - Posted 2008-03-04 14:39:10 »

NIO? Some buffer based read method sigs are slightly unintuitive.

Kev

Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #2 - Posted 2008-03-04 14:56:46 »

No NIO, just plain old java.net/java.io

Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 784
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #3 - Posted 2008-03-04 19:37:38 »

If it happens VERY rarely, it can be corrupted TCP packets that 'luckily' are mangled exactly so that their CRC is correct again. I read somewhere on this forum that someone implemented a MD5-like protocol, and found 7 corrupt  tcp-packets over a period of a few years.



If it happens like every hour or so, it must be in the application-layer. I've used Sockets for years, and I have applications that keep up and running for months continiuously doing I/O - when they crash, I know it's always always my fault.

If you want to be sure your InputStream/OutputStream gets accessed from only 1 thread ever (and the code using it is too large to quickly analyze) you can make a InputStreamWrapper that checks the Thread.currentThread() against some specified Thread, to quickly identity the offender. It's also good to simply make another stream wrapper that writes everything sent/received to a file, to analyze later. (you can re-run the communication by reading the file back into your protocol-handler)

If that doesn't show any problems, it's simply in the protocol handling (or corrupt memory - not uncommon). I bet it's a binary protocol, so differences in character-sets on both systems wouldn't get you into problems, and as you're using IO, as apposed to NIO, you can be fairly sure you master the API, so you can rule out that one too.

In short, without seeing any code, i'm pretty sure the problem is a bug in your code (which means: fixable!)

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #4 - Posted 2008-03-05 10:12:18 »

Thanks Riven.

I only hope it's my code and not the server  Smiley

I've tried your suggestion, and it confirmed that the streams from the socket are not touched by multiple threads at once. There is some thread switching going on, but only during the log on procedure (the communication is synchronous during logon, and swithes to asynchronous after successfully logging on. After having logged on, there's no thread switching anymore).

The protocol is indeed binary, and although there can be text inside exchanged messages, the problem is on the binary level.
Sometimes the problem doesn't happen for days, sometimes it happens 10 times a day.

To be honest, I'm still not completely ruling out the server and maybe the trace they sent me lied  Smiley.
I think I'm going to do some tracing on the network level, outside of the client app...

Offline CommanderKeith
« Reply #5 - Posted 2008-03-05 12:49:07 »

It could be that you're not flushing streams.  When I found similar problems I wrote the number of bytes I should expect at the beginning of the message, and if that wasn't what I recieved, I stopped the program right there by throwing a runtime error.

Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #6 - Posted 2008-03-05 14:32:03 »

Could be. In which case it would be a bug in the server, which I don't have the sources of unfortunately...
I guess I'll have to do a network trace to make sure it's not in my code.

Anyway, thanks all for the replies.

Offline Riven
« League of Dukes »

JGO Overlord


Medals: 784
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #7 - Posted 2008-03-05 23:22:16 »

To be honest, I'm still not completely ruling out the server and maybe the trace they sent me lied  Smiley.
I think I'm going to do some tracing on the network level, outside of the client app...

Don't waste your time on that one. Java network IO is not buggy!


I don't really know what you mean by "the trace they sent me", but if it isn't already so, could you ask the server-guys to have an OutputStream subclass that writes to multiple OutputStreams.. like:

> newSocketOutput = new MultiOutputStream(socket.getOutputStream(), new FileOutputStream(dst));

Clientside, build a special InputStream... like:
> new SocketInput = new SnifferInputStream(socket.getInputStream(), new FileOUTPUTStream(dst));

When wait for the 'problem', request the file from the server, and read the file from the client. They should be identical, always.



BTW: Are you using in.skip() without handling the return-value properly? (Write an inputstream-wrapper that blows when... blah blah blah, you get the point Smiley)

As you have probably figured, I have a dozen of InputStream/OutputStream subclasses laying around, which speedup my network-debugging significantly. Grin

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #8 - Posted 2008-03-06 11:18:02 »

Quote
Don't waste your time on that one. Java network IO is not buggy!

Oh, I'm sure Java network I/O is not buggy.
The thing is, I'm now becoming pretty much sure it's the server, while the server guys say it must be the client (and my manager seems to lean towards the server guys) so I need some hard evidence outside of my client app, using an external packet sniffer or something.

The low level traces I make now is done similar to your suggestion.
The server side is C++, being developed by an external company, and I'm not sure how and where they create their traces. I can't even configure the server, I only have an IP address & portnumber to test with + some out of date and incomplete documentation...  Undecided

Quote
BTW: Are you using in.skip() without handling the return-value properly?
No

Quote
As you have probably figured, I have a dozen of InputStream/OutputStream subclasses laying around, which speedup my network-debugging significantly
Heh, well it seems like a handy approach.

Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #9 - Posted 2008-03-10 11:40:16 »

FWIW, the problem turned out to be in the server after all... Undecided
I had to convince everybody with wireshark traces, but at least everybody agrees now.

Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 784
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #10 - Posted 2008-03-10 21:32:20 »

They should be grateful Smiley

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline blahblahblahh

JGO Coder


Medals: 1


http://t-machine.org


« Reply #11 - Posted 2008-04-09 10:44:50 »

Hi there,

some data sent by the server to the client seems to get lost, leaving the client in a state where it just loses track of the protocol.


Define "lost".

Does the data leave the server's NIC?
Does the data get to the client's NIC?

malloc will be first against the wall when the revolution comes...
Offline JAW

Senior Member


Medals: 2



« Reply #12 - Posted 2008-05-09 09:49:45 »

At first I would log anything as it gets received. Can you already tell if the data never arrives at the most basic read method? When using TCP, I am very certain, that everythin arrives correctly. So I would be very sure, that it is a programming error somewhere in the data handling of the application.

So I would log what arrives and log what the application gets and try to find the place where its lost.

The problem is, finding such bugs is extremely hard when you cannot reproduce the error systematically. So you really have to log a lot, hope the error happens, and then search the logs.

-JAW
Pages: [1]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

Nickropheliac (15 views)
2014-08-31 22:59:12

TehJavaDev (23 views)
2014-08-28 18:26:30

CopyableCougar4 (33 views)
2014-08-22 19:31:30

atombrot (41 views)
2014-08-19 09:29:53

Tekkerue (40 views)
2014-08-16 06:45:27

Tekkerue (35 views)
2014-08-16 06:22:17

Tekkerue (25 views)
2014-08-16 06:20:21

Tekkerue (37 views)
2014-08-16 06:12:11

Rayexar (72 views)
2014-08-11 02:49:23

BurntPizza (49 views)
2014-08-09 21:09:32
List of Learning Resources
by Longor1996
2014-08-16 10:40:00

List of Learning Resources
by SilverTiger
2014-08-05 19:33:27

Resources for WIP games
by CogWheelz
2014-08-01 16:20:17

Resources for WIP games
by CogWheelz
2014-08-01 16:19:50

List of Learning Resources
by SilverTiger
2014-07-31 16:29:50

List of Learning Resources
by SilverTiger
2014-07-31 16:26:06

List of Learning Resources
by SilverTiger
2014-07-31 11:54:12

HotSpot Options
by dleskov
2014-07-08 01:59:08
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!