Java-Gaming.org    
Featured games (81)
games approved by the League of Dukes
Games in Showcase (498)
Games in Android Showcase (114)
games submitted by our members
Games in WIP (563)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  [SOLVED] Bytes And Java Strings  (Read 2824 times)
0 Members and 1 Guest are viewing this topic.
Offline jonjava
« Posted 2012-04-16 05:43:35 »

Hi.

I've recently been working a lot with java.nio, but mainly Channels, SelectionKey's and ByteBuffers.

I'm having difficulties with reading data through channels. (Everything sent and received through Channels are in plain Bytes). I can send and receive just fine, and my echo server works without problems. But like I said, I can't interpret the data that is being sent in a useful way.

What I've been doing is having test servers running on localhost and connecting to these through telnet. The Data I receive form telnet comes up as either gibberish or what seems to be nothing at all. for Debugging I've been trying to echo to the console with System.out.println() the data I've received but nothing sensible comes up in the slight. I've tried appending strings, reading asCharBuffer etc, the only thing remotely close to an alphabetical letter I've come to is the '?' letter... and those only appear after CR.

Plain text is written in telnet and nothing shows up on console no matter what - why is this??

Offline 65K
« Reply #1 - Posted 2012-04-16 09:11:45 »

Hard to say anything without code samples.
Especially because NIO is involved. Grin

Offline Stranger

Senior Member


Medals: 6



« Reply #2 - Posted 2012-04-16 11:05:08 »

I think the problem is in wrong encoding/decoding of strings.
So, this possibly can help you: http://stackoverflow.com/questions/1252468/java-converting-string-to-and-from-bytebuffer-and-associated-problems.

Anton
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline jonjava
« Reply #3 - Posted 2012-04-16 16:42:42 »

I see, thanks for highlighting character encoding. The SO page you linked sent me to an interesting article by Joel Spolsky (co.f. of SO ) called "The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)"

I was in the middle of reading Java NIO by Ron Hitchens and stopped just after covering channels, keys and buffers to try out the stuff I had read.

This was all fine except for the character encoding issues I had. I see now that Character Sets are covered in the book in the following chapter. :L Which I'm now going to read in mild embarrassment. I've been quite ignorant towards the importance of how bytes are interpreted and the decades long struggle and ingenuity that's been going into figuring out the whole mess.

The Article really opened my eyes quite a bit so thanks again:P

Offline Riven
« League of Dukes »

JGO Overlord


Medals: 799
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #4 - Posted 2012-04-16 18:12:46 »

If you're using the characters 0-9,a-z,A-Z, it should (almost) always work. If you still have odd characters, you simply have a bug in your networking code (byte transfer). No encoding will save you from that Smiley

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline DrZoidberg

Senior Member


Medals: 15



« Reply #5 - Posted 2012-04-16 21:03:24 »

btw. A CharBuffer always uses 16 bit unicode encoding.
Offline jonjava
« Reply #6 - Posted 2012-04-16 23:33:13 »

I fixed it by simply decoding the received data with utf-8.

1  
2  
3  
4  
5  
6  
protected static Charset cs = Charset.forName("utf-8");

// ...

CharBuffer buf = cs.decode(buffer);
      message += buf;


I checked my systems default charset with Charset.defaultCharset().name() and it spit out "windows-1252"

I'm thinking that might've been the issue.

Offline ra4king

JGO Kernel


Medals: 345
Projects: 3
Exp: 5 years


I'm the King!


« Reply #7 - Posted 2012-04-16 23:57:13 »

I recommend you just stick with UTF-8:
1  
2  
3  
4  
//write the String
byte[] b = myString.getBytes("UTF-8");
byteBuffer.putInt(b.length);
byteBuffer.put(b);


1  
2  
3  
4  
5  
6  
7  
8  
//read the String
int len = byteBuffer.getInt();

//make sure len is not a ridiculous number that could crash your application ;)

byte[] b = new byte[len];
byteBuffer.get(b);
String s = new String(b,"UTF-8");


EDIT: heh, didn't see that last message. It's best to avoid Charset since it's slow. See code above.

Offline jonjava
« Reply #8 - Posted 2012-04-17 00:15:10 »

If you're using the characters 0-9,a-z,A-Z, it should (almost) always work. If you still have odd characters, you simply have a bug in your networking code (byte transfer). No encoding will save you from that Smiley

I think the issue in its core was that since channels are inconsistent in how many bytes are actually read, that the decoder wraps around the bytes it is fed and keeps them in track for you.

I'll post my simple echo server in java.nio here for those interested to look it over.

http://www.java-gaming.org/?action=pastebin&id=62

I recommend you just stick with UTF-8:
1  
2  
3  
4  
//write the String
byte[] b = myString.getBytes("UTF-8");
byteBuffer.putInt(b.length);
byteBuffer.put(b);


1  
2  
3  
4  
5  
6  
7  
//read the String
int len = byteBuffer.getInt();

//make sure len is not a ridiculous number that could crash your application ;)

byte[] b = new byte[len];
String s = new String(b,"UTF-8");


EDIT: heh, didn't see that last message. It's best to avoid Charset since it's slow. See code above.

What if the non-blocking channel.read() didn't register all the bytes that you're attempting to read with len?

And in your reading code, where exactly do you read the data into the buffer? o.O
Am I blind or do I only see you making a string of an empty allocated byte array of length len?

Offline ra4king

JGO Kernel


Medals: 345
Projects: 3
Exp: 5 years


I'm the King!


« Reply #9 - Posted 2012-04-17 02:24:30 »

Oh whoops, I missed that line. I added it in the original post Tongue

And that's where you make sure you read all the bytes first. In my networking "library", I send an integer containing the length of the ByteBuffer that was sent. In my read method, I make sure I read at least that many bytes before returning the data.

Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline jonjava
« Reply #10 - Posted 2012-04-17 03:06:27 »

I see, that's quite nifty.

Does it go against concurrency if you read the bytes stuck inside a loop until you reach the len amount, or it is better to wait for the next round-about and keep your key selected? (Ie. keeping it in the Selectors.selected() set).

But what about protocols that don't adhere to the first int is the length of bytes sent? Or is this a non issue in general since you'll always know what you're going to get?

And why is the Charset class slow compared to the Strings getByte( str utf ) function?

Offline Riven
« League of Dukes »

JGO Overlord


Medals: 799
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #11 - Posted 2012-04-17 16:50:13 »

But what about protocols that don't adhere to the first int is the length of bytes sent? Or is this a non issue in general since you'll always know what you're going to get?
It's indeed a non-issue, because the used protocol must be known ahead of time.


And why is the Charset class slow compared to the Strings getByte( str utf ) function?
It's not. Both have the same implementation.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Pages: [1]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

BurntPizza (16 views)
2014-09-21 02:42:18

BurntPizza (13 views)
2014-09-21 01:30:30

moogie (13 views)
2014-09-21 00:26:15

UprightPath (24 views)
2014-09-20 20:14:06

BurntPizza (27 views)
2014-09-19 03:14:18

Dwinin (40 views)
2014-09-12 09:08:26

Norakomi (71 views)
2014-09-10 13:57:51

TehJavaDev (96 views)
2014-09-10 06:39:09

Tekkerue (49 views)
2014-09-09 02:24:56

mitcheeb (70 views)
2014-09-08 06:06:29
List of Learning Resources
by Longor1996
2014-08-16 10:40:00

List of Learning Resources
by SilverTiger
2014-08-05 19:33:27

Resources for WIP games
by CogWheelz
2014-08-01 16:20:17

Resources for WIP games
by CogWheelz
2014-08-01 16:19:50

List of Learning Resources
by SilverTiger
2014-07-31 16:29:50

List of Learning Resources
by SilverTiger
2014-07-31 16:26:06

List of Learning Resources
by SilverTiger
2014-07-31 11:54:12

HotSpot Options
by dleskov
2014-07-08 01:59:08
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!