Java-Gaming.org    
Featured games (81)
games approved by the League of Dukes
Games in Showcase (497)
Games in Android Showcase (114)
games submitted by our members
Games in WIP (563)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  Using FIleChannel to read floats  (Read 2248 times)
0 Members and 1 Guest are viewing this topic.
Offline gouessej
« Posted 2007-12-12 20:00:38 »

I have been trying to improve the speed of my way of reading large files containing floats. My first attempt gave this:

DataInputStream in=new DataInputStream(new BufferedInputStream(getClass().getResourceAsStream("/pic256/worldmap.data")));
levelCoordinatesBuffer=BufferUtil.newFloatBuffer(count*floatPerPrimitive);
for(i=0;i<count;i++)
           {levelCoordinatesBuffer.put(in.readFloat());
            levelCoordinatesBuffer.put(in.readFloat());
            levelCoordinatesBuffer.put(in.readFloat());
            levelCoordinatesBuffer.put(in.readFloat());
            levelCoordinatesBuffer.put(in.readFloat());
               }

My second attempt gives this:

levelCoordinatesBuffer=BufferUtil.newFloatBuffer(count*floatPerPrimitive);
try{
      ByteBuffer byteBuffer = BufferUtil.newByteBuffer(count*floatPerPrimitive*BufferUtil.SIZEOF_FLOAT);
      FileInputStream fis = new FileInputStream(new File(getClass().getResource("/pic256/worldmap.data").toURI()));
      fis.getChannel().read(byteBuffer);
      byteBuffer.position(0);
      //levelCoordinatesBuffer=byteBuffer.asFloatBuffer();
      levelCoordinatesBuffer.put(byteBuffer.asFloatBuffer());
       //byteBuffer = null;
     }
catch(URISyntaxException urise)
{
 urise.printStackTrace();
}
levelCoordinatesBuffer.position(0); 


The first method works but not the second. It is "as" I read only 0.0f. What seems to be wrong? How would you do it if you wanted to read plenty of floats as quick as possible?

Offline gouessej
« Reply #1 - Posted 2007-12-12 21:45:56 »

I've found my fault. The file may be written in a different order than the native order. When you create a direct buffer through BufferUtil, it has the same order as the native order. But how can I get the order of a file (BIG_ENDIAN or LITTLE_ENDIAN)?

Offline gouessej
« Reply #2 - Posted 2007-12-12 22:05:26 »

A file written with a Java program is always in BIG_ENDIAN. To prevent any problems while using nio buffers to read multibyte data, you must only force the byte order of the byte buffer you use at BIG_ENDIAN. I will put an example of my source code here if someone is interested. The main interest is to read a lot of data and to put it directly in the buffer of your choice.

Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline gouessej
« Reply #3 - Posted 2007-12-15 10:23:45 »

This example will be improved of course :

try{
           ByteBuffer byteBuffer = BufferUtil.newByteBuffer(count*floatPerPrimitive*BufferUtil.SIZEOF_FLOAT);
           ByteBuffer byteBuffer2 = BufferUtil.newByteBuffer(13*BufferUtil.SIZEOF_INT);
           FileInputStream fis = new FileInputStream(new File(getClass().getResource("/pic256/worldmap.data").toURI()));
           FileChannel fc=fis.getChannel();
           fc.position(0);
           fc.read(byteBuffer2);
           fc.read(byteBuffer);
           fc.force(true);
           fc.close();
           byteBuffer.position(0);
           byteBuffer.order(ByteOrder.BIG_ENDIAN);
           levelCoordinatesBuffer.put(byteBuffer.asFloatBuffer());
           byteBuffer.position(0);          
           IntBuffer tmpLevelCoordinatesBuffer2=BufferUtil.newIntBuffer(13);
           byteBuffer2.position(0);
           byteBuffer2.order(ByteOrder.BIG_ENDIAN);
           tmpLevelCoordinatesBuffer2.put(byteBuffer2.asIntBuffer());
           tmpLevelCoordinatesBuffer2.position(0);
           byteBuffer2.position(0);
          }
       catch(URISyntaxException urise)
       {
           urise.printStackTrace();
       }   

Offline broumbroum

Junior Member





« Reply #4 - Posted 2007-12-15 23:26:49 »

I think this goes a bit to far of the subject, reading float buffers. But as far as I can imagine, you want to get the quicker effect with reading. This is done by specifying a BufferedInputStream to wrap the File. This can be done throughout the RandomAccessFile wrapper or a BufferedInputStream on the usual input.
Sequently, the input reader is increased by powers of 2 by the speed when it fits the environment system bitwise reading, that is, the harddisk or ram bus bandwidth. usually harddisk specific readers can read up to 512 bytes "in logical one pass" and the ram bus can be even bigger. That's why setting up a reader for more than 512 bytes/read is useless on current harddisk drives.
If you want objective reading, use a RandomAccessFile, if you want bytes buffers, you better switch to a common 512 bytes reading loop.

::::... :..... :::::: ;;;:::™ b23:production 2006 GNU/GPL @ http://b23prodtm.webhop.info
on sf.net: /projects/sf3jswing
Java (1.6u10 plz) Web Start pool
dev' VODcast[/ur
Offline gouessej
« Reply #5 - Posted 2007-12-16 10:27:02 »

I think this goes a bit to far of the subject, reading float buffers.
No it doesn't go far from the subject as "levelCoordinatesBuffer" is a float buffer and I perform "levelCoordinatesBuffer.put(byteBuffer.asFloatBuffer());". I really used a FileChannel to read a file and to put the content in a FloatBuffer by using an intermediary ByteBuffer.

But as far as I can imagine, you want to get the quicker effect with reading.
Yes, that's what I want.

This is done by specifying a BufferedInputStream to wrap the File. This can be done throughout the RandomAccessFile wrapper or a BufferedInputStream on the usual input.
Sequently, the input reader is increased by powers of 2 by the speed when it fits the environment system bitwise reading, that is, the harddisk or ram bus bandwidth. usually harddisk specific readers can read up to 512 bytes "in logical one pass" and the ram bus can be even bigger. That's why setting up a reader for more than 512 bytes/read is useless on current harddisk drives.
If you want objective reading, use a RandomAccessFile, if you want bytes buffers, you better switch to a common 512 bytes reading loop.
But what I did above worked fine...when the file I want to read isn't remote and isn't in a Java archive.

Pages: [1]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

moogie (3 views)
2014-09-21 00:26:15

UprightPath (21 views)
2014-09-20 20:14:06

BurntPizza (27 views)
2014-09-19 03:14:18

Dwinin (40 views)
2014-09-12 09:08:26

Norakomi (70 views)
2014-09-10 13:57:51

TehJavaDev (96 views)
2014-09-10 06:39:09

Tekkerue (49 views)
2014-09-09 02:24:56

mitcheeb (70 views)
2014-09-08 06:06:29

BurntPizza (52 views)
2014-09-07 01:13:42

Longarmx (39 views)
2014-09-07 01:12:14
List of Learning Resources
by Longor1996
2014-08-16 10:40:00

List of Learning Resources
by SilverTiger
2014-08-05 19:33:27

Resources for WIP games
by CogWheelz
2014-08-01 16:20:17

Resources for WIP games
by CogWheelz
2014-08-01 16:19:50

List of Learning Resources
by SilverTiger
2014-07-31 16:29:50

List of Learning Resources
by SilverTiger
2014-07-31 16:26:06

List of Learning Resources
by SilverTiger
2014-07-31 11:54:12

HotSpot Options
by dleskov
2014-07-08 01:59:08
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!