Using FIleChannel to read floats

[gouessej]

I have been trying to improve the speed of my way of reading large files containing floats. My first attempt gave this:

DataInputStream in=new DataInputStream(new BufferedInputStream(getClass().getResourceAsStream("/pic256/worldmap.data")));
levelCoordinatesBuffer=BufferUtil.newFloatBuffer(count*floatPerPrimitive);
for(i=0;i<count;i++)
           {levelCoordinatesBuffer.put(in.readFloat());
            levelCoordinatesBuffer.put(in.readFloat());
            levelCoordinatesBuffer.put(in.readFloat());
            levelCoordinatesBuffer.put(in.readFloat());
            levelCoordinatesBuffer.put(in.readFloat());
               }

My second attempt gives this:

levelCoordinatesBuffer=BufferUtil.newFloatBuffer(count*floatPerPrimitive);
try{
      ByteBuffer byteBuffer = BufferUtil.newByteBuffer(count*floatPerPrimitive*BufferUtil.SIZEOF_FLOAT);
      FileInputStream fis = new FileInputStream(new File(getClass().getResource("/pic256/worldmap.data").toURI()));
      fis.getChannel().read(byteBuffer);
      byteBuffer.position(0);
      //levelCoordinatesBuffer=byteBuffer.asFloatBuffer();
      levelCoordinatesBuffer.put(byteBuffer.asFloatBuffer());
       //byteBuffer = null;
     }
catch(URISyntaxException urise)
{
 urise.printStackTrace();
}
levelCoordinatesBuffer.position(0); 


The first method works but not the second. It is "as" I read only 0.0f. What seems to be wrong? How would you do it if you wanted to read plenty of floats as quick as possible?

[gouessej]

I've found my fault. The file may be written in a different order than the native order. When you create a direct buffer through BufferUtil, it has the same order as the native order. But how can I get the order of a file (BIG_ENDIAN or LITTLE_ENDIAN)?

[gouessej]

A file written with a Java program is always in BIG_ENDIAN. To prevent any problems while using nio buffers to read multibyte data, you must only force the byte order of the byte buffer you use at BIG_ENDIAN. I will put an example of my source code here if someone is interested. The main interest is to read a lot of data and to put it directly in the buffer of your choice.

Games published by our own members! Check 'em out!

[gouessej]

This example will be improved of course :

try{
           ByteBuffer byteBuffer = BufferUtil.newByteBuffer(count*floatPerPrimitive*BufferUtil.SIZEOF_FLOAT);
           ByteBuffer byteBuffer2 = BufferUtil.newByteBuffer(13*BufferUtil.SIZEOF_INT);
           FileInputStream fis = new FileInputStream(new File(getClass().getResource("/pic256/worldmap.data").toURI()));
           FileChannel fc=fis.getChannel();
           fc.position(0);
           fc.read(byteBuffer2);
           fc.read(byteBuffer);
           fc.force(true);
           fc.close();
           byteBuffer.position(0);
           byteBuffer.order(ByteOrder.BIG_ENDIAN);
           levelCoordinatesBuffer.put(byteBuffer.asFloatBuffer());
           byteBuffer.position(0);          
           IntBuffer tmpLevelCoordinatesBuffer2=BufferUtil.newIntBuffer(13);
           byteBuffer2.position(0);
           byteBuffer2.order(ByteOrder.BIG_ENDIAN);
           tmpLevelCoordinatesBuffer2.put(byteBuffer2.asIntBuffer());
           tmpLevelCoordinatesBuffer2.position(0);
           byteBuffer2.position(0);
          }
       catch(URISyntaxException urise)
       {
           urise.printStackTrace();
       }   

[broumbroum]

I think this goes a bit to far of the subject, reading float buffers. But as far as I can imagine, you want to get the quicker effect with reading. This is done by specifying a BufferedInputStream to wrap the File. This can be done throughout the RandomAccessFile wrapper or a BufferedInputStream on the usual input.
Sequently, the input reader is increased by powers of 2 by the speed when it fits the environment system bitwise reading, that is, the harddisk or ram bus bandwidth. usually harddisk specific readers can read up to 512 bytes "in logical one pass" and the ram bus can be even bigger. That's why setting up a reader for more than 512 bytes/read is useless on current harddisk drives.
If you want objective reading, use a RandomAccessFile, if you want bytes buffers, you better switch to a common 512 bytes reading loop.

[gouessej]

I think this goes a bit to far of the subject, reading float buffers.

No it doesn't go far from the subject as "levelCoordinatesBuffer" is a float buffer and I perform "levelCoordinatesBuffer.put(byteBuffer.asFloatBuffer());". I really used a FileChannel to read a file and to put the content in a FloatBuffer by using an intermediary ByteBuffer.

But as far as I can imagine, you want to get the quicker effect with reading.

Yes, that's what I want.

This is done by specifying a BufferedInputStream to wrap the File. This can be done throughout the RandomAccessFile wrapper or a BufferedInputStream on the usual input.
Sequently, the input reader is increased by powers of 2 by the speed when it fits the environment system bitwise reading, that is, the harddisk or ram bus bandwidth. usually harddisk specific readers can read up to 512 bytes "in logical one pass" and the ram bus can be even bigger. That's why setting up a reader for more than 512 bytes/read is useless on current harddisk drives.
If you want objective reading, use a RandomAccessFile, if you want bytes buffers, you better switch to a common 512 bytes reading loop.

But what I did above worked fine...when the file I want to read isn't remote and isn't in a Java archive.