Questions about TGA Files

[Eliwood]

Few questions about the TGA format...

1) Is it true that TGA images will load in quicker than their PNG/GIF counterparts? If so, why?

2) Are there any good pre-written examples of TGA readers and writers out there?

[kevglass]

TGA images are percieved as loading faster because developers produce their own custom loaders rather than using ImageIO - which tends to be poor in performance for the formats it supports. This perception normally comes from creating textures rather than BufferedImages and writing a custom loader allows you to skip the BufferedImage creation steps.

The uncompressed version of TGA has the potential to be faster to load than PNG/GIF in the general case because you there is no decompression to perform - however, I would think this effect would be minimal.

Kev

[oNyx]

>no decompression to perform

If its in a jar there is decompression involved. However, thats only one decompression step and not two. Also if you use raw formats, which are packed into jars you end up with about the same size as gif/png anyways.

So, gif or png don't make much sense in java. The exception are applets where you load em directly from the server (one by one).

The speed difference is indeed minimal. Unless imageio is used.

BTW pre compressed (DXTn) textures are even faster to load. Simply because its less to load and there is also no decompression (unless you have to use the software decompression fallback in which case it would be still *very* fast).

>Are there any good pre-written examples of TGA readers and writers out there?

Sure. But what are you exactly looking for? Java2d, lwjgl, jogl... something else?

Games published by our own members! Check 'em out!

[Riven]

@kevglass
Eh, that is exactly the most significant part of it, decompression can take a lot of cpu-cyles on larger images.

300ms is not uncommon for 1024^2 PNG (image streamed from RAM).
With TGA it is basicly instantly loaded, less than 1ms (plain memcopy).

When the image is stored on a HDD, the HDD will become the bottleneck, but there is still a significant speed-difference (like factor 2 or 4...)

[Orangy Tang]

So, gif or png don't make much sense in java.

Thats a little unfair. A compression method specialised for a particular type of content is always going to have the advantage over a general purpose one.

[kevglass]

@kevglass
Eh, that is exactly the most significant part of it, decompression can take a lot of cpu-cyles on larger images.

I'm sure thats true, however I have a PNG loader locally thats implemented outside of ImageIO and has comparable performance to my TGA loader (which only loads decompressed tgas). ImageIO implementations definitely leave alot to be desired.

Kev

[oNyx]

So, gif or png don't make much sense in java.

Thats a little unfair. A compression method specialised for a particular type of content is always going to have the advantage over a general purpose one.

PNG is usually smaller than GIF. PNGOUT uses the same compression algo as kzip. LZMA is superior to deflate (zip/png).

A raw format with lzma compression is way smaller than the same with png/gif. With zip you only get about the same file size (raw->deflate vs deflate->deflate).

Only lossy formats can be smaller... so it seems (I tested this stuff a lot). So... use jpeg, wavelet, DXTn if possible and otherwise something raw (or gif/png for java2d stuff if you're feeling lazy... wont make much of a difference there either way... bout the same size... loads slightly slower).

If you don't believe me you can test this stuff on your own.

[Eliwood]

@kevglass
Eh, that is exactly the most significant part of it, decompression can take a lot of cpu-cyles on larger images.

I'm sure thats true, however I have a PNG loader locally thats implemented outside of ImageIO and has comparable performance to my TGA loader (which only loads decompressed tgas). ImageIO implementations definitely leave alot to be desired.

Kev

I think that a call like this would be part of the problem. I also wrap it in a MediaTracker (ick) to show the whole thing in one shot rather than gradually.
Toolkit.getDefaultToolkit().createImage(imageBytes);

IIRC, Toolkit is marginally better than using ImageIO but still within the same order of magnitude in terms of performance.

Perhaps switching that over to something more efficient would solve the problem.

@Oynx - LWJGL

[oNyx]

There is reader code over in feck and slick for example.

http://www.cokeandcode.com/feck
http://slick.cokeandcode.com/

Writing 24bit TGAs is very easy:
http://phpfi.com/119150 (OpenGL 1.1+)
http://www.matthiasmann.de/content/view/15/26/  (OpenGL 1.2+)

[Eliwood]

Just out of curiosity, what are my options for custom loaders if I want to stick to PNG files for the moment?

Games published by our own members! Check 'em out!

[oNyx]

OpenIL(devil) or ImageIO. However, both options suck.

Switch over to TGA and use ImageMagick for batch conversion.

mogrify -format tga *.png
(converts all *.png files to tga)

[Eliwood]

OpenIL(devil) or ImageIO. However, both options suck.

Switch over to TGA and use ImageMagick for batch conversion.

mogrify -format tga *.png
(converts all *.png files to tga)

Good idea. I'll just batch convert and just provide something to let all imported images get stored as TGA.

[tusaki]

TBH,

I don't really see the need to store my images as TGA.

Loading times are trivial whichever format you use and it would make more sense in my mind (just a feeling) to use a more "advanced" format like PNG. For some reason, to me, a project feels more "robust" and "mature" when it uses PNGs instead of TGAs. silly ofcourse... oh well.. just my 2ct's 

[oNyx]

>Loading times are trivial whichever format you use[...]

30sec+ vs <1sec on slower machines if ImageIO is used. And that for a single 512x512 skybox.

Longer loading times increase the upfront commitment, which decreases the replay rate.

>For some reason, to me, a project feels more "robust" and "mature" when it uses PNGs instead of TGAs.

How so? Because no space is saved and it loads slower? (If an installer is used, you would actually waste quite a lot of space.)

The average player wont look into your jars, but everyone will have to sit through the loading screens.

[Eliwood]

>Loading times are trivial whichever format you use[...]

30sec+ vs <1sec on slower machines if ImageIO is used. And that for a single 512x512 skybox.

Longer loading times increase the upfront commitment, which decreases the replay rate.

>For some reason, to me, a project feels more "robust" and "mature" when it uses PNGs instead of TGAs.

How so? Because no space is saved and it loads slower? (If an installer is used, you would actually waste quite a lot of space.)

The average player wont look into your jars, but everyone will have to sit through the loading screens.

Amen. My low end users currently have to put up with 5-10 second loading times using PNG's and Toolkit.createImage while the high end users still have to sit through a second or two. If the format my engine uses is completely transparent to the end user unless they snoop through my JAR's, does it really matter what I use?

[tusaki]

Well, I have to admit my projects usually involve lots of small files as opposed to large files. But 30 sec vs <1sec ... it sounds unbelievable... does ImageIO suck that bad?

[Eliwood]

Well, I have to admit my projects usually involve lots of small files as opposed to large files. But 30 sec vs <1sec ... it sounds unbelievable... does ImageIO suck that bad?

I think that's a little bit of an overstatement, but yes, it is quite bad if you are loading large files. When I use kev's TGA loader, I will give you some actual numbers reported by end users for comparison.

[darkprophet]

forget about TGA or PNG...DDS is the way to go for OpenGL games. Its simplish to parse and it includes mipmaps inside the format, so you dont need to calculate them. The reason I said DDS and not DXTn is because DDS is alot like TGA in the sense that DDS also supports raw, uncompressed data. However, when you use DXTn compression, its really rather neat.

The 2nd reason why I like DDS and a proprietary format is because there are plenty of importers/exporters for DDS already. But I have to admit, its not all rosy, it doesn't support floating point texture formats for use in HDR and things like that, but that should be remedied in DX10 i think..

DDS + TGA are all the formats your ever going to need IMHO.

DP

[oNyx]

>DDS + TGA are all the formats your ever going to need IMHO.

Well, dds also covers everything what tga can do (apart from that RLE crap, which no one needs). What dds lacks is something jpg/jp2 like. But the DXTn modes are a very appealing compromise, because it also saves vram.

Speaking of which... a rather complete dds loader would be nice to have.

[kevglass]

This tutorial seems to have a reasonable base for a DDS loader, if someone wanted to maybe tidy and generalise:

http://www.chriscohnen.de/cg/tutorials/DDStutorial/dds_tutorial.htm

Kev

[Evil-Devil]

This tutorial seems to have a reasonable base for a DDS loader, if someone wanted to maybe tidy and generalise:

http://www.chriscohnen.de/cg/tutorials/DDStutorial/dds_tutorial.htm

Kev

That loader suckz, as i can say. I used the loader in the first attempt and was very frustrated that much stuff had'nt worked. The easiest way to do your own DDS Loader is to stick into MSDN and NVidia documents. I preferered later one and found some good informations.

For some time i posted a small DDS loader at the lwjgl forums, since that i still not finished the loader due the lack of time.
It misses mipmaps and all the other advance stuff. Only basic loading is supported.
Anyway, to get a closer look take a look

http://www.evil-devil.com/dlfiles/dds_loader.rar

[kevglass]

Awesome

Kev

[Amos Wenger]

Matthiasman has implemented DDS loading suppor for Xith3D. I don't use it but I believe it's working well, though I don't know how complete it is..

[Eliwood]

Last question - I need to be able to load in TGA's into actual Java Images for use in say... the editor.

What's the easiest way to do this given that Kev's code returns a ByteBuffer? I tried transforming it into an IntBuffer, then an int[] to create a BufferedImage, but it seems that I can't do that because of the way the ByteBuffer was created.

[kevglass]

I'd change the code to load it into a raster or something, and build the buffered image from there.

Kev

[oNyx]

>I need to be able to load in TGA's into actual Java Images for use in say... the editor.

TGA image reader for ImageIO Available
http://www.java-gaming.org/forums/index.php?topic=2136.0

[Eliwood]

I actually tried that one last night, and it flopped on the images with transparency. That and... I didn't really think it was all that fast in loading in. Does that have something to do again with it being a plugin for ImageIO?

[Eliwood]

Running into some issues creating the raster. I inserted the following lines in a copy of your function at the bottom.

1   2   3   4   5   6   7   8   9   10   11   12   13   14   15   16   17   18   19   20   21   22   23   24

BufferedImage img = new BufferedImage (    TGALoader.getLastWidth(),    TGALoader.getLastHeight(),    TGALoader.getLastDepth() );       DataBufferByte bufferByte = new DataBufferByte(rawData, rawData.length);       img.setData (     Raster.createRaster     (    new BandedSampleModel    (       DataBuffer.TYPE_BYTE,       TGALoader.getLastWidth(),       TGALoader.getLastHeight(),       TGALoader.getLastDepth()    ),    bufferByte,    new Point(0, 0)     ) );

1   2   3   4   5   6

Exception in thread "AWT-EventQueue-0" java.lang.ArrayIndexOutOfBoundsException: 1    at java.awt.image.DataBufferByte.getElem(DataBufferByte.java:183)    at java.awt.image.BandedSampleModel.getPixels(BandedSampleModel.java:410)    at java.awt.image.Raster.getPixels(Raster.java:1569)    at java.awt.image.BufferedImage.setData(BufferedImage.java:1488)    at com.stencyl.data.util.TGALoader.loadIntoImage(Unknown Source)

Some quick print statements say that the image is loading properly though, so it's something in my code that's bad.

URL: Media/Backgrounds/Overworld 1.tga
Width: 1024
Height: 864
Depth: 8
Size: 3145728
DataBufferSize: 3145728

[kevglass]

Not sure which loader you're using (I have several for different things) but some of them generate a bigger buffer than the images for a POT texture in OpenGL.

Kev

[Eliwood]

I used this copy of your code.

http://www.cokeandcode.com/code/src/render/org/newdawn/render/texture/TGALoader.java

Edit: When I applied the same lines of code to your Slick TGALoader, I got this.

1   2   3

java.io.EOFException    at java.io.DataInputStream.readByte(DataInputStream.java:243)    at com.stencyl.data.util.TGALoader.loadImage2(Unknown Source)