texture compression

[Nate]

I have a game with 100mb+ in sprites. I only need a subset at any given time, so I don't load them all in memory at the same time. What can do I to reduce the file size of my game? What would be the suggested lossy compression?

Is this relevant?
http://blog.wolfire.com/2009/01/dxtc-texture-compression/

[Riven]

Yes. It is relevant.

[Nate]

Ok, I will look further into DXTC. I'm guessing it is the way to go since another option was not presented?

Games published by our own members! Go get 'em!

[Riven]

DXT{1|3|5} is uncompressed on the fly by the GPU.

Do not use DXT on gradients.

[princec]

DXT is also useful where you need a lot of textures in a scene.
Otherwise you might want to look at JPEG to compress your textures if all you're worried about is download size / disk space, which I'm having great results with.

Cas

[JL235]

Is this what DDS uses? I tried using DDS images instead of PNGs in one of my games and I found on several PCs at my university the game would crashing as the Intel cards were unable to handle them (or maybe old drivers to blame). Something to be aware of if your expecting your game to be able to run on office and low-end PCs.

Otherwise you might want to look at JPEG to compress your textures if all you're worried about is download size / disk space, which I'm having great results with.

Not if your expecting it to store any large blocks of colour, like a masking colour. JPEG typically messes them up.

I personally always use PNGs.

[Riven]

Is this what DDS uses?

Yes. DDS supports multiple (mipmapped) DXT images in one file

[Nate]

While I am really only concerned with download size, unfortunately I can't use JPEG because I need alpha.

Since I'm dealing with sprites, there is a bit of whitespace (especially for the larger sprites), even after I strip all whitespace from the edges. Because of this, DXT5 is giving me < 15% compression on average. In some cases, for spritesheets with larger sprites, the PNG is smaller than the DXT5. For spritesheets with many tiny sprites, I get up to ~30%.

I was hoping for better compression. At this point, I'm not sure DXT is worth the trouble for my specific scenario.

[Roquen]

If DL size is the main concern, then you could JPEG the RGB and PNG the alpha and merge.

[princec]

That's what I do - except I store both data in one file and have a customised loader.

Also, when saving the JPEGS, I use quality 1.0 (maximum) - it still provides for very good compression, but without the horrible colour artifacts.

Cas

Games published by our own members! Go get 'em!

[zoto]

I use PNG images and run them threw Pngcrush. PNG supports loss less compression and alpha.

[princec]

FWIW, you are better off not compressing pngs or your alpha channels in JPEG, and using a more powerful compression like LZMA on the final jar.

Cas

[JL235]

While I am really only concerned with download size, unfortunately I can't use JPEG because I need alpha.

You could try saving as an 8-bit PNG. It's lossly and only allow pixels to be fully transparent or opaque, but it's also much smaller.

[Abuse]

You could try saving as an 8-bit PNG. It's lossly and only allow pixels to be fully transparent or opaque, but it's also much smaller.

Only lossy in so much as you are forcing color reduction; the compression is still lossless as png uses zlib deflate.

I'm suprised nobody has suggested jpeg2000. Better quality & compression than jpeg, and supports alpha. A bit of a bitch to decompress though.

[Nate]

I also use PNG crush, but it doesn't reduce the file size by much.

No luck using 8-bit PNGs, my sprites have shadows and a lot of varying levels of transparency.

I very much like the idea of saving a JPG and alpha separately! I implemented it very rough and quick:
[snip, see below]
I'm seeing ~45% smaller sizes versus the input PNG. Sweet! Now I just need to make my code not nasty.

Does it make sense to use PNG at all? Would it make more sense to just write the alpha for each pixel, then the JPEG data, and let the JAR compression make it small?

This seems like a very useful utility. I'd love to hear more ideas on implementing this. Code would be fantastic as well! I plan to make it available through my (BSD) Skorpios project (which also means the code should be Android friendly). It also seems like it would fit in well in Slick.

[Riven]

Does it make sense to use PNG at all? Would it make more sense to just write the alpha for each pixel, then the JPEG data, and let the JAR compression make it small?

PNG is simply GZ with overhead and a header.
JAR is simply GZip.

No doubt you knew that, but as princec said, LZMA is way better than GZ. Building the installer using LZMA would yield the smallest download size, if all lossless image data is not compressed in any way.

[Nate]

It seems very difficult to find any tools supporting JPEG2000. I found one, but it converted without allowing me to specify quality. It took a 192kb file to 152kb. Not really a fair comparison but the same file ran through the (crappy) code I posted above gives 102kb.

[Roquen]

JPEG2000 should give better results for lower bit-rates.  The trick here is that's more expensive method that JPEG and you'll be stuck with a pure Java decoder.  My gut feeling is it's probably not worth the hassle.  If I remember correctly there are patent issues with JPEG200...but don't quote me on it.

[Nate]

I've improved my code a bit. It writes the alpha data, then the JPEG data to the same file (which I've cleverly dubbed ".img"). If I give it a 192kB PNG, it gives me a 540kB IMG file. If I zip that file it becomes 66.6kB, with LZMA it becomes 61.6kB. 68% smaller! Sweet!

To be fair, there is a noticeable loss of quality, probably because I'm not setting the JPEG quality. Whatever the default is must be pretty low. I'll have to figure out how to pass my InputStream to setOutput later, gotta run right now! The ImageIO API is rubbish!

Here are the classes:
[snip, see below]
Any comments, criticisms, or improvements are very welcome!

[Nate]

Ok, the latest will now always be here:

IMGEncoder.java
IMGDecoder.java

When I write the alpha, if it is 0 (transparent) or 0xFF (opaque) it is followed by the number of times that value is repeated. This greatly decreased the size of the resulting IMG file, and also results in a smaller file once ZIP'ed or LZMA'ed.

The main method in IMGDecoder writes out a PNG so you can see what the round trip has done to your image. See here for some code that loads the IMG into an OpenGL texture.

I look forward to any feedback or optimization ideas you might have for IMGDecoder. Optimizing IMGEncoder isn't nearly as important.

Edit: My sprites are down to ~30mb from ~110mb! That is after LZMA and with JPEG quality of 0.75.