Image manipulation

[kingstone]

In our upcoming game we (SLX team, GBG Sweden) plan to use "post processing" on tilesets and sprites, i.e. we'll apply filters modifying Hue, Saturation, Lighting and such after an image is loaded.

1   2   3   4   5   6

//Pseudo code int[] pixels = new int[width*heigth]; img.getRGB(pixels,0,width,0,0,width,height); img=null; System.gc(); Thread.yield();   //To save some heap before recreating image processPixels(pixels); Image dest = Image.createRGBImage(pixels, width, height, true);

This poses a few problems. I hope to get some feedback before we start off the development process, and I'd like to share some thoughts as well.
First off, the code above will require a lot of free heap memory. It is a good idea to load image data in order of descending size to avoid out-of-memory errors.
Then there is the issue of phone model compatibility. Due to a buggy java implementation getRGB() and createRGBImage() will not work on Nokia 6600 and Nokia 7610. Since I plan to load-release-reload a lot of gfx the Nokia Series 60 phones (6600, 6630, etc.) will run out of memory eventually because of memory leaks. I also know that Nokia 6630 ignores all types of transparency; Nokia 6680 handles raw alpha channels correctly, but ignores color-key and palette transparency. What other bugs on these or other phones should one be aware of? We really should have a common list of known issues in J2ME implementations so one easily can get an overview.

As this is a commercial project, we aim to have the game compatible with lots of phone models, do you think this is all impossible if we use these post processing techniques?
Feedback please!

Mathias Johansson
SLX Entertainment

[Anon666]

I don't think your list of issues above is correct.

In our experience the 6630 and 6680 do not have any problems with transparency, the 2 are infact considered near-compatibles.
To my knowledge the 6600, 6630, 6680 ect do not leak memory on image creation. (unlike the 3650, and 7650 which leak loads).
Drawing images with drawRegion(with transforms) on the 6600 & 6630 will however leak memory, and will cause application slow-down and eventual crashes after a few 100,000 invocations. (less than a minute of application execution)
On a side note, Thread.yield() has issues on some handsets.

In general, I would suggest stearing well clear of getRGB & createRGBImage - it will almost certainly cause you no-end of compatability issues.

A superior alternative would be to perform the color filtering before the png is created - operating on the PLTE chunk of the raw png data.
This will be alot faster (as you will be able to operate on the palette directly), and will not be prone to any device issues. (in the vodafone drop list, there isn't a single handset that doesn't support transparency in png images)
The png format is simple to manipulate, and well documented - it should not take more than a few days to implement what you are wanting.

I believe we already have some code to do modulation of png palettes (for colorizing monochrome graphical fonts), unfortunately, its not my IP, so I can't simply give it away. (and certainly not for comercial purposes )

[kingstone]

Seems like I've got some things mixed up, let's see.

In our experience the 6630 and 6680 do not have any problems with transparency, the 2 are infact considered near-compatibles.

I'm refering to bug 2.21 in the 2nd Edition Platforms Known Issues v2.9 pdf that affects S60 2nd Edition, FP2 and FP3, all S60 2nd Edition, FP2 devices, Nokia N90 (SW 2.0530.3.5). Now that I look at it again I realize that it concerns the Nokia M3G (JSR-184) implementation, and since Im not dealing with 3D i guess I shouldn't worry about it.

To my knowledge the 6600, 6630, 6680 ect do not leak memory on image creation. (unlike the 3650, and 7650 which leak loads).

My bad, I got that impression from a thread on forum.nokia.com. So the 3650 and 7650 are the only handsets with createImage problems?

Drawing images with drawRegion(with transforms) on the 6600 & 6630 will however leak memory, and will cause application slow-down and eventual crashes after a few 100,000 invocations. (less than a minute of application execution)

I'm not currently using drawRegion but does this leak affect other any functions, i.e. are there other functions (e.g. Sprite.paint or TiledLayer.paint) that uses drawRegion?

On a side note, Thread.yield() has issues on some handsets.

Thanks, I'll use sleep(), for 20-50ms as adviced in the Known Issues In The Nokia 6600 MIDP 2.0 Implementation pdf instead.

In general, I would suggest stearing well clear of getRGB & createRGBImage - it will almost certainly cause you no-end of compatability issues.

A superior alternative would be to perform the color filtering before the png is created - operating on the PLTE chunk of the raw png data.
This will be alot faster (as you will be able to operate on the palette directly), and will not be prone to any device issues. (in the vodafone drop list, there isn't a single handset that doesn't support transparency in png images)
The png format is simple to manipulate, and well documented - it should not take more than a few days to implement what you are wanting.

I believe we already have some code to do modulation of png palettes (for colorizing monochrome graphical fonts), unfortunately, its not my IP, so I can't simply give it away. (and certainly not for comercial purposes )

Im going to take your advice and stay clear of createRGBImage and getRGB, they really seem to cause nothing but problems. Apart from the max 4k pixels bug on Nokia 6600 this method will result in a considerable heap usage peak which will make lots of older handsets run out of memory.

I had a brief look at the official png specification at http://www.w3.org/TR/PNG/ a while back and it seemed like it would take some time to figure out how the modify the PLTE chunk and fix the CRC and such. So if you have any links to threads, articles or tutorials on that it would be really appreciated. As for the color manipulation bit, there are lots of code and pseudo code available that converts RGB to HSL and vice versa but they are all using floats which is not supported by the J2ME. But I don't think that will be so much of a problem and I'll make sure I'll post the integer RGB<->HSL functions here when they're done in case anyone else is interested. Making a commercial game doesn't mean you can't share your code or ideas with other developers, just that you try and earn some money in the process. We are actually discussing if we should distribute the games for free on our website in addition to the commercial distribution.

Thanks for the response!

Games published by our own members! Check 'em out!

[OverKill]

Lemme go through the posts and add my comments:

The techniques you are describing are midp2.0 specific so you won't be able to seamlessly (like that ever happens in j2me) port the stuff to midp1.0

Images in heap:
This depends on the device. I have not played with all the devices but can tell you that some devices will decompress all images and hold them kinda like bmps in the memory. So holding them in a buffer and dumping the image would not be a great loss. IF there is a loss.

The nokia problem with the mem leak comes from the phones not holding the images in java but in native heap.
And like Anon said, it is the 3650 and 7650.
One workaround is to use nokias direct graphics. (similar to the getRGB stuff)
The other, prefetch and hold all your images.

A list of device problems is the foundation of any serious j2me development company. You probably won't have a chance getting a complete list.

Anon is pretty smart and we do similar as well.
PNG manipulation is pretty easy, though on some devices, you also need to modify the checksums.
If you are aiming for j2me, make sure your code targets all devices.

[kingstone]

OverKill: I thought it'd be impossible to have the game im designing midp 1.0 compatible but I'm going to read up on how they differ before i decide that. Maybe the real question is if the midp 1.0 handsets have enough heap.. Im not sure what you mean by all devices but I'll of course try and make it compatible with as many as possible.

Too bad that game development companies don't share bug lists or ideas & source code for that matter among each other, it would benefit the mobile games and therefor the market as a whole and I don't think anyone would loose anything on it really.

I might try using DirectGraphics for the 3650 and 7650 models although ive read some threads that it wount solve the problem (while others believe it does help).

I got the png palette manipilation working last week including, recalculating checksums. I've implemented a few simple functions (e.g. colorize) so far but I havn't begun working with the RGB<->HSL conversion yet.

[kingstone]

RGB->HSL & HSL->RGB code is up and running. If anyone is interested in it, let me know and i'll mail or post the code for you.

[OverKill]

Yes, it would make mobile development easier but then again I can understand the comps as they invest a lot of money into this stuff.
So sadly, unless my comp says I can share stuff, I sadly have to keep it to myself.

Well with midp1.0 you can target the greatest amount of devices.
All devices I mean to target as many as you can.