Java-Gaming.org    
Featured games (79)
games approved by the League of Dukes
Games in Showcase (476)
Games in Android Showcase (106)
games submitted by our members
Games in WIP (533)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  Pure Java PNG loader, with efficient & clear code...  (Read 5980 times)
0 Members and 1 Guest are viewing this topic.
Offline DzzD
« Posted 2010-02-26 19:15:15 »

I have been looking for some PNGDecoder for a project I am working on, all was looking weird / complexe / dirty coded and hard to hunderstand, so last night and as often .... I've decided to reinvent the well and to built my own Smiley

- it support the 4 filter type for filter method 0
- for now only decoding of RGB24 & RGB32 are implemented, but adding GREY & PALETTE one should be a kid game (5/6 code line more for each)
- it is based on high efficiency & low memory usage (also did not allocate tones of buffer)
- only critical chunks are decoded but extended should be pretty easy to add

even if it is a first shot and it still need some improvment, I have tried to make the code as clear as possible, and it may be of interrest to better understand PNG

NB : some object are missing as RGBArrayImage and Log but can be easily replaced

EDIT : please follow the thread post to get the lastest version

Offline DzzD
« Reply #1 - Posted 2010-02-26 19:18:11 »

too long ... file is attached to this post

Offline Riven
« League of Dukes »

JGO Overlord


Medals: 743
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #2 - Posted 2010-02-26 19:31:32 »

Can you 'feed' the pixels line-by-line to the application, so that you can scale massive images?

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline DzzD
« Reply #3 - Posted 2010-02-26 20:03:55 »

Sorry, I dont understand exacly what you means but I try an answer, so first about the implementation you can inded do whatever is possible to do with PNG  format as it is a full decoder and than you can control what is done and how it is done.

about streaming and as far as I have understand :

=> PNG can break main data in several IDAT chunk
=> you can decode start of the image without the end
=> PNG doesn't allow uncompressing part of an image without uncompressing previous data (due to the zlib compression that compress the whole data once)

I dont get the point about scaling large image ?

ps: that's funny because I've worked on this decoder for such work (scaling huge streamed image) but not in the manner you descibe

Offline DzzD
« Reply #4 - Posted 2010-02-26 20:07:29 »

if anybody get problem on using it due to lack of RGBArrayImage let me know, this is just an object that represent an RGB image (with or without Alpha) and can return its integer RGB or ARGB pixels array (can be easily replaced by a BufferedImage)

Offline Abuse

JGO Coder


Medals: 11


falling into the abyss of reality


« Reply #5 - Posted 2010-02-26 20:20:15 »

Handy bit of code, though it's dependence on the java.util.zip.Inflater to decompress the DEFLATE stream limits it's use (and performance) somewhat.


Make Elite IV:Dangerous happen! Pledge your backing at KICKSTARTER here! https://dl.dropbox.com/u/54785909/EliteIVsmaller.png
Offline DzzD
« Reply #6 - Posted 2010-02-26 20:24:46 »

Handy bit of code, though it's dependence on the java.util.zip.Inflater to decompress the DEFLATE stream limits it's use (and performance) somewhat.
you are right, yup  Undecided, I would have prefer a pure one two but.... I believe some exist (for any platform too) but really not have the motivation to look fot it

Offline DzzD
« Reply #7 - Posted 2010-02-26 20:39:58 »

here is one : http://www.jcraft.com/jzlib/

I did not look to it yet, dont know how it is complexe,performing or well designed but at least it may this time enable "very very pure java PNG" decompression Smiley

Offline Riven
« League of Dukes »

JGO Overlord


Medals: 743
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #8 - Posted 2010-02-26 22:07:07 »

ps: that's funny because I've worked on this decoder for such work (scaling huge streamed image) but not in the manner you descibe

Well, you guessed right, as that's exactly what I meant. I've written such code, for fun and profit, and it worked with an interface like:

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
public interface StreamingImage
{
    public void initiateImage(int w, int h);
    public void scanlineAvailable(int[] rgb);
    public void reachedEnd();
}

imageSource.stream(inputStream, new StreamingImage() {
      ...
   }
);




I really need something similar for JPEG (streaming per MCU instead of scanline), but I don't feel like writing it ATM.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline DzzD
« Reply #9 - Posted 2010-02-26 22:14:42 »

I really need something similar for JPEG (streaming per MCU instead of scanline), but I don't feel like writing it ATM.
as PNG I never found any clear coded JPG Decoder, and unfortunatly JPG is not as simple as PNG...

may I ask what exacly are you looking for ? more details ?

Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 743
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #10 - Posted 2010-02-26 22:25:58 »

as PNG I never found any clear coded JPG Decoder, and unfortunatly JPG is not as simple as PNG...

may I ask what exacly are you looking for ? more details ?

Memory constraints. People are uploading photos, not knowing they are 4000x3000 pixels, and to make them usable, the server has to scale it back to a reasonable amount. Currently, for JPGs, I launch a seperate JVM with enough RAM to prevent OOME. I need massive heap because those 4000x3000 images take much more RAM than the expected 36MB. I keep the JVM heap of the 'real app' as small as possible to keep GCs down to millis, as garbage collections on heaps of >1024M are downright destructive for a somewhat-near-realtime app.

Besides server stuff, for some applets, I'm dealing with fixed size 64MB heaps, of which I have 10MB to spare for image resizing. The heap can be set to be bigger since 6u10, but I have to support 1.4

Anyway, long story short, reducing every MCU to 1 pixel would be my goal.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline DzzD
« Reply #11 - Posted 2010-02-26 22:38:58 »

Anyway, long story short, reducing every MCU to 1 pixel would be my goal.
ok I better understand now, I think it is pretty doable (but may give some headack...)

may be have a look to this thread you posted in it some times ago, take a decoder and try to modify it.

reducing MCU to one pixel should not even requiere to perform IDCT : as if I am not wrong and if I remember well you look for the average color wich should be the frequency 0 (continous component signal) => the first FFT entries ( some explanation here ) (EDIT : hum...  this is very only theorical... really not sure of what I said)

EIDT : Wikipedia is alos pretty nice for JPG

Offline Riven
« League of Dukes »

JGO Overlord


Medals: 743
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #12 - Posted 2010-02-26 22:57:32 »

ok I better understand now, I think it is pretty doable (but may give some headack...)

may be have a look to this thread you posted in it some times ago, take a decoder and try to modify it.

reducing MCU to one pixel should not even requiere to perform IDCT : as if I am not wrong and if I remember well you look for the average color wich should be the frequency 0 (continous component signal) => the first FFT entries ( some explanation here ) (EDIT : hum...  this is very only theorical... really not sure of what I said)

EIDT : Wikipedia is alos pretty nice for JPG

What Nate said: GPL Sad

It's really interesting as a reference, but I certainly don't want to 'taint' my project with such a license.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 743
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #13 - Posted 2010-02-26 23:08:34 »

I'm browsing your code, and this worries me a bit:
in.skip(size);

InputStream.skip() is like InputStream.read(byte[]) in that it might or might not read/skip the requested amount of bytes. You might be confused by the default implementation of the skip method, but things like FileInputStream and a Socket.getInputStream can/will have their own implementation that are more likely to skip less bytes.

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
public static void skipFully(InputStream in, long bytes) throws IOException
{
   while(bytes > 0)
   {
      long skipped = in.skip(bytes);
      if(skipped <= 0)
         throw new EOFException();
      bytes -= skipped;
   }
}

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline DzzD
« Reply #14 - Posted 2010-02-26 23:20:15 »

I'm browsing your code, and this worries me a bit:
yes I said it was " a first shot" made last night, feel free to correct it and post your update. it means that skip may not skip the amount requested ? EDIT : ok I see your code, nice to know

nothing related but here is the best JPG reference I can find

Offline DzzD
« Reply #15 - Posted 2010-02-26 23:29:25 »

I have update my version with the above skipFully method, If you got some others advice / bug found, let me know I will add them and post the corrected version in a later a post

Offline Abuse

JGO Coder


Medals: 11


falling into the abyss of reality


« Reply #16 - Posted 2010-02-27 00:14:51 »

here is one : http://www.jcraft.com/jzlib/

I did not look to it yet, dont know how it is complexe,performing or well designed but at least it may this time enable "very very pure java PNG" decompression Smiley

When I wrote an M3G Loader for j2me (a work-around for phones that have serious bugs in their native M3GLoaders), I wrote a light weight zlib inflater (necessary because the M3G format uses DEFLATE streams in a similar way to PNG.)
It was based upon the inflater code found in LodePNG.

Although the code is dire to read (and in C *shudder*), it was preferable to attempting to remove the OO structure (and thus bloat) from many of the existing zlib inflater implementations.
When I was looking for a candidate library to rip-off, it became apparent that the kind of people who write zlib libraries are the same kind of people who are appalling at documenting their code!

And my comments from the time....  Roll Eyes
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
         /**
          * small bytecode, 'fast', currently doesn't support windowed decompression. (only the entire stream in one go)
          * Based upon LodePNG (see /oldStuff/LodePNG.c )  
          */

         final int MY_INFLATOR = 0;
         /**
          * open source zlib decompressor that meets the specification set down for the j2se libs
          * massive bytecode.
          * Documented a little.
          */

         final int JAZZLIB_INFLATOR = 1;
         /**
          * open source zlib decompressor, totally undocumented, horrible code.
          * massive bytecode.
          */

         final int JZLIB_INFLATOR = 2;

Make Elite IV:Dangerous happen! Pledge your backing at KICKSTARTER here! https://dl.dropbox.com/u/54785909/EliteIVsmaller.png
Offline DzzD
« Reply #17 - Posted 2010-02-27 00:31:53 »

Smiley funny comments,

clearly LodePNG is just horrible to read .... , no candidate to write a pure deflate ?? I looked the specs a little when I read you previous post it was seeming doable, dont know if it would requier a long time

here are the spec and as always they are hard to read... seems that people writing RFC/spec and such never found the bold/underline/etc... and other words editor like buttons....

Offline DzzD
« Reply #18 - Posted 2010-02-27 01:06:50 »

arf ... hard...

Quote
Symbol  Code
------  ----
A       10
B       0
C       110
D       111

I.e., 0 precedes 10 which precedes 11x, and 110 and 111 are lexicographically consecutive.


they say 0 precedes 10 ?? It does not seems to be the case , do I missed smething ?

Offline CommanderKeith
« Reply #19 - Posted 2010-02-27 01:49:37 »

Your code is very nice and readable, thanks for sharing  Smiley

I saw that the MINA apache networking project uses jzlib (http://www.jcraft.com/jzlib/) for zipping and I thought they must know the best choice so that's what I normally use too.

By the way, if you get time it would be great to see the code of RGBArrayImage

cheers  Cool
Keith

Offline DzzD
« Reply #20 - Posted 2010-02-27 02:16:28 »

I saw that the MINA apache networking project uses jzlib (http://www.jcraft.com/jzlib/) for zipping and I thought they must know the best choice so that's what I normally use too.
probably a good solution but too late now ! I have already started to work on my own, inded the main interrest is learning but it will be also cool to have a full pure java PNG, also deflating inflating seems to not requiere tones of code, I've fund a nice flash source code that may help me in this work.

the RGBArrayImage have really nothing  fancy, I just did not want fullfill my first post :
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  
42  
43  
44  
45  
46  
47  
48  
49  
50  
51  
52  
53  
54  
55  
56  
57  
58  
59  
60  
61  
62  
63  
64  
65  
66  
67  
68  
69  
70  
71  
72  
73  
74  
75  
76  
77  
78  
79  
80  
81  
82  
83  
84  
85  
86  
87  
88  
89  
90  
91  
92  
93  
94  
95  
96  
97  
98  
99  
100  
101  
102  
103  
104  
105  
106  
107  
108  
109  
110  
111  
112  
113  
114  
115  
import java.awt.image.*;
import java.awt.*;

/**
 * Image RGB dont les pixels peuvents etre lues et modifiés via un tableau d'entier
 */

public class RGBArrayImage
{
   private static boolean warningMIS=false;
   private int width;
   private int height;
   private int pixels[];
   private Image image;
   private MemoryImageSource mis;

    public RGBArrayImage(int width,int height,boolean alpha)
    {
       this.width=width;
       this.height=height;
       this.image=null;
       this.mis=null;
       this.pixels=null;
       
       try
       {
          Class c = Class.forName("java.awt.image.BufferedImage");
          PixelsBufferBI pbi=new PixelsBufferBI(this.width,this.height,alpha);
          this.pixels=pbi.getPixels();
          this.image=pbi.getImage();
       }
       catch(ClassNotFoundException e) //Old JVM ?
      {
          if(!warningMIS)
             Log.log("Old JVM => using MemoryImageSource");
          warningMIS=true;
          this.pixels=new int[this.width*this.height];
         
          if(alpha)
             this.mis=new MemoryImageSource(this.width,this.height,new DirectColorModel(32, 0xff0000, 0xff00, 0xff,0xFF000000), this.pixels, 0, this.width);
          else
             this.mis=new MemoryImageSource(this.width,this.height,new DirectColorModel(24, 0xff0000, 0xff00, 0xff), this.pixels, 0, this.width);
             
          this.mis.setAnimated(true);
         this.mis.setFullBufferUpdates(true);
         this.image = Toolkit.getDefaultToolkit().createImage(this.mis);  
       }
    }
   
    public void updateMIS()
    {
       if(this.isMIS())
          this.mis.newPixels();
    }
   
    public boolean isMIS()
    {
       return (this.mis!=null);
    }
   
    public RGBArrayImage(int width,int height)
    {
       this(width,height,false);
    }
   
    public Image getImage()
    {
       return this.image;
    }
   
    public int[] getPixels()
    {
       return this.pixels;
    }    
   
   
    public RGBArrayImage(Image image)
    {
       this(image.getWidth(null),image.getHeight(null));
       PixelGrabber pg=new PixelGrabber(image,0,0,this.width,this.height,this.pixels,0,this.width);
       try
       {
          pg.grabPixels();
       }
       catch(InterruptedException ie)
       {
          Log.log(ie);
       }
    }
   
   
    class PixelsBufferBI
   {
      private Image image;
      private int pixels[];
      public PixelsBufferBI(int width,int height,boolean alpha)
      {  
            BufferedImage BIImage;
            if(alpha)
               BIImage = new BufferedImage(width, height,BufferedImage.TYPE_INT_ARGB);
            else
               BIImage = new BufferedImage(width, height,BufferedImage.TYPE_INT_RGB);
            pixels=((DataBufferInt)(BIImage.getRaster().getDataBuffer())).getData();
            image=(Image)BIImage;        
      }
      public Image getImage()
      {
         return this.image;
      }
      public int[] getPixels()
      {
         return this.pixels;
      }
   }
   
}


and interresting stuff would be to make it  handle other kind of image implementation by ading inner classes and it will chose the best one

Offline DzzD
« Reply #21 - Posted 2010-03-01 07:24:43 »

little improvment :
 - add of grey & grey alpha
 - replacement of skip by skipFully

and if anyone got some spare time : you're welcome to make improvment and post them Smiley (here are the spec I've used for PNG http://www.w3.org/TR/PNG/#4Concepts.EncodingFiltering )

Missing are :
 8 bit Palette Image (should be the next that I'll do)
 Interlace method
 16 bit color images ( does it is really requiered ? )
 Extended chunks ( boring...)
 Custom deflater ( we will see ... )

I began a pure java deflater but specs are really unclear (RFC 1951) and I did not got a lot of time to work on it, so it wont be finished soon, but I ll keep this thread up if...


Pages: [1]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

pw (18 views)
2014-07-24 01:59:36

Riven (17 views)
2014-07-23 21:16:32

Riven (14 views)
2014-07-23 21:07:15

Riven (17 views)
2014-07-23 20:56:16

ctomni231 (45 views)
2014-07-18 06:55:21

Zero Volt (41 views)
2014-07-17 23:47:54

danieldean (32 views)
2014-07-17 23:41:23

MustardPeter (36 views)
2014-07-16 23:30:00

Cero (51 views)
2014-07-16 00:42:17

Riven (50 views)
2014-07-14 18:02:53
HotSpot Options
by dleskov
2014-07-08 03:59:08

Java and Game Development Tutorials
by SwordsMiner
2014-06-14 00:58:24

Java and Game Development Tutorials
by SwordsMiner
2014-06-14 00:47:22

How do I start Java Game Development?
by ra4king
2014-05-17 11:13:37

HotSpot Options
by Roquen
2014-05-15 09:59:54

HotSpot Options
by Roquen
2014-05-06 15:03:10

Escape Analysis
by Roquen
2014-04-29 22:16:43

Experimental Toys
by Roquen
2014-04-28 13:24:22
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!