Java-Gaming.org    
Featured games (91)
games approved by the League of Dukes
Games in Showcase (577)
games submitted by our members
Games in WIP (498)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  ZipInputStream/GZIPInputStream faster than FileInputStream  (Read 1463 times)
0 Members and 1 Guest are viewing this topic.
Offline DrewLols

Senior Member


Medals: 1
Projects: 1


Noob going through metamorphosis...


« Posted 2013-04-08 02:24:48 »

So I'm writing a .json parser, and I'm giving my parser support for .json, .zip, and .gz.  Both zip and gz would be compressed versions of the json file.  When specifying input streams, it looks like this...

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
public JsonParser(String jsonFile)
{
      builder = new StringBuilder();
      try
      {
         // Determines file type
        String fileType = jsonFile.substring(jsonFile.lastIndexOf('.')+1);
         fileType = fileType.toLowerCase();
         
         // If it is a .json file...
        if(fileType.equals("json"))
            in = new DataInputStream(new FileInputStream(jsonFile));
         
         // Otherwise, if it is a .zip file...
        else if(fileType.equals("zip"))
         {
            ZipInputStream zis = new ZipInputStream(new FileInputStream(jsonFile));
            zis.getNextEntry();
            in = new DataInputStream(zis);
         }
         
         // Otherwise, if it is a .gz file...
        else if(fileType.equals("gz"))
         {
            in = new DataInputStream(new GZIPInputStream(new FileInputStream(jsonFile)));
         }
         
         // Otherwise...
        else
            throw new RuntimeException("Extension " + fileType + " not accepted by JsonParser");
      }
      catch(IOException e)
      {
         e.printStackTrace();
      }
}


For SOME reason, when parsing the file when looking for a particular token, it is significantly faster to find it when using a GZIPInputStream or a ZipInputStream.  In case you are interested, I am parsing a .json file generated by Tiled which is half of a megabyte in size uncompressed.  Many nearly empty layers.  Anyway, the file is always read byte by byte with a DataInputStream on the outside.  Why would this happen?

Did you know that 90% of statistics are wrong?
Offline DrewLols

Senior Member


Medals: 1
Projects: 1


Noob going through metamorphosis...


« Reply #1 - Posted 2013-04-08 02:28:36 »

 Grin  Wait, wait, I have a theory suddenly!  I think it is because reading in bytes is slow, and ZipInputStream/GZIPInputStream read in fewer bytes while generating a bunch of them through decompression.  Because it is much faster to generate bytes than read them in from a file, parsing it would be faster.

Did you know that 90% of statistics are wrong?
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 605
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #2 - Posted 2013-04-08 05:53:35 »

ZipInputStream and GZIPInputStream do buffer behind the scenes, whereas DataInputStream does not.

1  
new DataInputStream(new BufferedInputStream(new FileInputStream(...)))


That would solve your performance problem Smiley

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Roquen
« Reply #3 - Posted 2013-04-08 07:46:17 »

Compressed data also has the advantage of moving less data and the extra computation can be faster than the data moves.
Offline Nate

JGO Kernel


Medals: 128
Projects: 3
Exp: 14 years


Esoteric Software


« Reply #4 - Posted 2013-04-08 10:36:28 »

Eg, reading LZO compressed data is often faster than uncompressed.

Pages: [1]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

xsi3rr4x (18 views)
2014-04-15 18:08:23

BurntPizza (15 views)
2014-04-15 03:46:01

UprightPath (28 views)
2014-04-14 17:39:50

UprightPath (13 views)
2014-04-14 17:35:47

Porlus (29 views)
2014-04-14 15:48:38

tom_mai78101 (54 views)
2014-04-10 04:04:31

BurntPizza (111 views)
2014-04-08 23:06:04

tom_mai78101 (212 views)
2014-04-05 13:34:39

trollwarrior1 (181 views)
2014-04-04 12:06:45

CJLetsGame (187 views)
2014-04-01 02:16:10
List of Learning Resources
by Longarmx
2014-04-08 03:14:44

Good Examples
by matheus23
2014-04-05 13:51:37

Good Examples
by Grunnt
2014-04-03 15:48:46

Good Examples
by Grunnt
2014-04-03 15:48:37

Good Examples
by matheus23
2014-04-01 18:40:51

Good Examples
by matheus23
2014-04-01 18:40:34

Anonymous/Local/Inner class gotchas
by Roquen
2014-03-11 15:22:30

Anonymous/Local/Inner class gotchas
by Roquen
2014-03-11 15:05:20
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!