Java-Gaming.org    
Featured games (81)
games approved by the League of Dukes
Games in Showcase (499)
Games in Android Showcase (118)
games submitted by our members
Games in WIP (568)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  ZipInputStream/GZIPInputStream faster than FileInputStream  (Read 1764 times)
0 Members and 1 Guest are viewing this topic.
Offline DrewLols

Senior Member


Medals: 3
Projects: 1


Noob going through metamorphosis...


« Posted 2013-04-08 00:24:48 »

So I'm writing a .json parser, and I'm giving my parser support for .json, .zip, and .gz.  Both zip and gz would be compressed versions of the json file.  When specifying input streams, it looks like this...

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
public JsonParser(String jsonFile)
{
      builder = new StringBuilder();
      try
      {
         // Determines file type
        String fileType = jsonFile.substring(jsonFile.lastIndexOf('.')+1);
         fileType = fileType.toLowerCase();
         
         // If it is a .json file...
        if(fileType.equals("json"))
            in = new DataInputStream(new FileInputStream(jsonFile));
         
         // Otherwise, if it is a .zip file...
        else if(fileType.equals("zip"))
         {
            ZipInputStream zis = new ZipInputStream(new FileInputStream(jsonFile));
            zis.getNextEntry();
            in = new DataInputStream(zis);
         }
         
         // Otherwise, if it is a .gz file...
        else if(fileType.equals("gz"))
         {
            in = new DataInputStream(new GZIPInputStream(new FileInputStream(jsonFile)));
         }
         
         // Otherwise...
        else
            throw new RuntimeException("Extension " + fileType + " not accepted by JsonParser");
      }
      catch(IOException e)
      {
         e.printStackTrace();
      }
}


For SOME reason, when parsing the file when looking for a particular token, it is significantly faster to find it when using a GZIPInputStream or a ZipInputStream.  In case you are interested, I am parsing a .json file generated by Tiled which is half of a megabyte in size uncompressed.  Many nearly empty layers.  Anyway, the file is always read byte by byte with a DataInputStream on the outside.  Why would this happen?

Did you know that 90% of statistics are wrong?
Offline DrewLols

Senior Member


Medals: 3
Projects: 1


Noob going through metamorphosis...


« Reply #1 - Posted 2013-04-08 00:28:36 »

 Grin  Wait, wait, I have a theory suddenly!  I think it is because reading in bytes is slow, and ZipInputStream/GZIPInputStream read in fewer bytes while generating a bunch of them through decompression.  Because it is much faster to generate bytes than read them in from a file, parsing it would be faster.

Did you know that 90% of statistics are wrong?
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 802
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #2 - Posted 2013-04-08 03:53:35 »

ZipInputStream and GZIPInputStream do buffer behind the scenes, whereas DataInputStream does not.

1  
new DataInputStream(new BufferedInputStream(new FileInputStream(...)))


That would solve your performance problem Smiley

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Online Roquen
« Reply #3 - Posted 2013-04-08 05:46:17 »

Compressed data also has the advantage of moving less data and the extra computation can be faster than the data moves.
Offline Nate

JGO Kernel


Medals: 149
Projects: 4
Exp: 14 years


Esoteric Software


« Reply #4 - Posted 2013-04-08 08:36:28 »

Eg, reading LZO compressed data is often faster than uncompressed.

Pages: [1]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

Pippogeek (39 views)
2014-09-24 16:13:29

Pippogeek (30 views)
2014-09-24 16:12:22

Pippogeek (20 views)
2014-09-24 16:12:06

Grunnt (46 views)
2014-09-23 14:38:19

radar3301 (28 views)
2014-09-21 23:33:17

BurntPizza (64 views)
2014-09-21 02:42:18

BurntPizza (36 views)
2014-09-21 01:30:30

moogie (42 views)
2014-09-21 00:26:15

UprightPath (51 views)
2014-09-20 20:14:06

BurntPizza (54 views)
2014-09-19 03:14:18
List of Learning Resources
by Longor1996
2014-08-16 10:40:00

List of Learning Resources
by SilverTiger
2014-08-05 19:33:27

Resources for WIP games
by CogWheelz
2014-08-01 16:20:17

Resources for WIP games
by CogWheelz
2014-08-01 16:19:50

List of Learning Resources
by SilverTiger
2014-07-31 16:29:50

List of Learning Resources
by SilverTiger
2014-07-31 16:26:06

List of Learning Resources
by SilverTiger
2014-07-31 11:54:12

HotSpot Options
by dleskov
2014-07-08 01:59:08
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!