Java-Gaming.org    
Featured games (79)
games approved by the League of Dukes
Games in Showcase (476)
Games in Android Showcase (106)
games submitted by our members
Games in WIP (530)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1] 2 3 ... 6
  ignore  |  Print  
  Java 4k Resources Thread  (Read 40229 times)
0 Members and 1 Guest are viewing this topic.
Offline Eli Delventhal

JGO Kernel


Medals: 42
Projects: 11


Game Engineer


« Posted 2009-12-01 17:28:12 »

I think that there are so many different compression tools and byte-saving techniques that the common programmer doesn't know about that it can be a real detriment to them, and often a bit unfair. We've had threads like this one before, so I figured I'd start a new one.

Applet Templates
Start with a good applet template. It will have all the base functionality you need in the fewest bytes possible.
http://www.java-gaming.org/topics/applet-templates/21626/view.html

Compress your class
There are several ways you can do this. Riven has amazingly provided us with an HTTP service that does all the dirty work for you.
Compile 'n Shrink - HTTP Service - http://www.indiespot.net/app/java-four-kay

If you do not use Riven's tool, then you should use Pack200 and at least one compression tool (maybe all of them!). This can be a bit time consuming but will save you a massive number of bytes.

Pack200
No matter how you do it, you will want your game to eventually be compressed with Pack200.
http://java.sun.com/j2se/1.5.0/docs/api/java/util/jar/Pack200.html
http://java.sun.com/j2se/1.5.0/docs/guide/deployment/deployment-guide/pack200.html

Compression Tools and Obfuscators
There are lots of different tools out there to squeeze a few extra bytes out of your jar. I will eventually include specific guides with how to use each tool as they are written.
ProGuard - http://proguard.sourceforge.net/
JShrink - http://www.e-t.com/jshrink.html
JoGa - http://www.nq4.de/

Smart Techniques
Note that a lot of these things will be automatically adjusted by most optimizers. Still isn't it fun to do a lot of it manually? Smiley
  • All global variables should be named only a single character. Local variables can have whatever name you want.
  • All constants should be declared static final so they can go in the constants pool.
  • In general, you should use the shortest possible method names you can, like System.nanoTime() versus System.currentTimeMillis().
  • Every string literal you create adds at least a couple bytes per character. As a result, keep your strings short.
  • Every time you use a method, it will add bytes equal to the number of characters in that method. However, each method will be added only once for all uses.
  • Keep your code all in one class, and put all your code in one Applet method (like start()).
  • Use local variables wherever possible. Because your game is in one method, you should be able to avoid using almost any globals.
  • When concatenating numbers to strings, use String.valueOf() rather than directly adding them together. This will save many bytes.

External Links
A nice big 4k guide: http://wiki.java.net/bin/view/Games/4KGamesDesign

Please please add replies to this, especially extra tips and guides on how to use the compression tools and pack200. We want this thread to eventually be full of step-by-step guides so that people can focus on their code.

See my work:
OTC Software
Offline Abuse

JGO Coder


Medals: 11


falling into the abyss of reality


« Reply #1 - Posted 2009-12-01 18:21:05 »

I'm not sure whether the below was an intentional simplification, if so I apologise for being overly picky with my technical correctness Roll Eyes

Quote
Every string literal you create adds exactly as many bytes as there are characters

Due to class files using a UTF-8 encoding (Modified UTF-8) to store string literals, this is not true.
Every string literal ( sl ) you create adds atleast 2 + sl.length bytes, and at most 2 + sl.length*3 bytes.

So if you ignore the 2 byte length, and limit your string to containing only characters from the first 127 positive byte values (~= western-latin alphabets), your statement is sort-of accurate - beyond that, it gets progressively more incorrect.

Make Elite IV:Dangerous happen! Pledge your backing at KICKSTARTER here! https://dl.dropbox.com/u/54785909/EliteIVsmaller.png
Offline pjt33
« Reply #2 - Posted 2009-12-01 18:28:13 »

  • All global variables should be named only a single character. Local variables can have whatever name you want.
  • All constants should be declared static final so they can go in the constants pool.
  • Every time you use a method, it will add bytes equal to the number of characters in that method. However, each method will be added only once for all uses.
If the obfuscators aren't capable of doing trivial optimisations like these then are they not completely useless? More useful advice would be to use a single character name for the class, because obfuscators won't rename that.

Quote
  • Keep your code all in one class, and put all your code in one Applet method (like start()).
I'm pretty sure that consensus from the earlier template discussion was that if you want your applet to behave correctly you have to spawn off a thread.
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline appel

JGO Wizard


Medals: 50
Projects: 4


I always win!


« Reply #3 - Posted 2009-12-01 18:33:17 »

Goodies link page from last year:

http://www.java-gaming.org/topics/links-tips-and-goodies/19461/view.html

Check out the 4K competition @ www.java4k.com
Check out GAMADU (my own site) @ http://gamadu.com/
Offline Eli Delventhal

JGO Kernel


Medals: 42
Projects: 11


Game Engineer


« Reply #4 - Posted 2009-12-01 20:57:43 »

Oh yeah, I meant to add: correct me my mistakes! Some things (like the one byte per character for strings) were pretty much me working from memory of lsat year, I'm sure I've got plenty of little issues up there.

Thanks for the clarifications. I'll keep editing this as we get more info. I have no experience with the optimizers, so if some things are pointless then I won't really know it.

I'm personally annoyed I have to use optimizers to be able to contend - the fun part for me is putting in static final etc. etc. in order to get the bytes down. And then doing a smart thing with a loop, etc. When your code literally goes from 4kb to 1/2kb with optimizers and compression, that's kind of frustrating. Part of the reason I made this thread is because last year I was a bit overwhelmed with the incredibly large number of things I had to do to compress my game, none of which had anything to do with code. I think it's a really steep hill you have to climb as someone new to 4k.

See my work:
OTC Software
Offline Markus_Persson

JGO Wizard


Medals: 14
Projects: 19


Mojang Specifications


« Reply #5 - Posted 2009-12-01 21:52:47 »

Pack200 brings my current project down to 1.47 kb from 2.05 kb. That's very very good. Cheesy

Play Minecraft!
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 742
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #6 - Posted 2009-12-01 21:53:17 »

If somebody has the tools that run on linux, I can make my server happy by doing some bruteforce zip-attacks on your cute JARs.

Last year we such an app, but had some reliance on *.exe file, and it was macroing some GUI, and my server is blind. (who did it? Appel??)

So if anybody it willing to give it a gentle kick towards linux, I'm going to write some HTTP service that does all the naughty stuff for you.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline appel

JGO Wizard


Medals: 50
Projects: 4


I always win!


« Reply #7 - Posted 2009-12-01 21:58:52 »

If somebody has the tools that run on linux, I can make my server happy by doing some bruteforce zip-attacks on your cute JARs.

Last year we such an app, but had some reliance on *.exe file, and it was macroing some GUI, and my server is blind. (who did it? Appel??)

So if anybody it willing to give it a gentle kick towards linux, I'm going to write some HTTP service that does all the naughty stuff for you.

I don't understand. Any security threat I need to worry about?

Check out the 4K competition @ www.java4k.com
Check out GAMADU (my own site) @ http://gamadu.com/
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 742
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #8 - Posted 2009-12-01 22:01:59 »

I don't understand. Any security threat I need to worry about?

Uh... somebody (you?) had make this massive script, that fed the JAR through at least 10 compressors/obfuscators. What I was trying to say was that I'd like to make this an online service for everybody to use. But it has to run on Linux, and without a desktop.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Markus_Persson

JGO Wizard


Medals: 14
Projects: 19


Mojang Specifications


« Reply #9 - Posted 2009-12-01 22:11:37 »

That would be really good!

Play Minecraft!
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline moogie

JGO Knight


Medals: 12
Projects: 6
Exp: 10 years


Java games rock!


« Reply #10 - Posted 2009-12-01 22:19:11 »

If somebody has the tools that run on linux, I can make my server happy by doing some bruteforce zip-attacks on your cute JARs.

Last year we such an app, but had some reliance on *.exe file, and it was macroing some GUI, and my server is blind. (who did it? Appel??)

So if anybody it willing to give it a gentle kick towards linux, I'm going to write some HTTP service that does all the naughty stuff for you.

I believe that is the 4kjo tool i had made... it was not ever really robust Tongue but more of an in house tool that i shared... warts and all Tongue

http://www.java-gaming.org/topics/4kjo-4k-java-optimiser-version-3-released/18085/view.html
Offline moogie

JGO Knight


Medals: 12
Projects: 6
Exp: 10 years


Java games rock!


« Reply #11 - Posted 2009-12-01 22:22:20 »

Pack200 brings my current project down to 1.47 kb from 2.05 kb. That's very very good. Cheesy

hmm now i am a little confused... i dont see a way to submit a pack200 version (pack.gz) of the jar on the java4k website.

Does this mean that the java4k website is not configured to deliver pack200 compressed jars?

Or do we have to embed a pack200 version attempt to decompress and add the embedded pack200 jar to the class loader of the applet at run time?
Offline pjt33
« Reply #12 - Posted 2009-12-01 22:34:36 »

I'm personally annoyed I have to use optimizers to be able to contend - the fun part for me is putting in static final etc. etc. in order to get the bytes down. And then doing a smart thing with a loop, etc.
Last year I used Proguard and then decompiled it and saved the last 60-odd bytes by hand-tweaking the bytecode, reassembling with Jasmin. You can still be hardcore if you want to be!
Offline appel

JGO Wizard


Medals: 50
Projects: 4


I always win!


« Reply #13 - Posted 2009-12-01 22:35:38 »

hmm now i am a little confused... i dont see a way to submit a pack200 version (pack.gz) of the jar on the java4k website.

Does this mean that the java4k website is not configured to deliver pack200 compressed jars?

Or do we have to embed a pack200 version attempt to decompress and add the embedded pack200 jar to the class loader of the applet at run time?

You can upload both .jar and .gz files as "applet jars".

Check out the 4K competition @ www.java4k.com
Check out GAMADU (my own site) @ http://gamadu.com/
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 742
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #14 - Posted 2009-12-01 22:49:29 »

Interresting results -- assuming they both sqeezed their JARs, as I couldn't find a tool to do it better...


Falcon4k (Alan_W)

   4,093 bytes (jar)
   3,485 bytes (pack.gz)

   4,125 bytes (re-zipped with winrar, for baseline)
   3,485 bytes (pack.gz => after winrar did the re-zip)

   3,467 bytes (pack.gz => 7z on .pack)



Flux4k (Michael Bliem)

   4,093 bytes (jar)
   3,331 bytes (pack.gz)

   3,990 bytes (jar => re-zipped with winrar, for baseline)
   3,288 bytes (pack.gz => after winrar did the re-zip)

   3,260 bytes (pack.gz => 7z on .pack)





Interesting to see that pack200 did a better job after a 'plain zipper' like WinRAR.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Abuse

JGO Coder


Medals: 11


falling into the abyss of reality


« Reply #15 - Posted 2009-12-01 22:55:19 »

Interresting results -- assuming they both sqeezed their JARs, as I couldn't find a tool to do it better...


Falcon4k (Alan_W)
   4,093 bytes (jar)
   4,125 bytes (re-zipped with winrar, for baseline)
   3,485 bytes (pack.gz)

Flux4k (Michael Bliem)
   4,093 bytes (jar)
   3,990 bytes (re-zipped with winrar, for baseline)
   3,331 bytes (pack.gz)


Use kzip or 7zip for the comparison against jar.
Still impressive savings though; anyone fancy writing an optimised pack200 compressor?

Nvm, redundant - you can skip the gzip step and use whatever gzip compatible super-efficient compressor you like.

I suppose the "pack"ing step might be further optimisable with a home-grown implementation.

Make Elite IV:Dangerous happen! Pledge your backing at KICKSTARTER here! https://dl.dropbox.com/u/54785909/EliteIVsmaller.png
Offline appel

JGO Wizard


Medals: 50
Projects: 4


I always win!


« Reply #16 - Posted 2009-12-01 22:57:11 »

Perhaps not everyone are aware that they can use pack200.

Check out the 4K competition @ www.java4k.com
Check out GAMADU (my own site) @ http://gamadu.com/
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 742
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #17 - Posted 2009-12-01 22:59:55 »

I updated the stats, BTW...

Still impressive savings though; anyone fancy writing an optimised pack200 compressor?

pack.gz is simply Java's default GZ deflater over a .pack file.

I think that's the only point where we can optimize by doing some bruteforce GZ-ing, as pack200 has to unzip your carefully crafted jar anyway.


Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Abuse

JGO Coder


Medals: 11


falling into the abyss of reality


« Reply #18 - Posted 2009-12-01 23:11:03 »

I'm curious; does pack200 invalidate the 'inject binary data into class files' space saving paradigm?

Make Elite IV:Dangerous happen! Pledge your backing at KICKSTARTER here! https://dl.dropbox.com/u/54785909/EliteIVsmaller.png
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 742
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #19 - Posted 2009-12-01 23:14:03 »

I updated the table again... it's getting smaller without any effort.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline moogie

JGO Knight


Medals: 12
Projects: 6
Exp: 10 years


Java games rock!


« Reply #20 - Posted 2009-12-01 23:55:04 »

I'm curious; does pack200 invalidate the 'inject binary data into class files' space saving paradigm?

Yes, i beleive that is the case as and class with an unknown attribuite (i.e. the binary data) is not compressed by pack 200.

however it is likely that having the data as a seperate file in a jar that is pack200 compressed will still be much smaller than an embedded data in a normal jar
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 742
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #21 - Posted 2009-12-02 00:22:23 »

Compile 'n Shrink - HTTP Service

http://www.indiespot.net/app/java-four-kay

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline pjt33
« Reply #22 - Posted 2009-12-02 01:14:56 »

I used kzip last year, but it doesn't support the gzip file format. Therefore here is a utility I've knocked up which converts zip files into gzip files. It makes various assumptions (only the first file in the zip is processed; the zip is assumed to give CRC, compressed size, and uncompressed size before the data block rather than after it). I've tested it with kzip and ensured that the result is correct.

Error handling is about what you'd expect from a throwaway tool (i.e. minimal).

Not yet tested on a .pack file, but I'm about to go to bed so I'll put it up for people to play with.

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  
42  
43  
44  
45  
46  
47  
48  
49  
50  
51  
52  
53  
54  
55  
56  
57  
58  
59  
60  
61  
62  
63  
64  
65  
66  
67  
68  
69  
70  
71  
72  
73  
74  
75  
76  
77  
78  
79  
80  
81  
82  
83  
84  
85  
86  
87  
88  
89  
90  
91  
92  
import java.io.*;

// Based on http://www.gzip.org/zlib/rfc-gzip.html
// and http://www.pkware.com/documents/casestudies/APPNOTE.TXT
public class Zip2Gzip
{
   // Usage: java Zip2Gzip [file.zip [file.gzip]]
  public static void main(String[] args) throws IOException
   {
      InputStream in = args.length < 1 ? System.in : new FileInputStream(args[0]);
      in = new BufferedInputStream(in);
      OutputStream out = args.length < 2 ? System.out : new FileOutputStream(args[1]);
      out = new BufferedOutputStream(out);

      // Start by writing a gzip header.
     // Magic.
     out.write(0x1f);
      out.write(0x8b);
      // Compression method: inflate.
     out.write(0x08);
      // Flags: we include no optional extras.
     out.write(0x00);
      // Timestamp: unavailable.
     out.write(0x00);
      out.write(0x00);
      out.write(0x00);
      out.write(0x00);
      // Extra flags: none.
     out.write(0x00);
      // OS: unknown.
     out.write(0xff);

      // The next block of output is the compressed data. We need to process
     // the zip file header to find it and know how long it is.
     // local file header signature     4 bytes  (0x04034b50)
     // version needed to extract       2 bytes
     // general purpose bit flag        2 bytes
     // compression method              2 bytes
     // last mod file time              2 bytes
     // last mod file date              2 bytes
     //   Total so far:                14 bytes
     for (int i = 0; i < 14; i++) in.read();
      // crc-32                          4 bytes
     int crc1 = in.read();
      int crc2 = in.read();
      int crc3 = in.read();
      int crc4 = in.read();
      // compressed size                 4 bytes
     int cmpSz =  (in.read() & 0xff) +
                  ((in.read() & 0xff) << 8) +
                  ((in.read() & 0xff) << 16) +
                  ((in.read() & 0xff) << 24);
      // uncompressed size               4 bytes
     int ucmpSz1 = in.read();
      int ucmpSz2 = in.read();
      int ucmpSz3 = in.read();
      int ucmpSz4 = in.read();
      // file name length                2 bytes
     int nameLen = (in.read() & 0xff) + ((in.read() & 0xff) << 8);
      // extra field length              2 bytes
     int xfLen = (in.read() & 0xff) + ((in.read() & 0xff) << 8);
      // file name (variable size)
     for (int i = 0; i < nameLen; i++) in.read();
      // extra field (variable size)
     for (int i = 0; i < xfLen; i++) in.read();

      // Data follows, so we can copy it to the output.
     byte[] buf = new byte[4096];
      while (cmpSz > 0)
      {
         int desired = cmpSz > buf.length ? buf.length : cmpSz;
         int len = in.read(buf, 0, desired);
         if (len == 0) throw new EOFException();
         out.write(buf, 0, len);
         cmpSz -= len;
      }

      // The output still needs the CRC32 and the uncompressed size.
     out.write(crc1);
      out.write(crc2);
      out.write(crc3);
      out.write(crc4);
      out.write(ucmpSz1);
      out.write(ucmpSz2);
      out.write(ucmpSz3);
      out.write(ucmpSz4);

      // Done. Be tidy.
     out.close();
      in.close();
   }
}
Offline appel

JGO Wizard


Medals: 50
Projects: 4


I always win!


« Reply #23 - Posted 2009-12-02 01:56:39 »

Compile 'n Shrink - HTTP Service

http://www.indiespot.net/app/java-four-kay

Cooooool! Cheesy

Check out the 4K competition @ www.java4k.com
Check out GAMADU (my own site) @ http://gamadu.com/
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 742
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #24 - Posted 2009-12-02 02:24:23 »

I used kzip last year, but it doesn't support the gzip file format. Therefore here is a utility ...

I'm using 7z's gz to compress the .pack file. Tomorrow I will try to see how it compares to the gz part of kzip.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline moogie

JGO Knight


Medals: 12
Projects: 6
Exp: 10 years


Java games rock!


« Reply #25 - Posted 2009-12-02 03:04:55 »

I've knocked up which converts zip files into gzip files.

Handy Smiley thanks!
Offline SimonH
« Reply #26 - Posted 2009-12-02 03:41:21 »

Compile 'n Shrink - HTTP Service

http://www.indiespot.net/app/java-four-kay
Uber-cool! Nice one!

People make games and games make people
Offline Alan_W

JGO Knight


Medals: 8
Projects: 3


Java tames rock!


« Reply #27 - Posted 2009-12-02 06:33:45 »

Interresting results -- assuming they both sqeezed their JARs, as I couldn't find a tool to do it better...


Falcon4k (Alan_W)

   4,093 bytes (jar)
...
   3,467 bytes (pack.gz => 7z on .pack)

...


Interesting to see that pack200 did a better job after a 'plain zipper' like WinRAR.

Good grief.  Absolutely amazing. Thanks Riven. I knew that pack200 zipped up all the files in an archive together, thus running the compression algorithm over the lot, rather than each one individually and had therefore only expected a noticeable benefit for archives containing multiple files.  I guess pack200 must also reduce some of the class overhead that comes with a standard java class.  I had intended to target jre1.4 again this year as the only thing I wanted out of jre1.5 was the nanosecond timer, but am now sitting down with a dazed expression.  Maybe leave Falcon4k as jre1.4 as it's pretty much finished, but on the other hand, I could fit a lot of level data in that 500 extra bytes.  Shocked

Time flies like a bird. Fruit flies like a banana.
Offline zammbi

JGO Coder


Medals: 4



« Reply #28 - Posted 2009-12-02 07:50:50 »

I say just ignore Java 1.4, or you could always make 2 versions but just submit the Java 1.5 version.

Current project - Rename and Sort
Offline pjt33
« Reply #29 - Posted 2009-12-02 09:21:30 »

I guess pack200 must also reduce some of the class overhead that comes with a standard java class.
IIRC at a minimum it exploits redundancy in the constant pool structure.
Pages: [1] 2 3 ... 6
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

pw (11 views)
2014-07-24 01:59:36

Riven (10 views)
2014-07-23 21:16:32

Riven (11 views)
2014-07-23 21:07:15

Riven (12 views)
2014-07-23 20:56:16

ctomni231 (42 views)
2014-07-18 06:55:21

Zero Volt (38 views)
2014-07-17 23:47:54

danieldean (32 views)
2014-07-17 23:41:23

MustardPeter (34 views)
2014-07-16 23:30:00

Cero (50 views)
2014-07-16 00:42:17

Riven (50 views)
2014-07-14 18:02:53
HotSpot Options
by dleskov
2014-07-08 03:59:08

Java and Game Development Tutorials
by SwordsMiner
2014-06-14 00:58:24

Java and Game Development Tutorials
by SwordsMiner
2014-06-14 00:47:22

How do I start Java Game Development?
by ra4king
2014-05-17 11:13:37

HotSpot Options
by Roquen
2014-05-15 09:59:54

HotSpot Options
by Roquen
2014-05-06 15:03:10

Escape Analysis
by Roquen
2014-04-29 22:16:43

Experimental Toys
by Roquen
2014-04-28 13:24:22
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!