Java-Gaming.org    
Featured games (79)
games approved by the League of Dukes
Games in Showcase (476)
Games in Android Showcase (106)
games submitted by our members
Games in WIP (532)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1] 2
  ignore  |  Print  
  7zip  (Read 9226 times)
0 Members and 1 Guest are viewing this topic.
Offline kaffiene
« Posted 2006-01-29 21:06:58 »

Hi Guys

When I use 7z to pack my jar file:

"c:\Program Files\7-zip\7z" a -mx=9 -tzip 4kg.jar build/classes/g.class META-INF/MANIFEST.MF

It packs okay, but when I double click in windows I get an error: "Invalid or corrupt jarfile....<name of jarfile>"

Any ideas?
Offline Alan_W

JGO Knight


Medals: 8
Projects: 3


Java tames rock!


« Reply #1 - Posted 2006-01-29 21:14:22 »

Probably you need to put the class at the top level, so as to match the manifest.  Of course I don't know what's in your manifest, but assuming that you are going for minimal size, then it'll just be the class name with no path.

I have
1  
7za a -mx=9 -mfb=255 -mpass=4 -tzip frag4k.jar G.class META-INF/MANIFEST.MF


However it is also worth trying kzip with various table sizes (/bxxx) as this can come out smaller, when you're really feeling the pinch.

1  
kzip /y /b128 katakanainvaders4k.jar G.class d


Alan

Time flies like a bird. Fruit flies like a banana.
Offline DonaldEKnuth

Junior Member





« Reply #2 - Posted 2006-01-29 21:15:41 »

I know I had this problem for a while myself, unfortunately I can't remember what it was. This post still has a purpose, try out KZip instead. It gave me waaaay better compression than 7zip (which I use for my daily needs, but KZip is to much more efficient to be ignored in this contest). Refrain from the GUI version though, it crashes on my computer while the command line version works fine.

My 4k 2006 Entry: Sokoban4k
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline kaffiene
« Reply #3 - Posted 2006-01-29 21:46:36 »

I think now this might be something other than 7z - I tried kzip and I get the same result (though, with smaller files!  Grin)

My mainfest has this in it:

Main-Class: g

(with two CR's after the first line)

my jar file has:

g.class
meta-inf\mainfest.mf

Is all of that correct?

Thanks for any help, by the way  Smiley
Offline Alan_W

JGO Knight


Medals: 8
Projects: 3


Java tames rock!


« Reply #4 - Posted 2006-01-29 22:12:19 »

/EDIT - You mispelt manifest in the above post - I assume that's just a typo...

Strictly speaking the manifest should also have a version, although it often works without.  Not sure whether omitting the version is safe on all platforms and java versions.

Manifest-Version: 1.0
Main-Class: G

The corrupt jar message normally appears when the manifest cannot be found, or it's wrong, so I'd double check it by comparing it with one generated by the IDE.

The only other help I can give is the complete source of one of my obfustication batch files (for windows).  Note that I generate a jar in the IDE, obfusticate it, then unpack, then repack it.

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
REM Obfusticate & compress jar

REM Use Proguard 3.2 if speed matters since 3.4 optimisation adversely effects application speed

REM Obfusticate the last compiled jar
C:\j2sdk1.4.2_10\bin\java -jar C:\User\Java\proguard3.2\lib\proguard.jar @config_1.pro
REM C:\j2sdk1.4.2_10\bin\java -jar C:\User\Java\proguard3.4\lib\proguard.jar @config_2.pro

REM Unpack the obfusticated jar
C:\j2sdk1.4.2_10\bin\jar xf C:\User\Java\Kanainvaders4k\dist\kanainvaders4k.jar
del kanainvaders4k.jar

REM Construct hiraganainvaders4k

REM crush the png
del d
copy ..\h t
pngcrush -bit_depth 1 -brute -l 9 -z 2 -reduce -rem alla t d
del t

REM Re-jar with maximum compression
kzip /y /r /b128 hiraganainvaders4k_m.jar G.class d META-INF/MANIFEST.MF
kzip /y /b128 hiraganainvaders4k.jar G.class d

REM Construct katakanainvaders4k

REM crush the png
del d
copy ..\k t
pngcrush -bit_depth 1 -brute -l 9 -z 2 -reduce -rem alla t d
del t

REM Re-jar with maximum compression
kzip /y /r /b128 katakanainvaders4k_m.jar G.class d META-INF/MANIFEST.MF
kzip /y /b128 katakanainvaders4k.jar G.class d

pause


and another

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
REM Obfusticate & compress jar

REM Use Proguard 3.2 since 3.4 optimisation adversely effects application speed

C:\j2sdk1.4.2_10\bin\java -jar C:\User\Java\proguard3.2\lib\proguard.jar @config_1.pro
REM C:\j2sdk1.4.2_10\bin\java -jar C:\User\Java\proguard3.4\lib\proguard.jar @config_2.pro

C:\j2sdk1.4.2_10\bin\jar xf C:\User\Java\Frag4k\dist\frag4k.jar
del frag4k.jar
7za a -mx=9 -mfb=255 -mpass=4 -tzip frag4k.jar G.class META-INF/MANIFEST.MF
pause

Time flies like a bird. Fruit flies like a banana.
Offline jbanes

JGO Coder


Projects: 1


"Java Games? Incredible! Mr. Incredible, that is!"


« Reply #5 - Posted 2006-01-30 01:49:28 »

This post still has a purpose, try out KZip instead. It gave me waaaay better compression than 7zip (which I use for my daily needs, but KZip is to much more efficient to be ignored in this contest).

In my experience, if you're getting better compression with KZip, you're doing something wrong. Same with Proguard over JoGa. While Proguard & JoGa together can produce the best results, Proguard can't do nearly as much as JoGa if you're writing tight code.

Quote
Strictly speaking the manifest should also have a version, although it often works without.  Not sure whether omitting the version is safe on all platforms and java versions.

I've never had it fail on any platform without the version.

The problem appears to be in the original post:

Quote
"c:\Program Files\7-zip\7z" a -mx=9 -tzip 4kg.jar build/classes/g.class META-INF/MANIFEST.MF

The zip program is going to store the "build/classes" directory in the zip file. Thus the JAR Executor won't be able to find the class. Make sure that all your files are in the same directory, so that the structure looks like this:

1  
2  
3  
4  
> ls -r
META-INF
META-INF/MANIFEST.MF
G.class


If you're still having problems, there's two things I'd suggest:

1. Pack the manifest BEFORE the class file. The JVM attempts to find the manifest first in the ZIP stream, so it can cause problems if it's out of order.

2. Watch the case of your files. Throughout your posts, you've changed the case of "G.class" and "META-INF/MANIFEST.MF". The class file MUST match the same case used inside the source file, and the manifest MUST be in upper case.

If you're still having troubles, post the output of "jar -tvf 4kg.jar"

Java Game Console Project
Last Journal Entry: 12/17/04
Offline kaffiene
« Reply #6 - Posted 2006-01-30 07:33:58 »

If you're still having troubles, post the output of "jar -tvf 4kg.jar"

Good catch:

C:\development\java\work\4kg>jar -tvf 4kg.jar
Error in JAR file format. zip-style comment?

I opened the jar file with Winzip and indeed, ProGuard had added a zip comment to the jar.  I've looked through the proguard docs but can't find out how to suppress this behaviour.  Any ideas?  I guess I should try JoGa and see if that fixes the problem.
Offline kaffiene
« Reply #7 - Posted 2006-01-30 09:07:14 »

oh, I got it now - the jars made with kzip don't work with java 1.5 - using the java 1.4 sdk they run okay
Offline DonaldEKnuth

Junior Member





« Reply #8 - Posted 2006-01-30 09:17:34 »

Quote
In my experience, if you're getting better compression with KZip, you're doing something wrong.
Sorry but I'm not giving much for that "experience". If you look at the documentation for KZip it explains why it gets higher compression rates than most other compression tools (it takes the time to search for the optimal directory size etc). It's still the same compression algorithm, KZip just takes the necessary time to find something closer to the optimal parameter settings.

How well data compresses is of course related to the structure of the data, but that "well written" compiled code should compress better or equally good with 7zip as with KZip is just plain wrong and in fact quite funny. I guess you know nothing about how the LZW compression works.

Could you explain how you got this "experience" in a less psuedo-scientific way?
Quote
oh, I got it now - the jars made with kzip don't work with java 1.5 - using the java 1.4 sdk they run okay
I'm using Java 1.5 SDK even though I compile for 1.4 and have used KZip successfully.

My 4k 2006 Entry: Sokoban4k
Offline oNyx

JGO Coder


Medals: 1


pixels! :x


« Reply #9 - Posted 2006-01-30 09:24:10 »

oh, I got it now - the jars made with kzip don't work with java 1.5 - using the java 1.4 sdk they run okay

kzipped jars work just fine with 1.5.

弾幕 ☆ @mahonnaiseblog
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline jbanes

JGO Coder


Projects: 1


"Java Games? Incredible! Mr. Incredible, that is!"


« Reply #10 - Posted 2006-01-30 14:23:04 »

Quote
In my experience, if you're getting better compression with KZip, you're doing something wrong.
Sorry but I'm not giving much for that "experience". If you look at the documentation for KZip it explains why it gets higher compression rates than most other compression tools (it takes the time to search for the optimal directory size etc). It's still the same compression algorithm, KZip just takes the necessary time to find something closer to the optimal parameter settings.

7Zip uses the same techniques, only automatically. I've tested both quite a bit, and what I've found is that  7Zip, used properly, always comes out ahead.

Quote
How well data compresses is of course related to the structure of the data, but that "well written" compiled code should compress better or equally good with 7zip as with KZip is just plain wrong and in fact quite funny. I guess you know nothing about how the LZW compression works.

No, I'm quite familiar with it. What I've noticed is that there's almost always more room for more features in games made by KZip users. There's no easy way to quantify this, so use what works best for you. But I'm willing to bet that there's a lot more you can pack in there. Smiley

Java Game Console Project
Last Journal Entry: 12/17/04
Offline DonaldEKnuth

Junior Member





« Reply #11 - Posted 2006-01-30 15:58:18 »

Well as I said, you make no sense and I suspect that your experience have mislead you to draw this conclusion. If you could give any argument at all in a technical manner why this would be so then I would at least consider it.

I am also convinced that there is more optimizations to be made in all of our entries, but to use some kind of "magic formula" where you use 7Zip and KZip to judge this just doesn't ring true to me. They both use the same compression algorithm, only different parameters. And what would be "good use of 7Zip"?

My 4k 2006 Entry: Sokoban4k
Offline jbanes

JGO Coder


Projects: 1


"Java Games? Incredible! Mr. Incredible, that is!"


« Reply #12 - Posted 2006-01-30 17:59:47 »

Well as I said, you make no sense and I suspect that your experience have mislead you to draw this conclusion. If you could give any argument at all in a technical manner why this would be so then I would at least consider it.

As I said, it's difficult to quantify on a general basis. I can look at your code and say, "If you replace that counter with a simple mathematical calculation, you'll save a huge amount pre-compression. Since you use math bytecodes regularly, this will also translate into excellent post-compression results." I *know* that you'd be reducing the number of tokens in the LZW stream, possibly avoiding a dictionary reset. But your problem over there may be unrelated to the problem some other person is having. The very nature of the beast makes it difficult to generalize about these things.

What I can say as a generalization is that the tighter your code gets, the less it responds to general compression. This is normal as you're reducing some of the redundancy yourself. You can adjust to this by deliberately using less efficient (but more redundant!) code pre-compression, but in the end you can't carry more information than the Shannon limit will allow. (Considerably less, actually, as LZW dosn't  get anywhere near the theoretical limits of data compression.)

Now KZip attempts to reduce the size of the file by allowing you to mess with the compression parameters to find the best balance for your file. 7Zip does much the same thing, except it does it algorithmically rather than brute-forced. My testing has shown that the 7Zip algorithms get much better results than KZip on files that are extremely difficult to compress. (e.g. Tightly written code.) KZip's brute force, OTOH, manages to get a little extra out of files that are easily compressable.

That being said, based on your comment about KZip's documentation (Documentation? What documentation?), I have a feeling that you're really referring to BJWFlate rather than KZip. In which case the results may be very different.

Java Game Console Project
Last Journal Entry: 12/17/04
Offline DonaldEKnuth

Junior Member





« Reply #13 - Posted 2006-01-30 19:28:54 »

I took the deliberty to download your game Xero which in jar file size is 4089 bytes, I then decompressed it and compressed it with KZip instead. It now occupies 3951 bytes (I have both files here if you want them) so I guess your own game is an example of this non-tight coding eh? Smiley

Do you know of any game (now that your own does not) which actually compresses better with 7Zip than KZip? It would be interesting to know, so please give me an URL to its jar-file. Also, when I said KZip documentation i meant the information available on its web page.


My 4k 2006 Entry: Sokoban4k
Offline jbanes

JGO Coder


Projects: 1


"Java Games? Incredible! Mr. Incredible, that is!"


« Reply #14 - Posted 2006-01-30 19:36:47 »

I'll pull up the tests I've done when I get a chance. That' s VERY interesting if you're getting better results. I think I may know why, though. From Ken's webpage:

12,800 bytes 11/23/2005

After several years of zero changes,  it looks like he's finally updated it. My copy is probably out of date now. I wonder if the J4K contest has anything to do with it? Smiley

Java Game Console Project
Last Journal Entry: 12/17/04
Offline DonaldEKnuth

Junior Member





« Reply #15 - Posted 2006-01-30 19:47:56 »

Quote
I'll pull up the tests I've done when I get a chance.
That would be very interesting!
Quote
That' s VERY interesting if you're getting better results.
Glad to have raised your interest, see for yourself if you don't believe me.
Quote
. I wonder if the J4K contest has anything to do with it? Smiley
Perhaps, he has gotten some well deserved attention.

I guess this ruins your whole concept which you have argued for above?  Grin
And if it doesn't, find me one game which compresses better with 7zip. (I have both the latest version of 7zip and Kzip so it's easy to verify.)

My 4k 2006 Entry: Sokoban4k
Offline jbanes

JGO Coder


Projects: 1


"Java Games? Incredible! Mr. Incredible, that is!"


« Reply #16 - Posted 2006-01-30 19:54:11 »

Ruins it? No. Modifies it? Yes. If there's new datum in the equation, then things become interesting again. Maybe I'll get to see if I can fit Just One More Feature(TM) into Xero. Wink

The Proguard vs. JoGa part still holds, though. I've taken to using both of them, but the results from Proguard optimizations were pretty disappointing in comparison to JoGa. But if you use both (thanks for that Kapta!) you get the best of both worlds. Smiley

Edit: It was Kapta that ran the tests, not Markus. My memory is failing me at my old age. Smiley

Java Game Console Project
Last Journal Entry: 12/17/04
Offline DonaldEKnuth

Junior Member





« Reply #17 - Posted 2006-01-30 20:08:08 »

Quote
Maybe I'll get to see if I can fit Just One More Feature(TM) into Xero. Wink
He he, that would be higher vertical speed right? Wink

Quote
The Proguard vs. JoGa part still holds, though. I've taken to using both of them, but the results from Proguard optimizations were pretty disappointing in comparison to JoGa. But if you use both (thanks for that Markus!) you get the best of both worlds. Smiley
Now that I have your attention, does it matter in which order one applies ProGuard and JoGa? In theory it could, as one might make an optimization impossible for the other but I'm inexperienced with both of them. For my game I think I applied ProGaurd first and then JoGa.

My 4k 2006 Entry: Sokoban4k
Offline jbanes

JGO Coder


Projects: 1


"Java Games? Incredible! Mr. Incredible, that is!"


« Reply #18 - Posted 2006-01-30 20:36:33 »

Quote
Maybe I'll get to see if I can fit Just One More Feature(TM) into Xero. Wink
He he, that would be higher vertical speed right? Wink

I already gave you that. If you haven't tried it already Run it again if you haven't tried it lately. I tweaked the speed to where it feels quite natural now. Smiley

Quote
Now that I have your attention, does it matter in which order one applies ProGuard and JoGa? In theory it could, as one might make an optimization impossible for the other but I'm inexperienced with both of them. For my game I think I applied ProGaurd first and then JoGa.

I believe that Kapta's findings were that Proguard then JoGa produces the best results. I've seen pretty much the same thing. It would appear that ProGuard and JoGa do different optimizations. Proguard's are more general and thus don't seem to interfere with JoGa's. But JoGa's optimizations seem to screw with some of the stuff that Proguard would try to modify, thus making it perform worse than it could.

Java Game Console Project
Last Journal Entry: 12/17/04
Offline Anon666

Junior Member




aka Abuse/AbU5e/TehJumpingJawa


« Reply #19 - Posted 2006-01-30 20:39:30 »

Have a go @ Proguard -> Joga/Jax -> ProGuard.

Though there are numerous peephole optimisations that none of the above obfuscators perform.

For those, you can either roll your own, or have a go with jode
Offline DonaldEKnuth

Junior Member





« Reply #20 - Posted 2006-01-31 15:57:13 »

Regarding peephole optimizations, are they worth it? Have you managed to fit something valuable in the size gap you got from making a peephole optimization?

My 4k 2006 Entry: Sokoban4k
Offline jbanes

JGO Coder


Projects: 1


"Java Games? Incredible! Mr. Incredible, that is!"


« Reply #21 - Posted 2006-01-31 16:05:42 »

Regarding peephole optimizations, are they worth it? Have you managed to fit something valuable in the size gap you got from making a peephole optimization?

I know that Abuse (Robin Chadwick) used peephole optimizations to get another 10-20 bytes during the second year of the contest. Unfortunately, it's such a painful process that he kind of burned himself out doing builds. IMHO, your best bet is to automate any optimizations you find by writing code in using the Bytecode Engineering Library. That way you can keep your builds automated rather than spending time on manually tweaking your code.

I personally don't think it's worth it, but then again I don't have quite as much time to burn as some of you young'uns. Wink

Java Game Console Project
Last Journal Entry: 12/17/04
Offline DonaldEKnuth

Junior Member





« Reply #22 - Posted 2006-01-31 16:09:22 »

Well I don't intend to do anything such this year either, it would have been nice to create another entry (some kind of action game this time) but it will have to wait. There got to be people here younger than me though, after all my first computer was a Commodore 64. Smiley (Some kind of fuzzy age scale.)

My 4k 2006 Entry: Sokoban4k
Offline jbanes

JGO Coder


Projects: 1


"Java Games? Incredible! Mr. Incredible, that is!"


« Reply #23 - Posted 2006-01-31 16:15:28 »

There got to be people here younger than me though, after all my first computer was a Commodore 64. Smiley (Some kind of fuzzy age scale.)

Yeah, but for all I know you were only 2 years old at the time!  Cheesy

Java Game Console Project
Last Journal Entry: 12/17/04
Offline DonaldEKnuth

Junior Member





« Reply #24 - Posted 2006-01-31 16:32:09 »

Quote
Yeah, but for all I know you were only 2 years old at the time!  Cheesy
That's why it's a fuzzy scale.

My 4k 2006 Entry: Sokoban4k
Offline woogley
« Reply #25 - Posted 2006-01-31 17:33:32 »

There got to be people here younger than me though

for what it's worth, I'm only 18 (turning 19 on february 16th)
Offline swpalmer

JGO Coder




Where's the Kaboom?


« Reply #26 - Posted 2006-01-31 17:36:31 »

I find it odd the Kzip using a larger search window to find the best match would be relevant when we are talking about source files that are tiny to begin with (i.e.the entire source data would likely fit into a 8k or 16k search window).  There must be something more to it, or I just don't have a good understanding of the deflate algorithm?

Anyway.. good info in this thread.  I may be able to get a few more bytes.

Offline Anon666

Junior Member




aka Abuse/AbU5e/TehJumpingJawa


« Reply #27 - Posted 2006-01-31 19:19:47 »

Regarding peephole optimizations, are they worth it? Have you managed to fit something valuable in the size gap you got from making a peephole optimization?

I know that Abuse (Robin Chadwick) used peephole optimizations to get another 10-20 bytes during the second year of the contest. Unfortunately, it's such a painful process that he kind of burned himself out doing builds. IMHO, your best bet is to automate any optimizations you find by writing code in using the Bytecode Engineering Library. That way you can keep your builds automated rather than spending time on manually tweaking your code.

I personally don't think it's worth it, but then again I don't have quite as much time to burn as some of you young'uns. Wink

Chaddock  Angry  Roll Eyes

Quote
Regarding peephole optimizations, are they worth it? Have you managed to fit something valuable in the size gap you got from making a peephole optimization?

I only started worrying about it when I had features already in, that I couldn't get to fit any other way.

Its definitely a worthwhile investment though, as JBanes pointed out, they can be quite easily automated, so you silently reap the benefits in any future games.
(It just so happens bytecode optimisation for size is a very real concern in J2ME development - my job)

One thing I realy want to automate, is replacing methods & method invocations, into sub-routines.
It will allow more modular program structure, without the overhead of methods.
(unfortunately, such an optimisation would be of no benefit to J2ME - as there is no JSR instruction Angry)
Offline jbanes

JGO Coder


Projects: 1


"Java Games? Incredible! Mr. Incredible, that is!"


« Reply #28 - Posted 2006-01-31 19:39:11 »

Chaddock  Angry  Roll Eyes

Err... what he said. Like I said, I'm not as young as some of you other young'uns. Now my memory is starting to go!  Grin

Quote
(It just so happens bytecode optimisation for size is a very real concern in J2ME development - my job)

Abuse, is that you?

Java Game Console Project
Last Journal Entry: 12/17/04
Offline Anon666

Junior Member




aka Abuse/AbU5e/TehJumpingJawa


« Reply #29 - Posted 2006-01-31 20:20:14 »

Chaddock  Angry  Roll Eyes

Err... what he said. Like I said, I'm not as young as some of you other young'uns. Now my memory is starting to go!  Grin

Quote
(It just so happens bytecode optimisation for size is a very real concern in J2ME development - my job)

Abuse, is that you?

yeah, dropped my other account ages back - 110 posts to be exact ^_^
Pages: [1] 2
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

pw (14 views)
2014-07-24 01:59:36

Riven (14 views)
2014-07-23 21:16:32

Riven (13 views)
2014-07-23 21:07:15

Riven (15 views)
2014-07-23 20:56:16

ctomni231 (43 views)
2014-07-18 06:55:21

Zero Volt (40 views)
2014-07-17 23:47:54

danieldean (32 views)
2014-07-17 23:41:23

MustardPeter (36 views)
2014-07-16 23:30:00

Cero (50 views)
2014-07-16 00:42:17

Riven (50 views)
2014-07-14 18:02:53
HotSpot Options
by dleskov
2014-07-08 03:59:08

Java and Game Development Tutorials
by SwordsMiner
2014-06-14 00:58:24

Java and Game Development Tutorials
by SwordsMiner
2014-06-14 00:47:22

How do I start Java Game Development?
by ra4king
2014-05-17 11:13:37

HotSpot Options
by Roquen
2014-05-15 09:59:54

HotSpot Options
by Roquen
2014-05-06 15:03:10

Escape Analysis
by Roquen
2014-04-29 22:16:43

Experimental Toys
by Roquen
2014-04-28 13:24:22
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!