Java-Gaming.org    
Featured games (79)
games approved by the League of Dukes
Games in Showcase (477)
Games in Android Showcase (108)
games submitted by our members
Games in WIP (536)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: 1 2 3 [4] 5 6
  ignore  |  Print  
  Java 4k Resources Thread  (Read 40475 times)
0 Members and 1 Guest are viewing this topic.
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 749
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #90 - Posted 2009-12-14 12:20:59 »

I might be wrong, and I'm probably looking at the wrong code, but I think I'm using session hijacking and a zeroday security hole.
I had a look for JavaScript but I couldn't find any, so I assumed you weren't actually hijacking the session. Hmm.

Packet sniffing doesn't show an actual session hijack, but the referrer should allow you to see who's requesting the image.

You think it's believable that I am guessing at how I did it? Grin I thought it was obvious I was messing with you Wink

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline steveyO
« Reply #91 - Posted 2009-12-14 13:03:11 »

My Guess:  The link to Rivens avatar is a servlet or mapped to a servlet (not an image)..  The servlet picks up the user id (this is available from the http request in the referer, 'Live HTTP headers' plugin for Firefox is your friend here to see whats going on). Riven has a db/lookup table of usernames/ids.. And creates the animated gif in the Servlet and returns that.  Very clever!

https://play.google.com/store/apps/details?id=com.bullsquared.alggame Annoying Little Gits (Android)
www.bullsquared.com   Play java (applet) games! www.chessclockpro.com Free Online Chess Clock
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 749
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #92 - Posted 2009-12-14 13:11:28 »

My Guess:  The link to Rivens avatar is a servlet or mapped to a servlet (not an image)..  The servlet picks up the user id (this is available from the http request in the referer, 'Live HTTP headers' plugin for Firefox is your friend here to see whats going on)

Riven has a db/lookup table of usernames/ids.. And creates the animated gif in the Servlet and returns that.  Very clever!
I seriously cannot 'pick up a user id', that would be a major security hole in the browser, so I do not have IDs in my database - I only have your IP, the "Referer: --" header probably... and some other headers like the user-agent.

I also do not have an ip/username mapping in my database.

this is my stdout for a request:
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
requestReceived: /files/logo4.png
addr: xxx.xxx.xxx.xxx
username-score-1: steveyO (1.3125)
username-score-2: zammbi  (0.3125)
username-score-3: thijs   (0.2500)
username-score-4: princec (0.2500)
username-score-5: pjt33   (0.0625)
username-score-6: Riven   (0.0625)
whois took: 2ms
>> reusing gif for: steveyO

Heuristics.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline moogie

JGO Knight


Medals: 12
Projects: 6
Exp: 10 years


Java games rock!


« Reply #93 - Posted 2009-12-15 09:46:30 »

It is strange, i cannot seem to be able to replicate the results of Riven's fantastic tool.

I have narrowed it down the the proguard or the java source compilation steps as i am able to replicate the pack 200 step exactly and am able to match the kzip step.

It is not a major issue, i am just a little bemused! I will just use Riven's tool to produce the .pack version of my entry and then I will create my own gz version as I seem to be able to produce a more compressed .gz version.

My old 4kjo tool is now obsolete as proguard is able to optimise better than any combination of any other optimiser (as well as proguard in conjunction with any other optimiser) but I can leaver the recompression aspect of the tool and will write a tool that takes a .pack.gz as input and then recompress it to achieve better compression. (well most of the time any way!)


Offline pjt33
« Reply #94 - Posted 2009-12-15 09:59:36 »

I will create my own gz version as I seem to be able to produce a more compressed .gz version.
Which tool do you use for gzip?
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 749
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #95 - Posted 2009-12-15 10:30:25 »

It seems the pack200 is sensitive to the JAR it gets.

I'm creating the JAR with ZipFile and ZipEntry. I bet that qualifies as doMagicStuff() Smiley


Edit:
Hm, you say that your .pack is equal in size - nevermind then.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 749
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #96 - Posted 2009-12-15 11:29:01 »

I did some more bruteforcing, and saved a handful of bytes.

old
A.normal.7z.pack.gz => 3201 bytes
A.progrd.7z.pack.gz => 3036 bytes
A.normal.kz.pack.gz => 3078 bytes
A.progrd.kz.pack.gz => 2934 bytes


new
A.normal.7z.pack.gz => 3201 bytes
A.progrd.7z.pack.gz => 3030 bytes
A.normal.kz.pack.gz => 3076 bytes
A.progrd.kz.pack.gz => 2927 bytes



What amazes me is that 7z is not deterministic.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline pjt33
« Reply #97 - Posted 2009-12-15 11:46:49 »

What amazes me is that 7z is not deterministic.
Nor is kzip. It has an option to randomise it, which it says is useful for running it lots of times and picking the smallest output, but even without that option the size of the file it produces can vary by a couple of bytes.
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 749
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #98 - Posted 2009-12-15 11:58:20 »

Nor is kzip. It has an option to randomise it, which it says is useful for running it lots of times and picking the smallest output, but even without that option the size of the file it produces can vary by a couple of bytes.

I already make use of it, since the last update actually Smiley

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline moogie

JGO Knight


Medals: 12
Projects: 6
Exp: 10 years


Java games rock!


« Reply #99 - Posted 2009-12-15 12:25:16 »

Which tool do you use for gzip?

multiple actually Smiley

I perform a similar brute force approach as Riven using kzip: multiple of 8 block size between 8 and 1024 and then for the "best" block size perform 30 random runs with the best block size. For each kzip run I then use DeflOpt which can sometimes shave an extra byte or two.

I then perform a brute force using BJWFlate given the same input to compress as kzip choosing split sizes as powers of 2 i.e. 4,8,16...8192. Similarly I use DeflOpt for each BJWFlate run.

The smallest resultant zip file out of all the runs is chosen to be passed to pjt33 zip to gzip converter. And thus the final gz is produced.

From previous years experience (and testing this year) some pack files are better compressed using kzip and some are better compressed using BJWflate.

Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 749
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #100 - Posted 2009-12-15 13:26:37 »

multiple actually Smiley

I perform a similar brute force approach as Riven using kzip: multiple of 8 block size between 8 and 1024 and then for the "best" block size perform 30 random runs with the best block size. For each kzip run I then use DeflOpt which can sometimes shave an extra byte or two.

I then perform a brute force using BJWFlate given the same input to compress as kzip choosing split sizes as powers of 2 i.e. 4,8,16...8192. Similarly I use DeflOpt for each BJWFlate run.

The smallest resultant zip file out of all the runs is chosen to be passed to pjt33 zip to gzip converter. And thus the final gz is produced.

From previous years experience (and testing this year) some pack files are better compressed using kzip and some are better compressed using BJWflate.



What's the (highest) difference in byte count between the best kzip and the best [DeflOpt + BJWFlate] ?

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline pjt33
« Reply #101 - Posted 2009-12-15 13:49:13 »

I then perform a brute force using BJWFlate given the same input to compress as kzip choosing split sizes as powers of 2 i.e. 4,8,16...8192. Similarly I use DeflOpt for each BJWFlate run.
According to the docs I've read DeflOpt is already included in BJWflate, so there's no extra benefit. Are you finding that to be incorrect?

I may have to install wine and play with BJWflate.
Offline pjt33
« Reply #102 - Posted 2009-12-15 14:18:13 »

BJWFlate.zip is no longer available on walbeehm.com. I managed to find it elsewhere, but I'd prefer to be cautious. Can anyone confirm the md5sum and cksum outputs?

md5sum: 95d7b11a281293045ae9c110881f8bf1  BJWFlate.exe
cksum: 492803991 39936 BJWFlate.exe
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 749
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #103 - Posted 2009-12-15 15:02:03 »

I was less cautious, so I'm already running it within Wine, with exactly the same md5sum.

N.B.: Also found it elsewhere, so that doesn't help you much

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 749
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #104 - Posted 2009-12-15 15:36:18 »

I did some more bruteforcing, and saved a handful of bytes.

old
A.normal.7z.pack.gz => 3201 bytes
A.progrd.7z.pack.gz => 3036 bytes
A.normal.kz.pack.gz => 3078 bytes
A.progrd.kz.pack.gz => 2934 bytes


new
A.normal.7z.pack.gz => 3201 bytes
A.progrd.7z.pack.gz => 3030 bytes
A.normal.kz.pack.gz => 3076 bytes
A.progrd.kz.pack.gz => 2927 bytes


Added support for BJWFlate:

A.normal.7z.pack.gz => 3201 bytes
A.progrd.7z.pack.gz => 3030 bytes
A.normal.kz.pack.gz => 3078 bytes
A.progrd.kz.pack.gz => 2930 bytes
A.normal.bj.pack.gz => 3107 bytes
A.progrd.bj.pack.gz => 2923 bytes

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 749
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #105 - Posted 2009-12-15 18:43:10 »

I added DeflOpt, and as pjt33 suspected, it didn't have any impact on filesize, so it's disabled for now.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline moogie

JGO Knight


Medals: 12
Projects: 6
Exp: 10 years


Java games rock!


« Reply #106 - Posted 2009-12-15 21:37:24 »

I added DeflOpt, and as pjt33 suspected, it didn't have any impact on filesize, so it's disabled for now.

I guess i mistakenly "saw" a benefit with deflopt and bjwflate... but most likely i was mistaken Tongue

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
a.java => 11103 bytes
a.class => 3940 bytes
a.normal.jar => 2332 bytes
a.progrd.jar => 1585 bytes
a.normal.pack => 2558 bytes
a.progrd.pack => 1636 bytes
a.normal.7z.pack.gz => 1728 bytes
a.progrd.7z.pack.gz => 1245 bytes
a.normal.kz.pack.gz => 1665 bytes
a.progrd.kz.pack.gz => 1226 bytes
a.normal.bj.pack.gz => 1609 bytes
a.progrd.bj.pack.gz => 1198 bytes


I just re-ran your updated online shrinking tool and it now compresses a further 28 bytes, very nice! now all that is left to be the ultimate tool for my purposes (he he) is for it to accept a JAR as input. I will be reading in a data file at run time so needs to be in the Jar. Smiley

excellent work!

I do have another idea on a completely different track of thought about how to shrink the size of the jar/pack.gz. It is a bit more complex and probably error prone but will be a nice challenge. Perhaps i should actually finish my java4k entry first Tongue



Offline Riven
« League of Dukes »

JGO Overlord


Medals: 749
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #107 - Posted 2009-12-15 22:59:58 »

I just re-ran your updated online shrinking tool and it now compresses a further 28 bytes, very nice! now all that is left to be the ultimate tool for my purposes (he he) is for it to accept a JAR as input. I will be reading in a data file at run time so needs to be in the Jar. Smiley

Done!

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline moogie

JGO Knight


Medals: 12
Projects: 6
Exp: 10 years


Java games rock!


« Reply #108 - Posted 2009-12-16 22:01:55 »

fantastic! thanks
Offline Markus_Persson

JGO Wizard


Medals: 14
Projects: 19


Mojang Specifications


« Reply #109 - Posted 2009-12-17 18:55:59 »

This is sooo obvious I can't believe it hasn't been brought up before:

Old:
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
public class M extends Applet implements Runnable {
   private int[] M = new int[32767]; // Shared state

   public void start() {
      new Thread(this).start();
   }

   public void run() {
             // Minecraft 4k game here
       }
}


2018 bytes.


But that adds M to the constant pool when run already is in there!
So what if we name the class and the member "run"?

New code:
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
public class run extends Applet implements Runnable {
   private int[] run = new int[32767]; // Shared state

   public void start() {
      new Thread(this).start();
   }

   public void run() {
             // Minecraft 4k game here
       }
}

2010 bytes.

Play Minecraft!
Offline Abuse

JGO Coder


Medals: 11


falling into the abyss of reality


« Reply #110 - Posted 2009-12-17 18:59:55 »

haha, n1

Though what does proguard do regarding the member?
Does it realize it's a shared name, or does it obfuscate it to a single letter?
Might have to run it with special parameters to prevent it obfuscating the member name?

Make Elite IV:Dangerous happen! Pledge your backing at KICKSTARTER here! https://dl.dropbox.com/u/54785909/EliteIVsmaller.png
Offline Markus_Persson

JGO Wizard


Medals: 14
Projects: 19


Mojang Specifications


« Reply #111 - Posted 2009-12-17 19:45:22 »

Good question, I'm not using proguard yet in my build chain.
You can make proguard keep the game of the member, though, so it shouldn't be a problem.

Play Minecraft!
Offline pjt33
« Reply #112 - Posted 2009-12-17 21:31:47 »

haha, n1

Though what does proguard do regarding the member?
Does it realize it's a shared name, or does it obfuscate it to a single letter?
Might have to run it with special parameters to prevent it obfuscating the member name?
It doesn't rename the class.

Ah. This wouldn't have worked before because of the jar file containing a longer name, but with pack200 and gzip the name of the class is stored only in the pack200's structures, so that's why this is suddenly better.
Offline Markus_Persson

JGO Wizard


Medals: 14
Projects: 19


Mojang Specifications


« Reply #113 - Posted 2009-12-17 21:36:38 »

Are you sure? "jar.class" is only two letters longer than "M.jar", and the savings are more than 2 bytes.

Play Minecraft!
Offline Abuse

JGO Coder


Medals: 11


falling into the abyss of reality


« Reply #114 - Posted 2009-12-17 21:55:55 »

It doesn't rename the class.

I was talking about the member, i.e.
1  
private int[] run = new int[32767]; // Shared state


I believe proguard will (unless you tell it not to) obfuscate this down to 'a', and in so doing reverse Markus's optimisation.

Make Elite IV:Dangerous happen! Pledge your backing at KICKSTARTER here! https://dl.dropbox.com/u/54785909/EliteIVsmaller.png
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 749
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #115 - Posted 2009-12-17 23:20:26 »

http://www.indiespot.net/app/java-four-kay

I added the option to keep all class members.

Maybe you should give it a try, my tiny examples turn out bigger, to I'm interested in the real deal.


normal Proguard config:
1  
2  
3  
4  
5  
6  
7  
-libraryjars /root/jdk1.6.0_15/jre/lib/rt.jar
-keep public class *
-keep class *
-allowaccessmodification
-overloadaggressively
-optimizationpasses 99
-defaultpackage ''


"keep members" Proguard config:
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
-libraryjars /root/jdk1.6.0_15/jre/lib/rt.jar
-keep public class *
-keep class *
-allowaccessmodification
-overloadaggressively
-optimizationpasses 99
-defaultpackage ''

-keepclassmembernames class * {
  private *;
  protected *;
  public *;
  * ;
}

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline pjt33
« Reply #116 - Posted 2009-12-17 23:41:34 »

I was talking about the member, i.e.
1  
private int[] run = new int[32767]; // Shared state


I believe proguard will (unless you tell it not to) obfuscate this down to 'a', and in so doing reverse Markus's optimisation.
Oh. I use
1  
-obfuscationdictionary proguard.dict
where proguard.dict contains
1  
2  
3  
4  
5  
6  
7  
Code
dispose
isActive
start
sleep
run
a

a is there because it's the name of the class, but if you're changing that you'd want to drop it. That should be enough names for most Java4k projects, I think. If you need more just run javap over your class file to work out what methods you're using and update as appropriate.
Offline Eli Delventhal

JGO Kernel


Medals: 42
Projects: 11
Exp: 10 years


Game Engineer


« Reply #117 - Posted 2009-12-17 23:48:08 »

How are you guys using Riven's tool in terms of adding images and other resources into the jar?

See my work:
OTC Software
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 749
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #118 - Posted 2009-12-18 00:04:53 »

How are you guys using Riven's tool in terms of adding images and other resources into the jar?

The diehards rolls their own.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline moogie

JGO Knight


Medals: 12
Projects: 6
Exp: 10 years


Java games rock!


« Reply #119 - Posted 2009-12-18 10:01:04 »

How are you guys using Riven's tool in terms of adding images and other resources into the jar?

My graphics are animations of 8 frames each which consists of a global palette of 6 colours which of which sequential frames are simply XORed with the previous frame. Very cheap to decode and give sufficient  compression (when gzipped) due to the similarities between frames.

I have all my resources in a single separate file which is added to a jar with my class file and then i let Riven's tool at it Smiley
Pages: 1 2 3 [4] 5 6
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

CogWheelz (16 views)
2014-07-30 21:08:39

Riven (22 views)
2014-07-29 18:09:19

Riven (14 views)
2014-07-29 18:08:52

Dwinin (12 views)
2014-07-29 10:59:34

E.R. Fleming (32 views)
2014-07-29 03:07:13

E.R. Fleming (12 views)
2014-07-29 03:06:25

pw (42 views)
2014-07-24 01:59:36

Riven (42 views)
2014-07-23 21:16:32

Riven (30 views)
2014-07-23 21:07:15

Riven (31 views)
2014-07-23 20:56:16
HotSpot Options
by dleskov
2014-07-08 03:59:08

Java and Game Development Tutorials
by SwordsMiner
2014-06-14 00:58:24

Java and Game Development Tutorials
by SwordsMiner
2014-06-14 00:47:22

How do I start Java Game Development?
by ra4king
2014-05-17 11:13:37

HotSpot Options
by Roquen
2014-05-15 09:59:54

HotSpot Options
by Roquen
2014-05-06 15:03:10

Escape Analysis
by Roquen
2014-04-29 22:16:43

Experimental Toys
by Roquen
2014-04-28 13:24:22
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!