Java-Gaming.org    
Featured games (79)
games approved by the League of Dukes
Games in Showcase (477)
Games in Android Showcase (109)
games submitted by our members
Games in WIP (537)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  Adding resources to the class path at run time...  (Read 1895 times)
0 Members and 1 Guest are viewing this topic.
Offline moogie

JGO Knight


Medals: 12
Projects: 6
Exp: 10 years


Java games rock!


« Posted 2005-07-20 08:32:35 »

I have made a little utility which wraps a given JAR file in a Bzip2 compression wrapper giving smaller JAR file sizes while keeping the same functionality.

It can be downloaded here: http://www.geocities.com/budgetanime/bJAR.html

It currently performs as excpected with Executable JARs. It does not work with normal Library JARs. In order to make library JARs work I need to be able to add resources to the class path from a static method. My current code which adds resources to the class path only works from the main thread. i.e. from the public static void main (String args[]) method or any called methods.

The code which adds the compressed JAR to the class path:

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  
42  
43  
44  
45  
46  
47  
48  
49  
50  
51  
52  
53  
54  
55  
56  
57  
58  
59  
60  
61  
62  
63  
64  
65  
66  
67  
68  
69  
70  
71  
72  
73  
74  
75  
76  
77  
78  
79  
80  
81  
82  
83  
84  
85  
86  
87  
88  
89  
90  
91  
92  
93  
94  
95  
import java.io.*;
import java.lang.reflect.*;
import java.net.*;
   


public class bJARLoader
{
    static private bJARLoader instance;
   
    static
    {
        instance=new bJARLoader();
       
//        URL dataURL=instance.getClass().getResource("data.bz2");
//        URL.setURLStreamHandlerFactory(new ExtendedStreamHandlerFactory(dataURL));
//        
//        try
//        {
//            addURL(new URL("bjar:data.jar"));
//        }
//        catch (IOException e)
//        {
//            e.printStackTrace();
//            System.exit(1);
//        }
       
    }
   
    static final int BUFFER = 2048;
    public static void main(String[] args) throws Exception
    {

        URL infoURL=instance.getClass().getResource("bJAR.info");
        URL dataURL=instance.getClass().getResource("data.bz2");
       
        URL.setURLStreamHandlerFactory(new ExtendedStreamHandlerFactory(dataURL));
        addURL(new URL("bjar:data.jar"));

        BufferedReader br = new BufferedReader(new InputStreamReader(infoURL.openStream()));

        String startClassName = br.readLine();
        if (startClassName!=null)
        {
            startClassName=startClassName.substring("Main-Class: ".length());
            runActualJar(startClassName,args);
        }

    }
   
      // The methods addFile and associated final Class[] parameters were
   // gratefully copied from
     // anthony_miguel @ http://forum.java.sun.com/thread.jsp?forum=32&thread=300557&tstart=0&trange=15
     private static final Class[] parameters = new Class[]{URL.class};
   
      public static void addFile(String s) throws IOException {
         File f = new File(s);
         addFile(f);
      }//end method
   
      public static void addFile(File f) throws IOException {
         addURL(f.toURL());
      }//end method
   
      public static void addURL(URL u) throws IOException {

         URLClassLoader sysloader = (URLClassLoader) ClassLoader.getSystemClassLoader();
         Class sysclass = URLClassLoader.class;

         try {
            Method method = sysclass.getDeclaredMethod("addURL",parameters);
            method.setAccessible(true);
            method.invoke(sysloader,new Object[]{ u });
         } catch (Throwable t) {
            t.printStackTrace();
            throw new IOException("Error, could not add URL to system classloader");
         }//end try catch
     }//end method


    // runs the main method of the compressed JAR
   private static void runActualJar(String startClassName, String[] variables) throws Exception
    {

        Class startClass = Class.forName(startClassName);

        Class[] sig = new Class[1];
        sig[0] = String[].class;
        Object[] param = new Object[1];
        param[0] = variables;

        Method bb = startClass.getMethod("main", sig);
        Object cc = bb.invoke(null, param);
    }
}


The actual adding occurs in the AddURL() method.

When i de-comment the block of code in the static block at the top of the listing hoping to add the JAR contents to the class path before instaniation, there is an exception indicating that the addURL() method for the system class loader  does not exisit. This method does seem to exist when called from the main method.

My question is how can i add resources to the main system class loader from a static block of code....
Offline Jeff

JGO Coder




Got any cats?


« Reply #1 - Posted 2005-07-23 06:17:51 »

Nope I dont think so.

My off the cuff reaction is that would open up nasty security holes.

If I can udnerstand what you are trying to do maybe Ican offer a better approach.  At the moment Im not entirely clear.

Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Offline moogie

JGO Knight


Medals: 12
Projects: 6
Exp: 10 years


Java games rock!


« Reply #2 - Posted 2005-07-24 02:11:59 »

Hmm... Thats what i was afraid of...

What i am trying to do is to make a new functionally identical JAR from a given input JAR but instead of using DEFLATE to compress the JAR it uses the Bzip2 Burrows Wheeler Block Matching compression.

It currently works well for an Executable JAR because of the fact there is an instaniation of a class and thus the system class loader has a "addURL" method which will add classes ,or a JAR in this case, to the classpath at runtime. This allows me to decompress the stored original JAR and add it to the system class loader. I then ultimately invoke the main method of the sorted original JAR.

i.e. this is how it currently works:

Making the bzip2 output jar:

1. convert the input jar to a temporary jar which is using the "STORE" compression technique.
2. compress the temp jar using bzip2 streams. (step 1 and 2 are performed in memory using streams)
3. copy the input jar's manifest and place it into the root directory of the output jar
4. copy the classes needed for decompressing the bzip stream into the output jar
5. copy the bzip2 compressed input jar into the output jar

running the bzip2 output jar:

1. read the stored manifest file and add the "class-path" resources to the system class loader if there is any external JARs needed
2. read the stored manifest file and find the main class.
3. decompress and add the bzip2 compressed input jar to the system class loader.
4. invoke the main method of the main class.

The problem exists that it does not work with "library jars" as they have static methods which are called. and i also need to have some sort of method protoypes in the output jar so that programs that use the input jar will be able to use the output jar as they need the same method signature.

Coupled with the fact that it seems that the system class loader does not have the "addURL" method when the method is invoked from static methods Sad
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline blahblahblahh

JGO Coder


Medals: 1


http://t-machine.org


« Reply #3 - Posted 2005-07-24 12:51:24 »

The problem exists that it does not work with "library jars" as they have static methods which are called. and i also need to have some sort of method protoypes in the output jar so that programs that use the input jar will be able to use the output jar as they need the same method signature.

Coupled with the fact that it seems that the system class loader does not have the "addURL" method when the method is invoked from static methods Sad

The problem is that this is java, not C++, and java is specifically and deliberately designed not to let you do things you might want to do, but only to let you do things that the original designers decided you ought to do. In C++, you can transparently override or hook into practically anything - as often said, C++ is not an actual language, it is a meta-language that you can turn into *any* language, because anything and everything can be customized.

Your customization is desirable, and I personally would like it Smiley, but like many many other great things people have come up with, it can't be used until Sun chooses to accept it and does it themselves, unless you really hack java to pieces, at which point most people will lose interest. This is one of the downsides of java, and you just have to live with it Sad.

malloc will be first against the wall when the revolution comes...
Offline moogie

JGO Knight


Medals: 12
Projects: 6
Exp: 10 years


Java games rock!


« Reply #4 - Posted 2005-07-25 02:58:30 »

In a flash of insight I have managed to get around the "system class loader does not have the "addURL" method when the method is invoked from static methods" problem.

It is a little contrived but it does seem to work. I am now launching  a thread from a static block of code which adds the jar to the class path. The main thread pauses until this jar-adding thread is complete.

now all i need to do is figure out how to make protoype classes from the classes in the input JAR. Is the class byte code definiton setup so that i can simply copy only the "header" infomation with out the logic?
Offline oNyx

JGO Coder


Medals: 1


pixels! :x


« Reply #5 - Posted 2005-07-25 03:10:31 »

Zip isnt all that great, right.

For what do you need better compression? Distribution of course. And there are already nice solutions for those common distribution ways. LZMA [+pack200] for installers and pack200/gzip for webstart (1.5+).

For example the lwjgl.jar (331kb) gets down to 61kb with p200/gzip compression.

Some days ago I rewrote my p200/gzip php/htaccess stuff. I'm pretty satisfied with the new version (there were several issues with the previous version)... I think I'll write an article soon.

You can try a silly demonstration here:
http://kaioa.com/src/tmp/BounceDemo2.jnlp
[It merely tells you in which form the archive traveled to you. Eg "I was a .jar" on 1.4.x jws clients and "I was a .jar.pack.gz" on 1.5+]

弾幕 ☆ @mahonnaiseblog
Offline moogie

JGO Knight


Medals: 12
Projects: 6
Exp: 10 years


Java games rock!


« Reply #6 - Posted 2005-07-25 04:06:30 »

Sure, i know i am probably re-inventing the wheel, but it is a challenge and i have not had much to do with class loaders and the like. It is more of an educational exercise for me.

there are some benefits of what i am doing:

1. it is compatible with java 1.2+ (i beleive, i know it is compatible with 1.4)
2. it allows for the option of even better compression algorithms later on. (i.e. it may be possible to use pack200 (LZMA) so that previous versions of java can get the same compression benefit as 1.5)
3. gives me a better understanding of how java works behind the scenes.

Offline Jeff

JGO Coder




Got any cats?


« Reply #7 - Posted 2005-07-25 04:39:25 »

Perfectly reasonable goal in Java.

I suspect you simply attacked it the wrong way.

URLCLassLoader doesnt care what format the data is in, as long as it can be
expressed as a URL.  There are URL handlers in the stndard library for all the usual kinds of
URLs, such as the jar.

The easiest way to extend class laoding types (or data formats in general) is simply to define a new URL type and
URL handler. 



Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Offline moogie

JGO Knight


Medals: 12
Projects: 6
Exp: 10 years


Java games rock!


« Reply #8 - Posted 2005-07-25 04:59:26 »

Quote
The easiest way to extend class laoding types (or data formats in general) is simply to define a new URL type and
URL handler. 

Yep, that is exactly what i have done. i have created a "Bzip2StreamHandler" which implements URLStreamHandler.

I have also set the URLStreamHandlerFactory in the URL to a Stream Handler factory which i have created.


On a side note i have found a java implementation of LZMA compression... i think i will change the program so it has a more plug-in architecture for compressors.

Offline blahblahblahh

JGO Coder


Medals: 1


http://t-machine.org


« Reply #9 - Posted 2005-07-25 11:09:54 »

URLCLassLoader doesnt care what format the data is in, as long as it can be
expressed as a URL.  There are URL handlers in the stndard library for all the usual kinds of
URLs, such as the jar.

And you don't get any problems with the instanceof bug^H^H^Hfeature?

I'm betting you do, in fact, and that it buggers up your code. Especially libraries! If not, well, fantastic, and doh! to me...

(I'd assumed this approach had been rejected already for this reason)

malloc will be first against the wall when the revolution comes...
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Jeff

JGO Coder




Got any cats?


« Reply #10 - Posted 2005-07-26 00:28:14 »

URLCLassLoader doesnt care what format the data is in, as long as it can be
expressed as a URL.  There are URL handlers in the stndard library for all the usual kinds of
URLs, such as the jar.

And you don't get any problems with the instanceof bug^H^H^Hfeature?


I dont knwo what bug youa re erring to but the one time I did this. no i had no trouble with it atall.

(It was for the worlds smallest EJB container, a reserach project, and the loader understood an extended form of the jar! type URL to allow it to load things from sub-jars within a jar file.)

Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Offline moogie

JGO Knight


Medals: 12
Projects: 6
Exp: 10 years


Java games rock!


« Reply #11 - Posted 2005-07-26 04:33:48 »

I believe i have almost at a stage where i can keep the functionality of a "library" JAR... however i have hit a stumbling block...

I seem not to be able to access some private fields from a subclass whoes parent it abstract...

here is some code to demonstrate this: (based on yawmark's code)

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
import java.lang.reflect.*;
import java.net.URLClassLoader;
import java.io.*;
import java.net.*;
 
class PrivateReflection {
 
    public static void main(String[] args) throws Exception {
        Private p = new Private();
        File f=new File("test.jar");
        URL[] urls=new URL[1];
        urls[0]=f.toURL();
        URLClassLoader loader=new URLClassLoader(urls);
        Class c=loader.getClass();

//        Class c=p.getClass();
     
        Field field = c.getField("packages");
        field.setAccessible(true);
        System.out.println(field.get(p));
    }
}
 
class Private {
    private String packages = "Can't get to me!";
     
    private void privateMethod() {
        System.out.println("The password is swordfish");
    }
}


There is no need for a test.jar file to be created.
I know for a fact that the ClassLoader abstract class defines a field called "packages" which is an HashMap.

How do i access this private field?
Offline moogie

JGO Knight


Medals: 12
Projects: 6
Exp: 10 years


Java games rock!


« Reply #12 - Posted 2005-07-27 03:59:27 »

well, i managed to remove classes and jars from the system class loader and replace them with the decompressed versions, however it seems that the cost of having to include the prototype classes with the output JAR negates the savings from using better compression.

Unless there is a way to dynamically create the class prototypes when the jar is first loaded into the system class loader I believe that it is not worth persuing re-compressing "library JARs". However it the re-compression of executable JARs is definitely feasiable, especially with LZMA compression.

Pages: [1]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

CogWheelz (17 views)
2014-08-01 22:53:16

CogWheelz (15 views)
2014-08-01 22:51:43

CopyableCougar4 (20 views)
2014-08-01 19:37:19

CogWheelz (19 views)
2014-07-30 21:08:39

Riven (27 views)
2014-07-29 18:09:19

Riven (16 views)
2014-07-29 18:08:52

Dwinin (14 views)
2014-07-29 10:59:34

E.R. Fleming (42 views)
2014-07-29 03:07:13

E.R. Fleming (13 views)
2014-07-29 03:06:25

pw (44 views)
2014-07-24 01:59:36
Resources for WIP games
by CogWheelz
2014-08-01 18:20:17

Resources for WIP games
by CogWheelz
2014-08-01 18:19:50

List of Learning Resources
by SilverTiger
2014-07-31 18:29:50

List of Learning Resources
by SilverTiger
2014-07-31 18:26:06

List of Learning Resources
by SilverTiger
2014-07-31 13:54:12

HotSpot Options
by dleskov
2014-07-08 03:59:08

Java and Game Development Tutorials
by SwordsMiner
2014-06-14 00:58:24

Java and Game Development Tutorials
by SwordsMiner
2014-06-14 00:47:22
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!