Java-Gaming.org    
Featured games (81)
games approved by the League of Dukes
Games in Showcase (492)
Games in Android Showcase (112)
games submitted by our members
Games in WIP (556)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  clear up on how data is stored/read  (Read 2410 times)
0 Members and 1 Guest are viewing this topic.
Offline Kommi

Junior Member




All opinions will be lined up and shot!


« Posted 2005-04-25 14:14:31 »

I was wondering if you guys can clear up a few questions for me concering how data is stored.

1. If I have a txt file that is 78,390 bytes long. If I wish to view and manipulate the raw binary data, I have to convert byte data into binary data and then write the binary data to a txt file. This binary data txt file is 8 times larger than the orignal txt file.  Is there a way to have access and/or manipulate the original txt file's binary data without having to write it to a file?

2. The previous is a compression related question, so I was also wondering if anyone can explain the principle upon which winZip or winRar work? How do these programs compress the data to a smaller size? What level of access do they have to the file they are compressing?

Edit: I have found an article on the Huffman Compression algorithm and have answered the second question myself. I am still interested in how I can access the data on that level with Java though. I have found a java applet that shows the process, but it doesnt actually write the compressed file.



Any answers to these questions are greatly appreciated.

Kommi
Offline rdcarvallo

Senior Member


Projects: 5
Exp: 15 years


2D Java games forever!


« Reply #1 - Posted 2005-04-25 17:03:54 »

Hi,
  How are you opening the file?? With a reader??

  This can help you..
1  
2  
3  
4  
5  
6  
7  
String filename = "myFile.txt";
DataInputStream dis = new DataInputStream(new FileInputStream(filename));
byte b = dis.readByte();
//Do what  yo need with the byte
if((b & 0x01) != 0){
  // It's an odd number!!
}


  Rafael.-
Offline Jeff

JGO Coder




Got any cats?


« Reply #2 - Posted 2005-04-26 03:52:55 »

Quote
I was wondering if you guys can clear up a few questions for me concering how data is stored.

1. If I have a txt file that is 78,390 bytes long. If I wish to view and manipulate the raw binary data, I have to convert byte data into binary data and then write the binary data to a txt file. This binary data txt file is 8 times larger than the orignal txt file.


Im not sure I udnerstand this. Do you mena to say youa re taking each bit sequentially  in the file and writing it out as a 0 or 1 valued byte?

If so why? Smiley

Quote

2. The previous is a compression related question, so I was also wondering if anyone can explain the principle upon which winZip or winRar work? How do these programs compress the data to a smaller size? What level of access do they have to the file they are compressing?

Edit: I have found an article on the Huffman Compression algorithm and have answered the second question myself. I am still interested in how I can access the data on that level with Java though. I have found a java applet that shows the process, but it doesnt actually write the compressed file.


Well, huffman is just oen of many compression schemes and a failry simple on.

But if yopu want to work with Zip style comrepssed fiels in Java the ehavy lifting is all done for you by Zip* classes Smiley  (ZipFile, ZipInputStream and ZipOutputStream).  A ncie Java feature.


Any answers to these questions are greatly appreciated.[/quote]

Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Kommi

Junior Member




All opinions will be lined up and shot!


« Reply #3 - Posted 2005-04-26 12:10:09 »

Well my point of confusion is that if I take a txt file and run a Huffman algorith on it, How would I write the compressed binary back to a smaller file? Just convert it back into text (it would be all jumpbled of course)?

Kommi
Offline rdcarvallo

Senior Member


Projects: 5
Exp: 15 years


2D Java games forever!


« Reply #4 - Posted 2005-04-26 14:59:51 »

Quote
Well my point of confusion is that if I take a txt file and run a Huffman algorith on it, How would I write the compressed binary back to a smaller file? Just convert it back into text (it would be all jumpbled of course)?


How is your binary data stored in the program? In a byte array, char array??

If each element in your array represents 1 bit you should pack it into bytes before writing to the output file.

 This code packs the the data from an array of ints (with values 0 or 1) into bytes and writes them to a file.

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
  private void save(String filename){
    DataOutputStream dos = new DataOutputStream(new FileOutputStream(filename));
    byte b;
    int pos = 0;
    int[] compressedData = new int[1000];
    //here you fill the array
   int total = 58; //this is the total bits
   while (pos < total) {
      b = encode(compressedData,pos);
      dos.writeByte(b);
      pos += 8;
    }
    dos.close();
  }
  private byte encode(int[] data, int pos) {
    int more = 0;
    byte b = 0;
    while (more < 8) {
      if (pos + more < data.length)
        b |= data[pos + more];
      if(more < 7)
        b <<= 1;
      more++;
    }
    return b;
  }  

Offline Jeff

JGO Coder




Got any cats?


« Reply #5 - Posted 2005-04-26 15:41:22 »

Quote
Well my point of confusion is that if I take a txt file and run a Huffman algorith on it, How would I write the compressed binary back to a smaller file? Just convert it back into text (it would be all jumpbled of course)?



As the man says, it is totally dependant  on how the data is stored and what exactly you are "huffman-ing."

If your goal is to reduce individual bytes down to sub-byte strinsg of bits (less then 8 bits per symbol) then you need to byte pack it or you will get no benefit.

That seems fairly obvious, so myabe Im missing something?


Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Offline Jeff

JGO Coder




Got any cats?


« Reply #6 - Posted 2005-04-26 15:42:34 »

Is there a particualr reason why you need a aprticualr encoding though?

This seems a lot of work to avoid using the Zip classes, which will gvie you much better comrepssion  then a simple huffman on the individual bytes.




Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Offline rdcarvallo

Senior Member


Projects: 5
Exp: 15 years


2D Java games forever!


« Reply #7 - Posted 2005-04-26 18:52:15 »

Quote
This seems a lot of work to avoid using the Zip classes, which will gvie you much better comrepssion  then a simple huffman on the individual bytes.


In jGuru I found the next code:
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
import java.io.*;
import java.util.zip.*;

public class zipper {
    public static final void main(String [] args){
      try {
          ZipOutputStream outStream = new ZipOutputStream (new
            FileOutputStream("out.zip"));
             
          writeEntry(outStream, "f1.txt", "this is some text");
          writeEntry(outStream, "f2.txt", "more text");
          writeEntry(outStream, "f3.txt", "text, text, and yet more text");
         
          outStream.close();
      }
      catch(Exception e){
          e.printStackTrace();
      }
    }

    static void writeEntry(ZipOutputStream stream,
                     String          filename,
                     String          text){
      try {
          ZipEntry e1 = new ZipEntry(filename);
          e1.setMethod(ZipEntry.DEFLATED);
          stream.putNextEntry(e1);
          stream.write(text.getBytes());
          stream.closeEntry();
      }
      catch(IOException e){
          e.printStackTrace();
      }
    }
}


This way the zip stream compresses the data for you, and you can put more than one file in the zip.

  Rafael.-



Offline Kommi

Junior Member




All opinions will be lined up and shot!


« Reply #8 - Posted 2005-04-27 12:17:07 »

No reason, I am just intersted in how compression works and trying to write an example of it. Thank you for your answers, they were very helpful.

Question: what kind of operator is |= ?

Kommi
Offline Orangy Tang

JGO Kernel


Medals: 56
Projects: 11


Monkey for a head


« Reply #9 - Posted 2005-04-27 12:38:04 »

a |= b

is equivilent to:

a = a | b

ie. a bitwise OR.

[ TriangularPixels.com - Play Growth Spurt, Rescue Squad and Snowman Village ] [ Rebirth - game resource library ]
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline rdcarvallo

Senior Member


Projects: 5
Exp: 15 years


2D Java games forever!


« Reply #10 - Posted 2005-04-27 12:52:41 »

The != operator is a short version for a = a|b.
The | is the binary or operator.
Offline Kommi

Junior Member




All opinions will be lined up and shot!


« Reply #11 - Posted 2005-04-27 13:29:19 »

hehe ok I dont know why I didnt see that in the first place Smiley

Kommi
Offline Jeff

JGO Coder




Got any cats?


« Reply #12 - Posted 2005-04-27 18:27:51 »

Oh!  Well if you are trying to learn about compression, might I suggest
you google "LZW"?

Lempel/Ziff is well documented. Unfortunately its also patented so you can't use it in a product without a lciense (Unisys's terms actually used to be pretty reasonable. They'd give youa  fre elicense for free stuff and charge you for comemrical use.)

Its quite aways beyond a simple huffman, though not so far that i think yould have any problems grapsing it.  I actually wrote an LZW decomrepssor in Java years ago in order to read PDF files, which use it.




Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Pages: [1]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

Nickropheliac (16 views)
2014-08-31 22:59:12

TehJavaDev (23 views)
2014-08-28 18:26:30

CopyableCougar4 (33 views)
2014-08-22 19:31:30

atombrot (42 views)
2014-08-19 09:29:53

Tekkerue (41 views)
2014-08-16 06:45:27

Tekkerue (35 views)
2014-08-16 06:22:17

Tekkerue (26 views)
2014-08-16 06:20:21

Tekkerue (37 views)
2014-08-16 06:12:11

Rayexar (73 views)
2014-08-11 02:49:23

BurntPizza (49 views)
2014-08-09 21:09:32
List of Learning Resources
by Longor1996
2014-08-16 10:40:00

List of Learning Resources
by SilverTiger
2014-08-05 19:33:27

Resources for WIP games
by CogWheelz
2014-08-01 16:20:17

Resources for WIP games
by CogWheelz
2014-08-01 16:19:50

List of Learning Resources
by SilverTiger
2014-07-31 16:29:50

List of Learning Resources
by SilverTiger
2014-07-31 16:26:06

List of Learning Resources
by SilverTiger
2014-07-31 11:54:12

HotSpot Options
by dleskov
2014-07-08 01:59:08
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!