Java-Gaming.org    
Featured games (91)
games approved by the League of Dukes
Games in Showcase (582)
games submitted by our members
Games in WIP (503)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: 1 [2] 3
  ignore  |  Print  
  Why does the Basic Java Hex Editor I made freeze on files around 2MB size and up  (Read 6321 times)
0 Members and 1 Guest are viewing this topic.
Offline erikd

JGO Ninja


Medals: 15
Projects: 4
Exp: 14 years


Maximumisness


« Reply #30 - Posted 2010-10-14 11:09:40 »

And if that doesn't work, run with -Xprof

Offline Riven
« League of Dukes »

JGO Overlord


Medals: 610
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #31 - Posted 2010-10-14 11:16:05 »

Doing a bit of 'functional' programming:
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
final StringBuilder tmp1 = new StringBuilder();
final StringBuilder tmp2 = new StringBuilder();
final AtomicInteger numOfBytesRead = new AtomicInteger(); // mutable Integer (overkill due to single threaded access)

Runnable updateUI = new Runnable()
{
      public void run()
      {
            parent.hexArea.append(tmp1.toString());
            parent.byteArea.append(tmp2.toString());
            parent.progressBar.setValue(numOfBytesRead.get());
            tmp1.setLength(0);
            tmp2.setLength(0);
       }
};

         while((ch = parent.bin.read()) != -1){
                 tmp1.append(HexEditor.getHex(ch));
                 tmp2.append((char)ch);

                                if(numOfBytesRead.incrementAndGet() % 256 == 0)
                                       updateUI.run();
         }

                        if(tmp2.length()>0)
                                       updateUI.run();

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline nsigma
« Reply #32 - Posted 2010-10-14 11:26:36 »

3 UI updates for every byte...

Surely that's 3 UI updates per byte buffer?

One thing that stands out immediately in the code is that there's lots of methods being called within the Runnable that should be in the EDT.  ( filechooser.getSelectedFile(), progressbar.setValue() & setMaximum(), frame.pack(), etc. ) 

Although the textarea.append() method is marked thread-safe, I'd still be tempted to post it in EventQueue.invokeLater().  This could speed things up quite a bit, as you're needlessly waiting for the text area to update (and to get the EDT lock) on every buffer you process.

Praxis LIVE - open-source intermedia toolkit and live interactive visual editor
Digital Prisoners - interactive spaces and projections
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Eli Delventhal

JGO Kernel


Medals: 42
Projects: 12


Game Engineer


« Reply #33 - Posted 2010-10-14 17:59:01 »

Yeah, assign a StringBuffer value after reading the whole file, then append the whole thing to the text area.

See my work:
OTC Software
Offline BoBear2681

JGO Coder


Medals: 18



« Reply #34 - Posted 2010-10-14 19:04:36 »

Yeah, assign a StringBuffer value after reading the whole file, then append the whole thing to the text area.

I'm assuming he wants to "see" the file get loaded into the hex editor chunks at a time though.

The EDT violations are a good point.  Even JTextArea.append() is no longer marked threadsafe in the Java 7 Javadocs, so you should wrap them in SwingUtilities.invokeLater() as well.  nsigma's probably right that you'll probably actually get a speed boost from it.

Another big bottleneck is that you're using two JTextAreas with line wrap enabled.  JTextArea is VERY slow wrapping long lines, and you seem to be creating a single super-long line for both text areas (well, it won't be for byteArea whenever it encounters a '\n', but still).  It'll get exponentially slower the longer the line is.  Multi-MB files are not a good idea with your current setup.

If you instead fix each line as a finite size, you'll get better performance.

You've also stopped replacing non-printable characters with ' ' in byteArea, which can make selecting the text in it slow/buggy.
Offline CyanPrime
« Reply #35 - Posted 2010-10-14 21:58:25 »

Okay, I'm working on wrapping the updateUI to a SwingUtilities.invokeLater() but now when it loads a big file (around 2mb or bigger) it freezes. Here is the source:

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  
42  
43  
44  
45  
46  
47  
48  
49  
50  
51  
52  
53  
54  
55  
56  
57  
58  
59  
60  
61  
62  
63  
64  
65  
66  
67  
68  
69  
70  
71  
72  
73  
74  
75  
76  
77  
78  
79  
80  
81  
82  
83  
84  
85  
86  
87  
88  
89  
90  
91  
92  
93  
94  
95  
96  
97  
98  
99  
100  
101  
102  
103  
104  
105  
106  
107  
108  
109  
110  
111  
112  
113  
114  
115  
116  
117  
118  
119  
120  
121  
122  
123  
124  
125  
126  
127  
128  
129  
130  
131  
132  
133  
134  
135  
136  
137  
138  
139  
140  
141  
142  
143  
144  
145  
146  
147  
148  
149  
150  
151  
152  
153  
154  
155  
156  
157  
158  
159  
160  
161  
162  
163  
164  
165  
166  
167  
168  
169  
170  
171  
172  
173  
174  
175  
176  
177  
178  
179  
180  
181  
182  
183  
184  
185  
186  
import java.awt.*;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;

import javax.swing.*;
import java.util.Vector;
import java.io.BufferedInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.InputStream;

public class HexEditor extends JFrame{
   static final String HEXES = "0123456789ABCDEF";
   
   JScrollPane hexScroll;
   JScrollPane byteScroll;
   JPanel panel;
   JTextArea hexArea;
   JTextArea byteArea;
   JFileChooser chooser;
   FileInputStream fin;
   BufferedInputStream bin;
   JMenuBar menuBar;
   JMenu file;
      JMenuItem load;
     
   JProgressBar progressBar;
   
   StringBuffer hexStr;
   StringBuffer byteStr;
   
   int numOfBytesRead;

   public HexEditor(){
      super("Cypri's java hex editor");
      setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
     
      hexStr = new StringBuffer("");
      byteStr = new StringBuffer("");
     
      progressBar = new JProgressBar();
     
      chooser = new JFileChooser();
     
      load = new JMenuItem("Load");
         load.addActionListener(new ActionListener(){
            public void actionPerformed(ActionEvent event) {
               Thread hflt = new Thread(new HexFileLoader(passMe()));
               hflt.start();        
            }
         });
     
      file = new JMenu("File");
      file.add(load);
     
      menuBar = new JMenuBar();

      menuBar.add(file);
     
      hexArea = new JTextArea();
      byteArea = new JTextArea();
      hexArea.setLineWrap(true);
      hexArea.setWrapStyleWord(true);
      byteArea.setLineWrap(true);
   
      hexScroll = new JScrollPane(ScrollPaneConstants.VERTICAL_SCROLLBAR_ALWAYS, ScrollPaneConstants.HORIZONTAL_SCROLLBAR_ALWAYS);
      byteScroll = new JScrollPane(ScrollPaneConstants.VERTICAL_SCROLLBAR_ALWAYS, ScrollPaneConstants.HORIZONTAL_SCROLLBAR_ALWAYS);
     
      panel = new JPanel();
     
      panel.add(hexScroll);
      panel.add(byteScroll);
      hexScroll.setViewportView(hexArea);
      byteScroll.setViewportView(byteArea);
     
      hexScroll.setPreferredSize(new Dimension(440,480));
      byteScroll.setPreferredSize(new Dimension(200,480));

      getContentPane().setLayout(new BorderLayout());
      getContentPane().add(BorderLayout.NORTH, menuBar);
      getContentPane().add(BorderLayout.CENTER, panel);
      getContentPane().add(BorderLayout.SOUTH, progressBar);
      pack();
      setVisible(true);
   }
   
   public void openFile(){
      chooser.showOpenDialog(null);
   }
   
   public void packMe(){
      pack();
   }
   
   public void revalidateMe(){
      repaint();
   }
   
   public HexEditor passMe(){
      return this;
   }
   
   
     public static String getHex( byte [] raw ) {
       if ( raw == null ) {
         return null;
       }
       
       final StringBuilder hex = new StringBuilder( 2 * raw.length );
       
       for ( final byte b : raw ) {
         hex.append(HEXES.charAt((b & 0xF0) >> 4))
            .append(HEXES.charAt((b & 0x0F)))
            .append(" ");
       }
       
       return hex.toString();
     }

   
   public static void main(String[] args){
       HexEditor app = new HexEditor();
   }
}

class UpdateUI implements Runnable {
    HexEditor parent;
    UpdateUI(HexEditor parent) {
           this.parent = parent;
       }
   
    public void run() {
       parent.hexArea.setText(parent.hexStr.toString());
       parent.byteArea.setText(parent.byteStr.toString());
       parent.progressBar.setValue(parent.numOfBytesRead);
       parent.packMe();
    }
}

class HexFileLoader implements Runnable {
    HexEditor parent;
    HexFileLoader(HexEditor parent) {
        this.parent = parent;
    }

    public void run() {
      try{  
         parent.openFile();
         parent.fin = new FileInputStream(parent.chooser.getSelectedFile());
         parent.bin = new BufferedInputStream(parent.fin);
         
         System.out.println("(int) parent.chooser.getSelectedFile().getTotalSpace(): " + (int) parent.chooser.getSelectedFile().length());
         
         parent.progressBar.setMaximum((int) parent.chooser.getSelectedFile().length());
         parent.progressBar.setValue(0);
         
         System.out.println("Load start.");
     
         parent.hexStr.setLength(0);
         parent.byteStr.setLength(0);
         parent.numOfBytesRead = 0;
         byte[] buf;
         
         if((int) parent.chooser.getSelectedFile().length() > 10000)
            buf = new byte[10000];
         
         else
            buf = new byte[(int) parent.chooser.getSelectedFile().length()];
         
         while(parent.bin.read(buf) != -1){
            parent.hexStr.append(HexEditor.getHex(buf));
            parent.byteStr.append(new String(buf));
            parent.numOfBytesRead += buf.length;
            SwingUtilities.invokeLater(new UpdateUI(parent));
         }

         System.out.println("Out of loop.");
         
         
      }
     
      catch(Exception e){
         e.printStackTrace();
      }
    }
}
Offline cylab

JGO Knight


Medals: 34



« Reply #36 - Posted 2010-10-15 14:53:52 »

I fixed some stuff like:

- taking the return of BufferedReader.read() into account
- changing to invokeAndWait() instead of invokeLater() to not spam the EvenQueue (might have caused the freeze)
- doing append([delta]) instead of setText([fullText]) on the TextAreas to prevent that drastic slow down
- removed redundant storage of the text (in the textarea and in the hexStr/byteStr vars)
- increased the read buffer
- add closing of the file
- moved stuff around a bit

Kind of works, but still waaaay slow

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  
42  
43  
44  
45  
46  
47  
48  
49  
50  
51  
52  
53  
54  
55  
56  
57  
58  
59  
60  
61  
62  
63  
64  
65  
66  
67  
68  
69  
70  
71  
72  
73  
74  
75  
76  
77  
78  
79  
80  
81  
82  
83  
84  
85  
86  
87  
88  
89  
90  
91  
92  
93  
94  
95  
96  
97  
98  
99  
100  
101  
102  
103  
104  
105  
106  
107  
108  
109  
110  
111  
112  
113  
114  
115  
116  
117  
118  
119  
120  
121  
122  
123  
124  
125  
126  
127  
128  
129  
130  
131  
132  
133  
134  
135  
136  
137  
138  
139  
140  
141  
142  
143  
144  
145  
146  
147  
148  
149  
150  
151  
152  
153  
154  
155  
156  
157  
158  
159  
160  
161  
162  
163  
164  
165  
166  
167  
168  
169  
170  
171  
172  
173  
174  
175  
176  
177  
178  
179  
180  
181  
182  
183  
184  
185  
186  
187  
188  
189  
190  
191  
192  
193  
194  
195  
196  
197  
198  
199  
200  
201  
202  
203  
204  
205  
206  
package cyanprime.hexedit;

import java.awt.*;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;

import javax.swing.*;
import java.io.BufferedInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;

public class HexEditor extends JFrame
{
    static final String HEXES = "0123456789ABCDEF";
    JScrollPane hexScroll;
    JScrollPane byteScroll;
    JPanel panel;
    JTextArea hexArea;
    JTextArea byteArea;
    JFileChooser chooser;
    JMenuBar menuBar;
    JMenu file;
    JMenuItem load;
    JProgressBar progressBar;
    int numOfBytesRead;

    public HexEditor()
    {
        super("Cypri's java hex editor");
        setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);

        progressBar = new JProgressBar();
        chooser = new JFileChooser();

        load = new JMenuItem("Load");
        load.addActionListener(new ActionListener()
        {

            public void actionPerformed(ActionEvent event)
            {
                openFile();
            }
        });

        file = new JMenu("File");
        file.add(load);

        menuBar = new JMenuBar();

        menuBar.add(file);

        hexArea = new JTextArea();
        byteArea = new JTextArea();
        hexArea.setLineWrap(true);
        hexArea.setWrapStyleWord(true);
        byteArea.setLineWrap(true);

        hexScroll = new JScrollPane(ScrollPaneConstants.VERTICAL_SCROLLBAR_ALWAYS, ScrollPaneConstants.HORIZONTAL_SCROLLBAR_ALWAYS);
        byteScroll = new JScrollPane(ScrollPaneConstants.VERTICAL_SCROLLBAR_ALWAYS, ScrollPaneConstants.HORIZONTAL_SCROLLBAR_ALWAYS);

        panel = new JPanel();

        panel.add(hexScroll);
        panel.add(byteScroll);
        hexScroll.setViewportView(hexArea);
        byteScroll.setViewportView(byteArea);

        hexScroll.setPreferredSize(new Dimension(440, 480));
        byteScroll.setPreferredSize(new Dimension(200, 480));

        getContentPane().setLayout(new BorderLayout());
        getContentPane().add(BorderLayout.NORTH, menuBar);
        getContentPane().add(BorderLayout.CENTER, panel);
        getContentPane().add(BorderLayout.SOUTH, progressBar);
        pack();
        setVisible(true);
    }

    public void openFile()
    {
        // reset all stuff
       hexArea.setText("");
        byteArea.setText("");

        chooser.showOpenDialog(null);
        final File selectedFile = chooser.getSelectedFile();
        System.out.println("(int) parent.chooser.getSelectedFile().getTotalSpace(): " + (int) selectedFile.length());

        progressBar.setMaximum((int) selectedFile.length());
        progressBar.setValue(0);

        Thread hflt = new Thread(new HexFileLoader(selectedFile));
        hflt.start();
    }

    public void packMe()
    {
        pack();
    }

    public void revalidateMe()
    {
        repaint();
    }

    public static String getHex(byte[] raw, int numBytes)
    {
        if (raw == null)
        {
            return null;
        }

        final StringBuilder hex = new StringBuilder(3 * numBytes);

        for (int c = 0; c < numBytes; c++)
        {
            final byte b = raw[c];
            hex.append(HEXES.charAt((b & 0xF0) >> 4)).append(HEXES.charAt((b & 0x0F))).append(" ");
        }

        return hex.toString();
    }

    public static void main(String[] args)
    {
        HexEditor app = new HexEditor();
    }

    class UpdateUI implements Runnable
    {
        final String hexChunk;
        final String byteChunk;
        final int numOfBytedRead;

        UpdateUI(String hexChunk, String byteChunk, int numOfBytedRead)
        {
            this.hexChunk = hexChunk;
            this.byteChunk = byteChunk;
            this.numOfBytedRead = numOfBytedRead;
        }

        public void run()
        {
            HexEditor.this.hexArea.append(hexChunk);
            HexEditor.this.byteArea.append(byteChunk);
            HexEditor.this.progressBar.setValue(numOfBytedRead);
            HexEditor.this.packMe();
        }
    }

    class HexFileLoader implements Runnable
    {
        final File selectedFile;

        HexFileLoader(File selectedFile)
        {
            this.selectedFile = selectedFile;
        }

        public void run()
        {
            BufferedInputStream bin = null;
            try
            {
                bin = new BufferedInputStream(new FileInputStream(selectedFile));

                System.out.println("Load start.");

                int numOfBytesRead = 0;
                byte[] buf;

                buf = new byte[100000];

                int numBytes = 0;
                while ((numBytes = bin.read(buf)) != -1)
                {
                    final String hexChunk = HexEditor.getHex(buf, numBytes);
                    final String byteChunk = new String(buf, 0, numBytes);
                    numOfBytesRead += numBytes;
                    SwingUtilities.invokeAndWait(new UpdateUI(hexChunk, byteChunk, numOfBytesRead));
                }

                System.out.println("Out of loop.");
            }
            catch (Exception e)
            {
                e.printStackTrace(System.err);
            }
            finally
            {
                if(bin!=null)
                {
                    try
                    {
                        bin.close();
                    }
                    catch(IOException ex)
                    {
                        System.err.println("WARNING: error closing file! "+ex.getMessage());
                    }
                }
            }
        }
    }
}

Mathias - I Know What [you] Did Last Summer!
Offline nsigma
« Reply #37 - Posted 2010-10-15 16:26:43 »

- changing to invokeAndWait() instead of invokeLater() to not spam the EvenQueue (might have caused the freeze)

Most of the changes you've made are good, but this one makes no sense to me.  You're effectively moving the UI update back into the processing thread by waiting for it.  I don't understand why you call it spamming the event queue?  Surely the point of a queue (in most cases) is to hand over your task and get on with what you're doing.  It's not like this cuts down on what the EventQueue is doing either, you're just waiting for it to do it!

Also, I wonder if increasing the buffer size is actually a good idea, as you have to create a String to hold each buffer.  Is lots of small objects or fewer large objects more efficient?

Praxis LIVE - open-source intermedia toolkit and live interactive visual editor
Digital Prisoners - interactive spaces and projections
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 610
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #38 - Posted 2010-10-15 16:34:27 »

Most of the changes you've made are good, but this one makes no sense to me.  You're effectively moving the UI update back into the processing thread by waiting for it.  I don't understand why you call it spamming the event queue?  Surely the point of a queue (in most cases) is to hand over your task and get on with what you're doing.  It's not like this cuts down on what the EventQueue is doing either, you're just waiting for it to do it!

Actually, this is required if you'd be loading massive amounts of data: preventing one thread from producing data faster than other threads can consume it. It will, for example, potentially lead to OOME, in the general case. In this case, however, it's not really required, although it's a good thing to do.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline nsigma
« Reply #39 - Posted 2010-10-15 17:19:12 »

Actually, this is required if you'd be loading massive amounts of data: preventing one thread from producing data faster than other threads can consume it. It will, for example, potentially lead to OOME, in the general case.

Yes, I understand that, though I'd question whether to call that the "general case".  In cases of high memory usage I'd probably be tempted to use some sort of blocking queue to throttle things down when required, but not force a block each time through the loop.  Even if the loading thread got rapidly ahead of the event queue in this case, it shouldn't use much more memory than the whole file will once loaded as the Strings are eligible for GC once they've been appended to the text area.

Amazing random coincidence that your sig changed from impressed to disappointed to go with this post  Tongue

Praxis LIVE - open-source intermedia toolkit and live interactive visual editor
Digital Prisoners - interactive spaces and projections
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 610
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #40 - Posted 2010-10-15 17:24:59 »

Yes, I understand that, though I'd question whether to call that the "general case".  In cases of high memory usage I'd probably be tempted to use some sort of blocking queue to throttle things down when required, but not force a block each time through the loop.  Even if the loading thread got rapidly ahead of the event queue in this case, it shouldn't use much more memory than the whole file will once loaded as the Strings are eligible for GC once they've been appended to the text area.
You are right that 'in the general case' you want some kind of buffer with some size. Syncing with the thread effectively means a buffer with a size of zero: SynchronousQueue. Anyway, way offtopic.

Amazing random coincidence that your sig changed from impressed to disappointed to go with this post  Tongue

My signature as AI.




</offtopic>

CyanPrime: why don't you simply display byte[N..M] at a time, instead of the whole file? You can use (another) scrollbar to seek through the file and load/save your current range.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline BoBear2681

JGO Coder


Medals: 18



« Reply #41 - Posted 2010-10-15 19:53:43 »

You still haven't removed line wrapping.  I removed line wrapping from your app, inserted a newline after each 10,000 'bytes' added to each text area, along with the other suggestions provided, and things were much more responsive.  It took a few seconds to load a 10MB zip file, but the UI was much more responsive the entire time.
Offline CyanPrime
« Reply #42 - Posted 2010-10-15 20:45:41 »

Quote
- changing to invokeAndWait() instead of invokeLater() to not spam the EvenQueue (might have caused the freeze)
Yeah, invokeLater() seems to freeze while invokeAndWait() doesn't freeze.

Quote
You still haven't removed line wrapping.  I removed line wrapping from your app, inserted a newline after each 10,000 'bytes' added to each text area, along with the other suggestions provided, and things were much more responsive.  It took a few seconds to load a 10MB zip file, but the UI was much more responsive the entire time.

You're correct. I am now working on it, and seeing extremely good results. Thank you both for your work.

To anyone who has helped in this thread, do you want to be in the credits?

Offline cylab

JGO Knight


Medals: 34



« Reply #43 - Posted 2010-10-15 22:23:45 »

I don't understand why you call it spamming the event queue?

There is only one queue and with invokeLater() in the loop you are inserting massive amounts of ui updates as fast as you can. So every other component using invokeLater() now has to enqueue it's work behind maybe 20 (slow) ui updates. By doing so you effectively bring swing to a halt...

Also, I wonder if increasing the buffer size is actually a good idea, as you have to create a String to hold each buffer.  Is lots of small objects or fewer large objects more efficient?

It was just to reduce the overall amount of ui updates needed. I don't think that memory handling or allocation speed has any significant impact in this use case.

Mathias - I Know What [you] Did Last Summer!
Offline BoBear2681

JGO Coder


Medals: 18



« Reply #44 - Posted 2010-10-15 22:38:18 »

There is only one queue and with invokeLater() in the loop you are inserting massive amounts of ui updates as fast as you can. So every other component using invokeLater() now has to enqueue it's work behind maybe 20 (slow) ui updates. By doing so you effectively bring swing to a halt...

In this particular case especially though, I don't think we're handing the EDT more than it can handle.  I modified the code to not wrap lines, keep each line to 1024 "bytes," and to update the UI every 36 lines.  On my particular machine I can load a 10MB file in 10 seconds consistently with invokeLater(), 15 seconds if I use invokeAndWait().  YMMV, of course.

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  
42  
43  
44  
45  
46  
47  
48  
49  
50  
51  
52  
53  
54  
55  
56  
57  
58  
59  
60  
61  
62  
63  
64  
65  
66  
67  
68  
69  
70  
71  
72  
73  
74  
75  
76  
77  
78  
79  
80  
81  
82  
83  
84  
85  
86  
87  
88  
89  
90  
91  
92  
93  
94  
95  
96  
97  
98  
99  
100  
101  
102  
103  
104  
105  
106  
107  
108  
109  
110  
111  
112  
113  
114  
115  
116  
117  
118  
119  
120  
121  
122  
123  
124  
125  
126  
127  
128  
129  
130  
131  
132  
133  
134  
135  
136  
137  
138  
139  
140  
141  
142  
143  
144  
145  
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import java.util.Vector;
import java.io.*;

public class HexEditor extends JFrame{

   private static final String HEXES = "0123456789ABCDEF";
   
   JScrollPane hexScroll;
   JScrollPane byteScroll;
   JPanel panel;
   JTextArea hexArea;
   JTextArea byteArea;
   JFileChooser chooser;
   FileInputStream fin;
   BufferedInputStream bin;
   JMenuBar menuBar;
   JMenu file;
   JMenuItem load;
     
   JProgressBar progressBar;

   public HexEditor(){
      super("Cypri's java hex editor");
      setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
     
      progressBar = new JProgressBar();
     
      chooser = new JFileChooser();
     
      load = new JMenuItem("Load");
      load.addActionListener(new ActionListener(){
         public void actionPerformed(ActionEvent event) {
            Thread hflt = new Thread(new HexFileLoader());
            hflt.start();        
         }
      });
     
      file = new JMenu("File");
      file.add(load);
     
      menuBar = new JMenuBar();

      menuBar.add(file);
     
      hexArea = new JTextArea(25, 16*3);
      hexArea.setFont(new Font("Monospaced", Font.PLAIN, 13));
      byteArea = new JTextArea(25, 16);
      byteArea.setFont(new Font("Monospaced", Font.PLAIN, 13));
      hexScroll = new JScrollPane(hexArea);
      byteScroll = new JScrollPane(byteArea);
     
      panel = new JPanel(new BorderLayout());
      panel.add(hexScroll, BorderLayout.LINE_START);
      panel.add(byteScroll, BorderLayout.LINE_END);

      getContentPane().setLayout(new BorderLayout());
      getContentPane().add(BorderLayout.NORTH, menuBar);
      getContentPane().add(BorderLayout.CENTER, panel);
      getContentPane().add(BorderLayout.SOUTH, progressBar);
      pack();
   }

   public static void appendHex(StringBuilder sb, int ch) {
      sb.append(HEXES.charAt((ch & 0xF0) >> 4))
            .append(HEXES.charAt(ch & 0x0F))
            .append(' ');
   }


   public static void main(String[] args){
      SwingUtilities.invokeLater(new Runnable() {
         public void run() {
            HexEditor app = new HexEditor();
            app.setVisible(true);
         }
      });
   }

   class HexFileLoader implements Runnable {
   
       public void run() {
         try{  
            chooser.showOpenDialog(HexEditor.this);
            fin = new FileInputStream(chooser.getSelectedFile());
            bin = new BufferedInputStream(fin);
           
            System.out.println("File length: " + chooser.getSelectedFile().length());
   
            SwingUtilities.invokeLater(new Runnable() {
               public void run() {
                  progressBar.setMaximum((int) chooser.getSelectedFile().length());
                  progressBar.setValue(0);
               }
            });
   
            int ch;
            System.out.println("Load start.");
            long start = System.currentTimeMillis();
            StringBuilder sb = new StringBuilder();
            StringBuilder sb2 = new StringBuilder();
            hexArea.setText("");
            byteArea.setText("");
            int numOfBytesRead = 0;
            int count = 0;        

            while ((ch=bin.read())!=-1) {
               HexEditor.appendHex(sb, ch);
               if (ch<0x20 || ch==0x7f) {
                  ch = ' ';
               }
               sb2.append((char)ch);
               numOfBytesRead++;
               count++;
               if ((count%1024)==0) {
                  sb.append('\n');
                  sb2.append('\n');
               }
               if (count==1024*36) { // 36 lines
                 hexArea.append(sb.toString());
                  byteArea.append(sb2.toString());
                  sb.setLength(0);
                  sb2.setLength(0);
                  count = 0;
                  final int b = numOfBytesRead;
                  SwingUtilities.invokeLater(new Runnable() {
                     public void run() {
                        progressBar.setValue(b);
                     }
                  });
               }
            }

            long time = System.currentTimeMillis() - start;
            System.out.println("Load completed in: " + (time/1000f) + " seconds");
           
         } catch(Exception e) {
            e.printStackTrace();
         }
       }
   }

}


Offline cylab

JGO Knight


Medals: 34



« Reply #45 - Posted 2010-10-15 22:56:49 »

In this particular case especially though, I don't think we're handing the EDT more than it can handle.  I modified the code to not wrap lines, keep each line to 1024 "bytes," and to update the UI every 36 lines.  On my particular machine I can load a 10MB file in 10 seconds consistently with invokeLater(), 15 seconds if I use invokeAndWait().  YMMV, of course.

That's because without wrapping and overall fewer updates (bigger buffer), the load on the queue is drastically reduced.

Mathias - I Know What [you] Did Last Summer!
Offline Eli Delventhal

JGO Kernel


Medals: 42
Projects: 12


Game Engineer


« Reply #46 - Posted 2010-10-15 23:09:38 »

If you want to update the GUI more frequently (once per cell or whatever), you can always sleep the Thread and that should keep the queue from getting too big too fast. Obviously the file gets read less quickly, but you're balancing cosmetics  versus functionality at that point.

See my work:
OTC Software
Offline nsigma
« Reply #47 - Posted 2010-10-16 00:00:21 »

This is in danger of becoming a camel!  Roll Eyes [1]

There is only one queue and with invokeLater() in the loop you are inserting massive amounts of ui updates as fast as you can. So every other component using invokeLater() now has to enqueue it's work behind maybe 20 (slow) ui updates. By doing so you effectively bring swing to a halt...

OK, understand what you're getting at, but it's not like it's posting a Runnable for every character. If my late night maths is right, it's a difference of posting 15,000 characters at a time vs 150,000 characters at a time.  And can the String get so big that adding it in one event actually freezes the EDT more?  Personally, if we were really talking about lots of events on the EDT, I'd find a way to coalesce them, but that would probably be overkill here!

In this particular case especially though, I don't think we're handing the EDT more than it can handle.  I modified the code to not wrap lines, keep each line to 1024 "bytes," and to update the UI every 36 lines.  On my particular machine I can load a 10MB file in 10 seconds consistently with invokeLater(), 15 seconds if I use invokeAndWait().  YMMV, of course.

But your append() methods aren't on the EDT anyway?!


hmm ... Riven's sig is now enlightened with my cod ... there's something fishy going on ...  Grin



[1] http://en.wiktionary.org/wiki/a_camel_is_a_horse_designed_by_committee

Praxis LIVE - open-source intermedia toolkit and live interactive visual editor
Digital Prisoners - interactive spaces and projections
Offline CyanPrime
« Reply #48 - Posted 2010-10-16 00:47:07 »

In this particular case especially though, I don't think we're handing the EDT more than it can handle.  I modified the code to not wrap lines, keep each line to 1024 "bytes," and to update the UI every 36 lines.  On my particular machine I can load a 10MB file in 10 seconds consistently with invokeLater(), 15 seconds if I use invokeAndWait().  YMMV, of course.

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  
42  
43  
44  
45  
46  
47  
48  
49  
50  
51  
52  
53  
54  
55  
56  
57  
58  
59  
60  
61  
62  
63  
64  
65  
66  
67  
68  
69  
70  
71  
72  
73  
74  
75  
76  
77  
78  
79  
80  
81  
82  
83  
84  
85  
86  
87  
88  
89  
90  
91  
92  
93  
94  
95  
96  
97  
98  
99  
100  
101  
102  
103  
104  
105  
106  
107  
108  
109  
110  
111  
112  
113  
114  
115  
116  
117  
118  
119  
120  
121  
122  
123  
124  
125  
126  
127  
128  
129  
130  
131  
132  
133  
134  
135  
136  
137  
138  
139  
140  
141  
142  
143  
144  
145  
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import java.util.Vector;
import java.io.*;

public class HexEditor extends JFrame{

   private static final String HEXES = "0123456789ABCDEF";
   
   JScrollPane hexScroll;
   JScrollPane byteScroll;
   JPanel panel;
   JTextArea hexArea;
   JTextArea byteArea;
   JFileChooser chooser;
   FileInputStream fin;
   BufferedInputStream bin;
   JMenuBar menuBar;
   JMenu file;
   JMenuItem load;
     
   JProgressBar progressBar;

   public HexEditor(){
      super("Cypri's java hex editor");
      setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
     
      progressBar = new JProgressBar();
     
      chooser = new JFileChooser();
     
      load = new JMenuItem("Load");
      load.addActionListener(new ActionListener(){
         public void actionPerformed(ActionEvent event) {
            Thread hflt = new Thread(new HexFileLoader());
            hflt.start();        
         }
      });
     
      file = new JMenu("File");
      file.add(load);
     
      menuBar = new JMenuBar();

      menuBar.add(file);
     
      hexArea = new JTextArea(25, 16*3);
      hexArea.setFont(new Font("Monospaced", Font.PLAIN, 13));
      byteArea = new JTextArea(25, 16);
      byteArea.setFont(new Font("Monospaced", Font.PLAIN, 13));
      hexScroll = new JScrollPane(hexArea);
      byteScroll = new JScrollPane(byteArea);
     
      panel = new JPanel(new BorderLayout());
      panel.add(hexScroll, BorderLayout.LINE_START);
      panel.add(byteScroll, BorderLayout.LINE_END);

      getContentPane().setLayout(new BorderLayout());
      getContentPane().add(BorderLayout.NORTH, menuBar);
      getContentPane().add(BorderLayout.CENTER, panel);
      getContentPane().add(BorderLayout.SOUTH, progressBar);
      pack();
   }

   public static void appendHex(StringBuilder sb, int ch) {
      sb.append(HEXES.charAt((ch & 0xF0) >> 4))
            .append(HEXES.charAt(ch & 0x0F))
            .append(' ');
   }


   public static void main(String[] args){
      SwingUtilities.invokeLater(new Runnable() {
         public void run() {
            HexEditor app = new HexEditor();
            app.setVisible(true);
         }
      });
   }

   class HexFileLoader implements Runnable {
   
       public void run() {
         try{  
            chooser.showOpenDialog(HexEditor.this);
            fin = new FileInputStream(chooser.getSelectedFile());
            bin = new BufferedInputStream(fin);
           
            System.out.println("File length: " + chooser.getSelectedFile().length());
   
            SwingUtilities.invokeLater(new Runnable() {
               public void run() {
                  progressBar.setMaximum((int) chooser.getSelectedFile().length());
                  progressBar.setValue(0);
               }
            });
   
            int ch;
            System.out.println("Load start.");
            long start = System.currentTimeMillis();
            StringBuilder sb = new StringBuilder();
            StringBuilder sb2 = new StringBuilder();
            hexArea.setText("");
            byteArea.setText("");
            int numOfBytesRead = 0;
            int count = 0;        

            while ((ch=bin.read())!=-1) {
               HexEditor.appendHex(sb, ch);
               if (ch<0x20 || ch==0x7f) {
                  ch = ' ';
               }
               sb2.append((char)ch);
               numOfBytesRead++;
               count++;
               if ((count%1024)==0) {
                  sb.append('\n');
                  sb2.append('\n');
               }
               if (count==1024*36) { // 36 lines
                 hexArea.append(sb.toString());
                  byteArea.append(sb2.toString());
                  sb.setLength(0);
                  sb2.setLength(0);
                  count = 0;
                  final int b = numOfBytesRead;
                  SwingUtilities.invokeLater(new Runnable() {
                     public void run() {
                        progressBar.setValue(b);
                     }
                  });
               }
            }

            long time = System.currentTimeMillis() - start;
            System.out.println("Load completed in: " + (time/1000f) + " seconds");
           
         } catch(Exception e) {
            e.printStackTrace();
         }
       }
   }

}



That code doesn't seem to display anything. Looks like you're missing the parent HexEditor in your calls in the HexFileLoader thread.
Offline BoBear2681

JGO Coder


Medals: 18



« Reply #49 - Posted 2010-10-16 01:56:54 »

That code doesn't seem to display anything. Looks like you're missing the parent HexEditor in your calls in the HexFileLoader thread.

Aye, you're right, actually though it's just because I'm batching up the updates to the UI.  It'll miss the last update (only updates on multiples of 1024*36 bytes).  I missed it because I only tested on large files.  Just add this right before the printing out of the runtime:

1  
2  
3  
4  
5  
6  
7  
8  
9  
if (count>0) {
   hexArea.append(sb.toString());
   byteArea.append(sb2.toString());
   SwingUtilities.invokeLater(new Runnable() {
      public void run() {
         progressBar.setValue(progressBar.getMaximum());
      }
   });
}

Offline BoBear2681

JGO Coder


Medals: 18



« Reply #50 - Posted 2010-10-16 02:06:18 »

OK, understand what you're getting at, but it's not like it's posting a Runnable for every character. If my late night maths is right, it's a difference of posting 15,000 characters at a time vs 150,000 characters at a time.  And can the String get so big that adding it in one event actually freezes the EDT more?  Personally, if we were really talking about lots of events on the EDT, I'd find a way to coalesce them, but that would probably be overkill here!

Yeah, I tested with updating the UI after reading different amounts of bytes, and it never ran poorly (i.e. the UI was always responsive, even updating after every byte), so it just became a matter of how much you wanted to throw at it at a time.  I stopped upping the buffer at 1024*36 just because it seemed "fast enough" for a simple test. Cyan can certainly tweak the buffer size to whatever he likes.

Quote
But your append() methods aren't on the EDT anyway?!

Yeah, I cheated a little, because prior to Java 7 JTextArea.append() was listed as threadsafe in its Javadoc.  Unfortunately, as of Java 7, that's no longer the case.  I believe they took that promise away from all the various JTextComponents' methods to modify their text.  So you probably should wrap those as well.
Offline CyanPrime
« Reply #51 - Posted 2010-10-16 02:42:53 »

@BoBear

Can I ask why you used:
1  
if ((count%1024)==0) {


and not something like:

1  
if (count == 1024) {


They'd be the same thing, right?
Offline SimonH
« Reply #52 - Posted 2010-10-16 02:47:53 »

They'd be the same thing, right?
Only mostly;
(2048%1024)==0 true
2048==0 false

People make games and games make people
Offline CyanPrime
« Reply #53 - Posted 2010-10-17 04:27:52 »

Only mostly;
(2048%1024)==0 true
2048==1024 false

Ah, alright. Thanks.

Anyway now I am working on synchronizing the two JTextAreas. I think I have the add working correctly, but for the remove I have to override my JTextArea's Document's remove method. Anyway this is what I got so far (Modified from BoBear2681 's take on the code from: October 15, 2010, 01:38:18 pm)

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  
42  
43  
44  
45  
46  
47  
48  
49  
50  
51  
52  
53  
54  
55  
56  
57  
58  
59  
60  
61  
62  
63  
64  
65  
66  
67  
68  
69  
70  
71  
72  
73  
74  
75  
76  
77  
78  
79  
80  
81  
82  
83  
84  
85  
86  
87  
88  
89  
90  
91  
92  
93  
94  
95  
96  
97  
98  
99  
100  
101  
102  
103  
104  
105  
106  
107  
108  
109  
110  
111  
112  
113  
114  
115  
116  
117  
118  
119  
120  
121  
122  
123  
124  
125  
126  
127  
128  
129  
130  
131  
132  
133  
134  
135  
136  
137  
138  
139  
140  
141  
142  
143  
144  
145  
146  
147  
148  
149  
150  
151  
152  
153  
154  
155  
156  
157  
158  
159  
160  
161  
162  
163  
164  
165  
166  
167  
168  
169  
170  
171  
172  
173  
174  
175  
176  
177  
178  
179  
180  
181  
182  
183  
184  
185  
186  
187  
188  
189  
190  
191  
192  
193  
194  
195  
196  
197  
198  
199  
200  
201  
202  
203  
204  
205  
206  
207  
208  
209  
210  
211  
212  
213  
214  
215  
216  
217  
218  
219  
220  
221  
222  
223  
224  
225  
226  
227  
228  
229  
230  
231  
232  
233  
234  
235  
236  
237  
238  
239  
240  
241  
242  
243  
244  
245  
246  
247  
248  
249  
250  
251  
252  
253  
254  
255  
256  
257  
258  
259  
260  
261  
262  
263  
264  
265  
266  
267  
268  
269  
270  
271  
272  
273  
274  
275  
276  
277  
278  
279  
280  
281  
282  
283  
284  
285  
286  
287  
288  
289  
290  
291  
292  
293  
294  
295  
296  
297  
298  
299  
300  
301  
302  
303  
304  
305  
306  
307  
308  
309  
310  
311  
312  
313  
314  
315  
316  
317  
318  
319  
320  
321  
322  
323  
324  
325  
326  
327  
328  
329  
330  
331  
332  
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import javax.swing.event.DocumentEvent;
import javax.swing.event.DocumentListener;
import javax.swing.event.UndoableEditListener;
import javax.swing.text.AttributeSet;
import javax.swing.text.BadLocationException;
import javax.swing.text.Document;
import javax.swing.text.Element;
import javax.swing.text.Position;
import javax.swing.text.Segment;

import java.util.Vector;
import java.io.*;

public class HexEditor extends JFrame{

   private static final String HEXES = "0123456789ABCDEF";
   
   JScrollPane hexScroll;
   JScrollPane byteScroll;
   JPanel panel;
   JTextArea hexArea;
   JTextArea byteArea;
   JFileChooser chooser;
   FileInputStream fin;
   BufferedInputStream bin;
   JMenuBar menuBar;
   JMenu file;
   JMenuItem load;
     
   JProgressBar progressBar;

   public HexEditor(){
      super("Cypri's java hex editor");
      setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
     
      progressBar = new JProgressBar();
     
      chooser = new JFileChooser();
     
      load = new JMenuItem("Load");
      load.addActionListener(new ActionListener(){
         public void actionPerformed(ActionEvent event) {
            Thread hflt = new Thread(new HexFileLoader());
            hflt.start();        
         }
      });
     
      file = new JMenu("File");
      file.add(load);
     
      menuBar = new JMenuBar();

      menuBar.add(file);
     
      hexArea = new JTextArea(25, 16*3);
      hexArea.setFont(new Font("Monospaced", Font.PLAIN, 13));
      byteArea = new JTextArea(25, 16);
      byteArea.setFont(new Font("Monospaced", Font.PLAIN, 13));
      hexScroll = new JScrollPane(hexArea);
      byteScroll = new JScrollPane(byteArea);
     
     
     
      hexArea.getDocument().addDocumentListener(new DocumentListener() {
         @Override
         public void changedUpdate(DocumentEvent e) {
           
         }

         @Override
         public void insertUpdate(DocumentEvent e) {
            try{
               
               byteArea.insert(new String(hexStringToDisplayableByteArray(hexArea.getText(e.getOffset(), e.getLength()).replace(" ","").replace("\n", ""))), e.getOffset()/2);
               
               System.out.println("Added: " + hexArea.getText(e.getOffset(), e.getLength()).replace(" ","").replace("\n", ""));
            }
           
            catch(Exception ex){
               ex.printStackTrace();
            }
         }

         @Override
         public void removeUpdate(DocumentEvent e) {
            try{
               if(hexArea.getText(e.getOffset(), e.getLength()) != " ")
                  System.out.println("removed: " + hexArea.getText(e.getOffset(), e.getLength()));
            }
           
            catch(Exception ex){
               ex.printStackTrace();
            }
         }
           
        });

     
      panel = new JPanel(new BorderLayout());
      panel.add(hexScroll, BorderLayout.LINE_START);
      panel.add(byteScroll, BorderLayout.LINE_END);

      getContentPane().setLayout(new BorderLayout());
      getContentPane().add(BorderLayout.NORTH, menuBar);
      getContentPane().add(BorderLayout.CENTER, panel);
      getContentPane().add(BorderLayout.SOUTH, progressBar);
      pack();
   }

   public static void appendHex(StringBuilder sb, int ch) {
      sb.append(HEXES.charAt((ch & 0xF0) >> 4))
            .append(HEXES.charAt(ch & 0x0F))
            .append(' ');
   }
   
   public static byte[] hexStringToByteArray(String s) {
       int len = s.length();
       byte[] data = new byte[len / 2];
       for (int i = 0; i < len; i += 2) {
           data[i / 2] = (byte) ((Character.digit(s.charAt(i), 16) << 4)
                                + Character.digit(s.charAt(i+1), 16));
       }
       return data;
   }
   
   public static byte[] hexStringToDisplayableByteArray(String s) {
       int len = s.length();
       byte[] data = new byte[len / 2];
       for (int i = 0; i < len; i += 2) {
           data[i / 2] = (byte) ((Character.digit(s.charAt(i), 16) << 4)
                                + Character.digit(s.charAt(i+1), 16));
           
           if (data[i / 2]<0x20 || data[i / 2]==0x7f) {
              data[i / 2] = ' ';
         }
       }
       return data;
   }


   public static void main(String[] args){
      SwingUtilities.invokeLater(new Runnable() {
         public void run() {
            HexEditor app = new HexEditor();
            app.setVisible(true);
         }
      });
   }

   class HexFileLoader implements Runnable {
   
       public void run() {
         try{  
            chooser.showOpenDialog(HexEditor.this);
            fin = new FileInputStream(chooser.getSelectedFile());
            bin = new BufferedInputStream(fin);
           
            System.out.println("File length: " + chooser.getSelectedFile().length());
   
            SwingUtilities.invokeLater(new Runnable() {
               public void run() {
                  progressBar.setMaximum((int) chooser.getSelectedFile().length());
                  progressBar.setValue(0);
               }
            });
   
            int ch;
            System.out.println("Load start.");
            long start = System.currentTimeMillis();
            StringBuilder sb = new StringBuilder();
            StringBuilder sb2 = new StringBuilder();
            hexArea.setText("");
            byteArea.setText("");
            int numOfBytesRead = 0;
            int count = 0;        

            while ((ch=bin.read())!=-1) {
               HexEditor.appendHex(sb, ch);
               if (ch<0x20 || ch==0x7f) {
                  ch = ' ';
               }
               sb2.append((char)ch);
               numOfBytesRead++;
               count++;
               if ((count%15)==0) {
                  sb.append('\n');
                  sb2.append('\n');
               }
               if (count==15*36) { // 36 lines
                 hexArea.append(sb.toString());
                  byteArea.append(sb2.toString());
                  sb.setLength(0);
                  sb2.setLength(0);
                  count = 0;
                  final int b = numOfBytesRead;
                  SwingUtilities.invokeLater(new Runnable() {
                     public void run() {
                        progressBar.setValue(b);
                     }
                  });
               }
            }

            if (count>0) {
               hexArea.append(sb.toString());
               byteArea.append(sb2.toString());
               SwingUtilities.invokeLater(new Runnable() {
                  public void run() {
                     progressBar.setValue(progressBar.getMaximum());
                  }
               });
            }
           
            long time = System.currentTimeMillis() - start;
            System.out.println("Load completed in: " + (time/1000f) + " seconds");
           
         } catch(Exception e) {
            e.printStackTrace();
         }
       }
   }

   /*class CypriDoc implements Document{

      @Override
      public void addDocumentListener(DocumentListener arg0) {
         // TODO Auto-generated method stub
         
      }

      @Override
      public void addUndoableEditListener(UndoableEditListener arg0) {
         // TODO Auto-generated method stub
         
      }

      @Override
      public Position createPosition(int arg0) throws BadLocationException {
         // TODO Auto-generated method stub
         return null;
      }

      @Override
      public Element getDefaultRootElement() {
         // TODO Auto-generated method stub
         return null;
      }

      @Override
      public Position getEndPosition() {
         // TODO Auto-generated method stub
         return null;
      }

      @Override
      public int getLength() {
         // TODO Auto-generated method stub
         return 0;
      }

      @Override
      public Object getProperty(Object arg0) {
         // TODO Auto-generated method stub
         return null;
      }

      @Override
      public Element[] getRootElements() {
         // TODO Auto-generated method stub
         return null;
      }

      @Override
      public Position getStartPosition() {
         // TODO Auto-generated method stub
         return null;
      }

      @Override
      public String getText(int arg0, int arg1) throws BadLocationException {
         // TODO Auto-generated method stub
         return null;
      }

      @Override
      public void getText(int arg0, int arg1, Segment arg2)
            throws BadLocationException {
         // TODO Auto-generated method stub
         
      }

      @Override
      public void insertString(int arg0, String arg1, AttributeSet arg2)
            throws BadLocationException {
         // TODO Auto-generated method stub
         
      }

      @Override
      public void putProperty(Object arg0, Object arg1) {
         // TODO Auto-generated method stub
         
      }

      @Override
      public void remove(int arg0, int arg1) throws BadLocationException {
         // TODO Auto-generated method stub
         
      }

      @Override
      public void removeDocumentListener(DocumentListener arg0) {
         // TODO Auto-generated method stub
         
      }

      @Override
      public void removeUndoableEditListener(UndoableEditListener arg0) {
         // TODO Auto-generated method stub
         
      }

      @Override
      public void render(Runnable arg0) {
         // TODO Auto-generated method stub
         
      }*/

   
}
Offline i30817

Junior Member





« Reply #54 - Posted 2010-10-17 08:56:01 »

You're about to enter a world of pain if you insist on optimizing swing.text.*
There is a reason netbeans rewrote that whole area for their editor - and i that only append to a text area and don't modify it had to buffer the output of the parsers :
http://code.google.com/p/bookjar-utils/source/browse/BookJar-utils/src/util/swing/BufferedStyledDocumentBuilder.java
(i don't even know how this code interacts with the document model, i only know the result *looks* alright and loads faster)

and *reimplement the gapbuffer*

http://code.google.com/p/bookjar/source/browse/BookJar/src/ui/parsers/SequenceStyledDocument.java

(this implementation is append only, and i doubt some other things work, but removes the need for a huge array - that is a problem when you are appending and need more space, because there is a point there where you have *two* huge arrays - i had exceptions loading war and peace in my reader  Grin - only a 5 mb ascii file. Best is to know the size at the start of course.
Offline i30817

Junior Member





« Reply #55 - Posted 2010-10-17 09:14:59 »

If i were you, i'd try to use the netbeans framework. It would help you with normal infrastructure, and at the same time give you efficient text oriented components. Never tried it though, looks hard to learn, and maybe they didn't optimize large file handling.

Remember the memory problem with char[] in swing.text.* is the gapbuffer (and to a lesser extent, if you're using them, the AttributeSets), i memory profiled, but you can do the same on the netbeans profiler.
I bet those crazy readers / editors that can handle gigabytes files have parsers/readers so integrated that the model you see is mapped to the file itself. When you do change something, or need to read the text they probably know to send the parser read that part of the file and apply the changes all over again to get the final result to display.
Offline BoBear2681

JGO Coder


Medals: 18



« Reply #56 - Posted 2010-10-17 17:39:38 »

But is it really worth it, reinventing so much of the wheel (with limited features) like that, when what he has now works with files of several MB?  It may be worth it in some cases I suppose.

You're right about the text package not being memory efficient though, and a better solution for a hex editor would probably be to just use a JTable with a custom TableModel backed by the byte array for the file.  As of right now, a single byte is represented by 3 chars (2 hex characters and a space), but with the JTable approach each byte would just be the byte itself, and a custom renderer could be used to render if as 6A or FF, for example.
Offline i30817

Junior Member





« Reply #57 - Posted 2010-10-18 00:39:57 »

This is exactly why i think keeping Java binary compatibility forever is very, very stupid.
There is lots of very, very bad code in the java std lib. Other languages update their libs so the warts are removed/reworked, but not java, oh no.

f**k it, it's the one reason i seriously considered using scala. Without projects like swinglabs it would be unbearable. Nothing is going to help you with swing.text.* - it is just not used for anything serious.
Offline zammbi

JGO Coder


Medals: 4



« Reply #58 - Posted 2010-10-18 01:56:02 »

Quote
This is exactly why i think keeping Java binary compatibility forever is very, very stupid.

Well seems JavaFX GUI controls are meant to replace SWING.
http://amyfowlersblog.wordpress.com/2010/09/21/a-heartfelt-ramble-on-swing-javafx/
But this doesn't help atm...

Current project - Rename and Sort
Offline CyanPrime
« Reply #59 - Posted 2010-10-18 02:42:54 »

Okay, I had to override both the PlanDocument class and the JTextArea class, but I think I got it working how I want it. How's this look?

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  
42  
43  
44  
45  
46  
47  
48  
49  
50  
51  
52  
53  
54  
55  
56  
57  
58  
59  
60  
61  
62  
63  
64  
65  
66  
67  
68  
69  
70  
71  
72  
73  
74  
75  
76  
77  
78  
79  
80  
81  
82  
83  
84  
85  
86  
87  
88  
89  
90  
91  
92  
93  
94  
95  
96  
97  
98  
99  
100  
101  
102  
103  
104  
105  
106  
107  
108  
109  
110  
111  
112  
113  
114  
115  
116  
117  
118  
119  
120  
121  
122  
123  
124  
125  
126  
127  
128  
129  
130  
131  
132  
133  
134  
135  
136  
137  
138  
139  
140  
141  
142  
143  
144  
145  
146  
147  
148  
149  
150  
151  
152  
153  
154  
155  
156  
157  
158  
159  
160  
161  
162  
163  
164  
165  
166  
167  
168  
169  
170  
171  
172  
173  
174  
175  
176  
177  
178  
179  
180  
181  
182  
183  
184  
185  
186  
187  
188  
189  
190  
191  
192  
193  
194  
195  
196  
197  
198  
199  
200  
201  
202  
203  
204  
205  
206  
207  
208  
209  
210  
211  
212  
213  
214  
215  
216  
217  
218  
219  
220  
221  
222  
223  
224  
225  
226  
227  
228  
229  
230  
231  
232  
233  
234  
235  
236  
237  
238  
239  
240  
241  
242  
243  
244  
245  
246  
247  
248  
249  
250  
251  
252  
253  
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import javax.swing.event.DocumentEvent;
import javax.swing.event.DocumentListener;
import javax.swing.event.UndoableEditListener;
import javax.swing.text.AttributeSet;
import javax.swing.text.BadLocationException;
import javax.swing.text.Document;
import javax.swing.text.Element;
import javax.swing.text.PlainDocument;
import javax.swing.text.Position;
import javax.swing.text.Segment;

import java.util.Vector;
import java.io.*;

public class HexEditor extends JFrame{

   private static final String HEXES = "0123456789ABCDEF";
   
   JScrollPane hexScroll;
   JScrollPane byteScroll;
   JPanel panel;
   CypriTextArea hexArea;
   JTextArea byteArea;
   JFileChooser chooser;
   FileInputStream fin;
   BufferedInputStream bin;
   JMenuBar menuBar;
   JMenu file;
   JMenuItem load;
     
   JProgressBar progressBar;

   public HexEditor(){
      super("Cypri's java hex editor");
      setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
     
      progressBar = new JProgressBar();
     
      chooser = new JFileChooser();
     
      load = new JMenuItem("Load");
      load.addActionListener(new ActionListener(){
         public void actionPerformed(ActionEvent event) {
            Thread hflt = new Thread(new HexFileLoader());
            hflt.start();        
         }
      });
     
      file = new JMenu("File");
      file.add(load);
     
      menuBar = new JMenuBar();

      menuBar.add(file);
     
      hexArea = new CypriTextArea(25, 16*3);
      hexArea.setFont(new Font("Monospaced", Font.PLAIN, 13));
      byteArea = new JTextArea(25, 16);
      byteArea.setFont(new Font("Monospaced", Font.PLAIN, 13));
      byteArea.setDocument(new CypriDoc());
      hexScroll = new JScrollPane(hexArea);
      byteScroll = new JScrollPane(byteArea);
     
     
     
      hexArea.getDocument().addDocumentListener(new DocumentListener() {
         @Override
         public void changedUpdate(DocumentEvent e) {
           
         }

         @Override
         public void insertUpdate(DocumentEvent e) {
            try{
               
               byteArea.insert(new String(hexStringToDisplayableByteArray(hexArea.getText(e.getOffset(), e.getLength()).replace(" ","").replace("\n", ""))), e.getOffset()/2);
               
               System.out.println("Added: " + hexArea.getText(e.getOffset(), e.getLength()).replace(" ","").replace("\n", ""));
            }
           
            catch(Exception ex){
               ex.printStackTrace();
            }
         }

         @Override
         public void removeUpdate(DocumentEvent e) {
            try{
               System.out.println("removed: " + hexArea.doc.removedText);
            }
           
            catch(Exception ex){
               ex.printStackTrace();
            }
         }
           
        });

     
      panel = new JPanel(new BorderLayout());
      panel.add(hexScroll, BorderLayout.LINE_START);
      panel.add(byteScroll, BorderLayout.LINE_END);

      getContentPane().setLayout(new BorderLayout());
      getContentPane().add(BorderLayout.NORTH, menuBar);
      getContentPane().add(BorderLayout.CENTER, panel);
      getContentPane().add(BorderLayout.SOUTH, progressBar);
      pack();
   }

   public static void appendHex(StringBuilder sb, int ch) {
      sb.append(HEXES.charAt((ch & 0xF0) >> 4))
            .append(HEXES.charAt(ch & 0x0F))
            .append(' ');
   }
   
   public static byte[] hexStringToByteArray(String s) {
       int len = s.length();
       byte[] data = new byte[len / 2];
       for (int i = 0; i < len; i += 2) {
           data[i / 2] = (byte) ((Character.digit(s.charAt(i), 16) << 4)
                                + Character.digit(s.charAt(i+1), 16));
       }
       return data;
   }
   
   public static byte[] hexStringToDisplayableByteArray(String s) {
       int len = s.length();
       byte[] data = new byte[len / 2];
       for (int i = 0; i < len; i += 2) {
           data[i / 2] = (byte) ((Character.digit(s.charAt(i), 16) << 4)
                                + Character.digit(s.charAt(i+1), 16));
           
           if (data[i / 2]<0x20 || data[i / 2]==0x7f) {
              data[i / 2] = ' ';
         }
       }
       return data;
   }


   public static void main(String[] args){
      SwingUtilities.invokeLater(new Runnable() {
         public void run() {
            HexEditor app = new HexEditor();
            app.setVisible(true);
         }
      });
   }

   class HexFileLoader implements Runnable {
   
       public void run() {
         try{  
            chooser.showOpenDialog(HexEditor.this);
            fin = new FileInputStream(chooser.getSelectedFile());
            bin = new BufferedInputStream(fin);
           
            System.out.println("File length: " + chooser.getSelectedFile().length());
   
            SwingUtilities.invokeLater(new Runnable() {
               public void run() {
                  progressBar.setMaximum((int) chooser.getSelectedFile().length());
                  progressBar.setValue(0);
               }
            });
   
            int ch;
            System.out.println("Load start.");
            long start = System.currentTimeMillis();
            StringBuilder sb = new StringBuilder();
            StringBuilder sb2 = new StringBuilder();
            hexArea.setText("");
            byteArea.setText("");
            int numOfBytesRead = 0;
            int count = 0;        

            while ((ch=bin.read())!=-1) {
               HexEditor.appendHex(sb, ch);
               if (ch<0x20 || ch==0x7f) {
                  ch = ' ';
               }
               sb2.append((char)ch);
               numOfBytesRead++;
               count++;
               if ((count%15)==0) {
                  sb.append('\n');
                  sb2.append('\n');
               }
               if (count==15*36) { // 36 lines
                 hexArea.append(sb.toString());
                  byteArea.append(sb2.toString());
                  sb.setLength(0);
                  sb2.setLength(0);
                  count = 0;
                  final int b = numOfBytesRead;
                  SwingUtilities.invokeLater(new Runnable() {
                     public void run() {
                        progressBar.setValue(b);
                     }
                  });
               }
            }

            if (count>0) {
               hexArea.append(sb.toString());
               byteArea.append(sb2.toString());
               SwingUtilities.invokeLater(new Runnable() {
                  public void run() {
                     progressBar.setValue(progressBar.getMaximum());
                  }
               });
            }
           
            long time = System.currentTimeMillis() - start;
            System.out.println("Load completed in: " + (time/1000f) + " seconds");
           
         } catch(Exception e) {
            e.printStackTrace();
         }
       }
   }

   class CypriDoc extends PlainDocument{

      String removedText;
     
      public String getRemoved(){
         return removedText;
      }
     
      @Override
      public void remove(int offs, int len) throws BadLocationException {
         removedText = getText(offs, len);
         super.remove(offs, len);
      }
   }
   
   class CypriTextArea extends JTextArea{
      CypriDoc doc;
     
      public CypriTextArea(int rows, int columns){
         super(25, 16*3);
         doc = new CypriDoc();
         super.setDocument(doc);
      }
     
   }
   
}
Pages: 1 [2] 3
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

xsi3rr4x (72 views)
2014-04-15 18:08:23

BurntPizza (68 views)
2014-04-15 03:46:01

UprightPath (79 views)
2014-04-14 17:39:50

UprightPath (65 views)
2014-04-14 17:35:47

Porlus (80 views)
2014-04-14 15:48:38

tom_mai78101 (104 views)
2014-04-10 04:04:31

BurntPizza (164 views)
2014-04-08 23:06:04

tom_mai78101 (260 views)
2014-04-05 13:34:39

trollwarrior1 (210 views)
2014-04-04 12:06:45

CJLetsGame (220 views)
2014-04-01 02:16:10
List of Learning Resources
by SHC
2014-04-18 03:17:39

List of Learning Resources
by Longarmx
2014-04-08 03:14:44

Good Examples
by matheus23
2014-04-05 13:51:37

Good Examples
by Grunnt
2014-04-03 15:48:46

Good Examples
by Grunnt
2014-04-03 15:48:37

Good Examples
by matheus23
2014-04-01 18:40:51

Good Examples
by matheus23
2014-04-01 18:40:34

Anonymous/Local/Inner class gotchas
by Roquen
2014-03-11 15:22:30
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!