BufferStrategy based buffer flipping

[Jeff]

This one also gets asked alot.  Here is my encapsulation of
all the work necessary to do page flipping for either windowe or
full screen mode from the Scroller example program:

First, there is a generic interface called a ScreenHandler:

1   2   3   4   5   6   7   8   9   10   11   12   13   14   15   16   17   18   19   20   21

/*  * ScreenHandler.java  *  * Created on May 17, 2001, 9:19 AM  */ package com.worldwizards.scroller; import java.awt.*; import javax.swing.*; /**  *  * @author  Jeff Kesselman  * @version  */ public interface ScreenHandler {     public Graphics getCurrentGraphics();     public boolean swapScreens();     public Image getVolatieBuffer();     public Image getBuffer();     public void add(JComponent c); }

The real work is done by an implementation of this called (for hsitorical reasons) FullscreenHandler.  Its contructor takes a boolean.  if true it goes into full screen mode, if false it creates a frame instead.
(Note that there are very few differences  in the code path for each.)

1   2   3   4   5   6   7   8   9   10   11   12   13   14   15   16   17   18   19   20   21   22   23   24   25   26   27   28   29   30   31   32   33   34   35   36   37   38   39   40   41   42   43   44   45   46   47   48   49   50   51   52   53   54   55   56   57   58   59   60   61   62   63   64   65   66   67   68   69   70   71   72   73   74   75   76   77   78   79   80   81   82   83   84   85   86   87   88   89   90   91   92   93   94   95   96   97   98   99   100   101   102   103   104   105   106   107   108   109   110   111   112   113   114   115   116   117   118   119   120   121   122   123   124   125   126   127   128   129   130   131   132   133   134   135   136   137   138   139   140   141   142   143   144   145   146   147   148   149   150   151   152   153   154   155   156   157   158   159   160   161   162   163   164   165   166   167   168   169   170

/*  * FullscreenHandler.java  *  * Created on May 17, 2001, 9:24 AM  */ package com.worldwizards.scroller; import java.awt.*; import java.awt.event.*; import java.awt.image.BufferStrategy; import javax.swing.*; /**  *  * @author  Jeff Kesselman  * @version  */ public class FullscreenHandler implements ScreenHandler {    BufferStrategy bufferStrategy;    Graphics lastGraphics;    GraphicsDevice device;      JFrame mainFrame;    static int screenTopLeftX;    static int screenTopLeftY;    static int screenWidth;    static int screenHeight;        private static DisplayMode[] BEST_DISPLAY_MODES = new DisplayMode[] {         new DisplayMode(640, 480, 32, 0),         new DisplayMode(640, 480, 16, 0),         new DisplayMode(640, 480, 8, 0)     };     /** Creates new FullscreenHandler */     public FullscreenHandler (boolean fullScreen) {         try {             GraphicsEnvironment env = GraphicsEnvironment.                 getLocalGraphicsEnvironment();             device = env.getDefaultScreenDevice();             GraphicsConfiguration gc = device.getDefaultConfiguration();             mainFrame = new JFrame(gc);             mainFrame.getContentPane().setLayout(null);             mainFrame.getContentPane().setIgnoreRepaint(true);             mainFrame.getRootPane().setIgnoreRepaint(true);             mainFrame.getLayeredPane().setIgnoreRepaint(true);             mainFrame.setIgnoreRepaint(true);             if (fullScreen) {                 mainFrame.setUndecorated(true);                 device.setFullScreenWindow(mainFrame);                 if (device.isDisplayChangeSupported()) {                     chooseBestDisplayMode(device);                 }             }  else {                   DisplayMode dm = device.getDisplayMode();                 screenWidth = dm.getWidth();                 screenHeight = dm.getHeight();                 //mainFrame.setUndecorated(true);                mainFrame.setSize(new Dimension(screenWidth,screenHeight));                                 Insets ins = mainFrame.getInsets();                 screenTopLeftX = ins.left;                 screenTopLeftY = ins.top;                 screenWidth -= screenTopLeftX;                 screenHeight -= screenTopLeftY;                 mainFrame.show();             }             mainFrame.createBufferStrategy(2);             bufferStrategy = mainFrame.getBufferStrategy();                                                   } catch (Exception e){             device.setFullScreenWindow(null);         }             }         public void setRepaintOff(Component c){         c.setIgnoreRepaint(true);             if (c instanceof java.awt.Container) {             Component[] children = ((Container)c).getComponents();             for(int i=0;i<children.length;i++) {                 setRepaintOff(children[i]);             }         }     }             public void add(JComponent c) {             setRepaintOff(c);         mainFrame.getContentPane().add(c);     }       public void addMouseMotionListener(MouseMotionListener listener) {         mainFrame.addMouseMotionListener(listener);     }         public void addMouseListener(MouseListener listener) {         mainFrame.addMouseListener(listener);     }         public void paintComponents(Graphics g) {         mainFrame.getContentPane().paintComponents(g);     }         private static DisplayMode getBestDisplayMode(GraphicsDevice device) {         for (int x = 0; x < BEST_DISPLAY_MODES.length; x++) {             DisplayMode[] modes = device.getDisplayModes();             for (int i = 0; i < modes.length; i++) {                 if (modes[i].getWidth() == BEST_DISPLAY_MODES[x].getWidth()                    && modes[i].getHeight() == BEST_DISPLAY_MODES[x].getHeight()                    && modes[i].getBitDepth() == BEST_DISPLAY_MODES[x].getBitDepth()                    ) {                     return BEST_DISPLAY_MODES[x];                 }             }         }         return null;     }         private static void chooseBestDisplayMode(GraphicsDevice device) {         DisplayMode best = getBestDisplayMode(device);         if (best != null) {             device.setDisplayMode(best);         }         screenWidth = best.getWidth();         screenHeight = best.getHeight();     }     public Graphics getCurrentGraphics(){         lastGraphics = bufferStrategy.getDrawGraphics();         lastGraphics.setClip(0,0,screenWidth,screenHeight);         lastGraphics.translate(screenTopLeftX,screenTopLeftY);         return lastGraphics;     }         public boolean swapScreens(){         boolean done = false;         if (!bufferStrategy.contentsLost()){             bufferStrategy.show();             done = true;         }             lastGraphics.dispose();         return done;     }         public Frame getFrame(){         return mainFrame;     }         public int getWidth() {         return mainFrame.size().width;     }         public int getHeight() {         return mainFrame.size().height;     }         public void dispose() {         device.setFullScreenWindow(null);     }         public Image getVolatieBuffer() {         // try ot get a volatile image the size of the screen        return device.getDefaultConfiguration().createCompatibleVolatileImage(getWidth(),             getHeight());     }         public Image getBuffer() {         return mainFrame.createImage(getWidth(),getHeight());     }         }

Finally here's an exmple main loop from Scroller showing its use:

1   2   3   4   5   6   7   8   9   10   11   12   13   14   15   16   17   18   19   20   21   22   23

<snip>    screen = new FullscreenHandler(false); <snip>   long frameStartTime;   while(gameNotDone) { // game loop      frameStartTime = System.currentTimeMillis();       Graphics g = screen.getCurrentGraphics();       g.clearRect(0,0,screen.getWidth(),screen.getHeight());       bkgPainter.paintBackground(g,0,0,           screen.getWidth(),screen.getHeight());             sprites.paint(g);       screen.swapScreens();       sprites.animate();       long deltatime = System.currentTimeMillis()-frameStartTime;       while (deltatime<(1000/FPS)){          try {             Thread.sleep((1000/FPS)-deltatime);          } catch (Exception e) {             e.printStackTrace();          }         deltatime = System.currentTimeMillis()-frameStartTime;       }    }

A few notes on the above:
(1) Its simplified from whats in scroller.  Scroller handles a scrolling
    background, mouse clicks for sprite movement, etc.
(2) The use of System.currentTimeMillis(0 is not good and will fail on    
     Win95, Win98, and WinME  systems which have very bad
     accuracy for  System.currentTimeMillis().  For a better timing
    solution see the "Sleep Timer hack" posted sperately.

[Jeff]

I've been asked to explain the video model above to peopel whose only experence with Video is the 'at-arms-length" thinsg you do with AWT.

First, forget about AWT.,  We are going aroudn it here and for very good reasons. AWT gives you niether the performance nor the control necessary for cutting edge games.

I'm going to start by explaing the Full Screen buffer flipping model and then, by extension, will talk about the windowed mode version.

Your graphics card has a big hunk of RAM on it.  Only a small part of that RAM is displayed on screen at any given time.  Which part is displayed is controlled by something called the "video pointer."

The video pointer points at the memory for the first pixel on your screen.  (Usually the top left one in order to match the right-to-left and top-to-bottom way video monitors scan out pixels.  Otehr mreo complex hardware designs are possible though where the system corrects for the mismatch when displaying video by doing things like starting at the last pixel and scanning backwards through memory.)

The memory from the start pixel up to the last displayable pize is displayed.  So for instance if yo uare in 32 bit (4byte) pixel mode and are set to 1024 x 768 resolution, the next 1024 * 768 * 4 = 3145728 bytes = 3 megabytes of RAM is displayed.

Now most vidoe cards have at least 16meg of ram these days.  This means that you could fit up to 5 of these screens on the card at once.  By simply  changing the video pointer from one screen to the next, you can animate at up to the frame scan rate (typically abt 70hz) the same way you animate with a flip book.

Okay, got that much?  There's more.

Video card RAM is special in other ways.  All the vidoe card's hardware accellerated drawing capailities draw to video ram.  If you copy image data from vidoe RAM to video RAM it is also much faster then ciopying to or from the computer's main  memory.

Okay so heres how it all gets put together.  I'll describe double buffer ing first and then show why in practice its best to extend to trippel buffering.

In double buffering you have one image on the screen and one in vidoe ram not yet displayed.  You do all your drawing to that image and, when it is done, with the write of a single pointer, flip the screens.  Since all the drawing is hidden and the flip only happens when you are done, you get nice clean aniamtion frames similar to (but much faster then) what double buffering in AWT gives you.
Assuming you can draw your screens at the scan rate of the video montior or better you can get animation rates up to the reresh rate of your monitor.  

In order to draw those frames msot efficiently, the image soruces for thinsg like ships, players, etc are also ideally held in another part of video RAM.

So thats double buffering, why do we nother to tripple buffer?  The reason is this.  Remember earlier I talked about how video pixels are scanned squentially otu of memory?  Thsi is because video monitors really only draw one pixel at a time.  They "scan" them out very quickly in left-to-right rows from top-to-bottom-- so quickly yo uarent even aware of it.  When its done, it moves its pixel-crayon (the electron gun beam) back to the top corner and starts all over again.  The period of time in which it is moving that pixel-crayon back is called the "video retrace."

If I were to chnage the video pointer while the pixel-crayon is in motion across the screen, I would get part of one frame in the pixels already drawn and part of the next frame in the pixels drawn after the change.  This is what causes the visual phenomenon of "video tear".
The solution is to wait to change the video pointer until the video retrace.  This action of waiting for the video retrace is called scan-synchronization.

What does all this have to do with tripple buffering?  I'm glad you asked  Lets say we finish drawing a frame and are now waiting for video retrace.  W cannot make any chnages to the buffer that is waiting for display, its done.  We cannot make any changes to the currently displayed buffer because its still on the screen.  The result is that all drawing stops while we wait for the video retrace.  Thsi is wasteful and hurts your frame rate.

If we had ANOTHER buffer we could start drawing to it without waiting  for the video retrace.  Enter tripple buffering.  In a tripple buffering scheme you have a ring of 3 buffers that are displayed in sequence.  At any given time one is on the screen, one can be waiting to be displayed, and one is being drawn to.

And THATS how full screen buffer flipping works.  BufferStrategy builds the buffer chain and manages it for you so all you need to do is request a graphics for the current buffer to draw to, darw to it, tell BufferStrategy to advance and show the next frame, and do it again.

Whew.  Got all that?  Time for the last complication.  Buffer Stratgey cna be used in Windowed as well as FullScreen mode.  How does that work? if you change the video pointer you lose everything that in all the other windows  on the screen.

The answer is that it fakes it.  It still gives you an offscreen buffer to draw to but rather then flipping the video pointer, when you request it to advance it copies the next frames data into the window's area in the displayed image.

Windows APIs provide no way to scan synchronize such a copy (an oversight on their part) so some tearing can and will occur in this mode.

[swpalmer]

In triple buffering mode when you call the Java API to swap buffers, what happens?  Typically in the double buffered case this is where it waits for the video retrace before it flips buffers and returns. In the triple buffer case you want to be able to go on to the next buffer without waiting.. so is it correct to assume the swap API does not block after drawing buffer 2.. but if you get buffer 3 drawn before it is done displaying buffer one then the swap API will block?
When the swap API does not block, I assume the, still to come, display buffer flip is triggered by the vertical interrupt, right?
In other words.. triple buffering just works and we don't have to worry about this stuff?
What about triple buffering with windowed mode? What triggers the blit in that case if it can't sync to the vertical retrace?

Games published by our own members! Check 'em out!

[kenrod]

In line with swpalmer, is there some native 'magic' that happens when you use BufferStrategies?

If I'm developing a straightforward applet with VolatileImages, is there any benefit to going with triple buffering as opposed to 'manual' double buffering (given that you can't sync to the retrace)?

And, if there IS, is this benefit only available if you use a BufferStrategy (as opposed to, say, 'drawImage'ing the buffers yourself)?

Is all this BufferStrategy stuff really only of benefit to full screen (where we DO, I presume, get some native 'magic')?

[Jeff]

In triple buffering mode when you call the Java API to swap buffers, what happens?  
<snip>
In other words.. triple buffering just works and we don't have to worry about this stuff?

Correct it basically blocks until the next buffer in the chain is free.

So it actually WILL block up some in 3 buffer mode  if you manage to draw 2 entire buffers with time elft over before the vertical retrace.  In general thats very unlikelt.

What about triple buffering with windowed mode? What triggers the blit in that case if it can't sync to the vertical retrace?

Blt is done right away and the call returns.  Note that you may not see it right away if you have an asynchronous display mechansim  (ie X-windows.)

[Jeff]

In line with swpalmer, is there some native 'magic' that happens when you use BufferStrategies?

There can be.  There may not be.  In general it is the responsability of the implementor of BufferStrategy to implement the buffer flipping chain in the visually cleanest and most efficient way for the system you are on and mode you are in.  Its safe to say if you can do better in straight Java code then the implementor screwed up.

If I'm developing a straightforward applet with VolatileImages, is there any benefit to going with triple buffering as opposed to 'manual' double buffering (given that you can't sync to the retrace)?

I've never used BufferStrategy in an Applet.  Im not sure it even makes sense there.  The problem being that BufferStrategy is designed for a tight continuos flip loop.  Thsi is appropriate when writing stand alone games because you can, and should, hog all the processor you can.  Applets though need to co-exist with a complex containing application.

And, if there IS, is this benefit only available if you use a BufferStrategy (as opposed to, say, 'drawImage'ing the buffers yourself)?

Is all this BufferStrategy stuff really only of benefit to full screen (where we DO, I presume, get some native 'magic')?

Well let me catagorize it this way: Its of most valeu in full screen applications.  Its still of value in windowed applications though because it gives you tight synchronous control of the painting, something most games really need.

I don't knwo if it even makes sense in Applets.  ive never eally thought about that til now.

[swpalmer]

when writing stand alone games because you can, and should, hog all the processor you can.

The key there is 'stand alone'  if you are coding for the PC/Mac that may not be the situation.  There is a reason that you trigger page flips with an interrupt and not a busy polling loop.

BufferStrategy is designed for a tight continuos flip loop.

What about for a game like say PacMan that could easily be applet based, where you should have tons of extra CPU, and you are using some method of frame rate limiting.  Is using a BufferStrategy for something that is frame rate limited in windowed mode still a good idea?  Or is it a waste at that point?

[blahblahblahh]

In an attempt to resurrect this thread, in order to get that last question answered , I'm going to ask another:

  - Is any of this worth doing for linux users, given that fullscreen mode is currently not available for platforms other than windows? (please correct me if this is wrong; it appears to be the case) ... is coding for linux, MAC OS X, etc, "just the same as for working in windowed mode: some benefit, but not much" ?

What about for a game like say PacMan that could easily be applet based, where you should have tons of extra CPU, and you are using some method of frame rate limiting.  Is using a BufferStrategy for something that is frame rate limited in windowed mode still a good idea?  Or is it a waste at that point?

[swpalmer]

Fullscreen mode is available on Mac OS X.  It has really poor performance in their current 1.4.1 implementation though.  I have hopes that some performance improvements are on the way.

[gregorypierce]

I haven't encountered their performance issues - though I am doing native rendering into the Window that comes from that framework. What types of performance issues are you seeing (particularly on OSX)?

Games published by our own members! Check 'em out!

[swpalmer]

There is another thread around here somewhere where I posted some results of running the Balls.jar test program linked at the start of the thread (not this thread).

Basically it showed that fullscreen mode was significantly slower than windowed mode ... but I didn't investigate further.

[gregorypierce]

It does get a lot slower on unaccelerated surfaces and as with LWJGL on OSX, figuring out which is which on that OS isn't quite straightforward. There are some pretty hairy bugs there. I'd bet good money that if you checked the acceleratedvideomemory it reports it always says -1. So what I did was take its Window, turn off all the decorations and then go into native rendering.

[swpalmer]

Yes it always says -1.. it also says:

Image Capabilities
-Accelerated : NO
-Volatile: NO

Buffer Capabilities
-Flipping: NO
--Full screen only: NO
--Multi-buffering: NO

Front Buffer
-Accelerated: NO
-Volatile: NO

Back Buffer
-Accelerated: YES (??)
-Volatile: NO


Seems kind of lame that there is no page flipping support.  Of course not having volatile/accelerated image support also stinks.