This is most likely a bug in the ati opengl drivers, you could try to install a new driver version.
Yes, this is a known bug in ATI's OpenGL drivers for Windows, see:http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5066318
This bug only occurs when exiting a Swing application (or any Java 2D app that uses VolatileImages). Unfortunately, ATI has not yet fixed this bug (as of their Catalyst 5.12 drivers), so upgrading won't help in this case. The bug is not reproducible when you use the sun.java2d.opengl.fbobject=true property, but that will only work on new-ish ATI boards.