PS. Ideally I would like to make the windows collapsible, to reveal more of the game map underneath when they're minimised.
You'll have a hard time doing that with Swing/AWT widgets without it looking like crap. In fact you'll have a hard time using Swing/AWT stuff anywhere in a game and getting it to look good, it breaks the immersion and just screams 'application' rather than 'game'.
You could quite easily contruct some basic GUI widgets yourself to be drawn directly onto your game display, you're only going to be needing simple controls really.