If none of the games at http://www.home.zonnet.nl/duijs24/jemu/
don't work in IE either, I assume java is not enabled in IE.
If you have java installed, you should see at Internet Options->Advanced a checkbox with 'Java (Sun)' which has to be enabled. If the checkbox is not there, then maybe java is not properly installed (re-installing java should do the trick).
I assume if the applet works on one browser, it should work in another browser as well, provided that they both support java (and not just the MS VM, which is not really java).
If you want to have your code running on the MS VM as well and you don't use any features of java 1.2 and up, you should compile your code with a -target 1.1 option.
I hope this helps,