There are a few things to consider. The major difference is that Java3D, Xith3D, jME, AgentFX and a host of others are all scene graphs, JOGL, LWJGL and some others are wrappers to lower level API's. On top of these layers are the scene graphs. jME for example, uses LWJGL underneath.
There are a number of discussions on why to use scenegraphs and why not to use them. They provide you with lots of useful features, scene culling, automatic handling of rendering of transparent objects, may provide other utilities like particle engines for example. On the other hand, generic scene graphs cannot provide any game specific implementations of things.
One will get you up and running quickly, the other offers you total flexability.
At the end of the day, as Mithrandir says, you have to decide which suits your requirements the best, and run with it. I've changed between the two several times, and it normally means throwing out great chunks of code.