This is generally how you use the Shadow Volume classes.
There is a ShadowNode class that you instantiate within the scene graph. When the renderer draws the scene graph the ShadowNode collects volumes for all of its children, for all of the lights that are in play (for it).
* You can add extra ShadowNodes beneath to turn shadows on and off for different sub items or for different light collections.
* A ShadowNodes children stop at the next ShadowNode down in the graph.
* ShadowNodes cache the volumes they create in case nothing much changes.
In simpleRender you use a ShadowRenderer object to render the shadow volumes that have been collected. This uses a modified version of StencilState that enables or disables the stencil test (I think its missing from the JME code I have) and a new ColorState class that turns on and off writing to the colour buffers.
* Culling should probably be turned off for the ShadowNode - but it will collect volumes for unculled children so perhaps some intelligent culling would be in order to stop unnecessary volumes being created. (They are cached however). Each of the shadows volumes is independently culled when they are rendered, so it should save on fill rate.
There are many ways you can render shadows - but I found additive lighting was too inefficient for my purposes (on at least one of my card) and it is also messier without full multi-pass rendering. So, I render the darker areas in one of two ways - either I render a shade of transparent black for each of the lights in play (as shown in the examples linked above) or I render for each of R G and B to allow coloured lights to have different effects on shadowed areas - which is nice, but due to the non-additive effect these are not attenuated.
There is potentially one even faster way which is to just render once for all lights - but then you don't get "deeper" shadows in areas of overlap, so I've left that out - if you want it really really quick then you use just a single light!
Now to algorithms for the stencil bit - with the state mods I mentioned above, all of this is done through a normal JME scenegraph node (to which the ShadowRenderer adds the volumes)- I managed to get rid of all LWJGL direct calls - which I am really happy about, its very clean and should port to JOGL fine - just implementing the new states and the adding a clearStencilBuffer method in line with mods I have made to the Renderer interface.
Ok to the ZPass / ZFail and all that jazz.
I am using ZPass and have half implemented a capping algorithm to stop problems when parts of the near plane are within the shadow volume. You need to do this for ZPass and it isn't that easy :-/ ZFail is much easier (but involves more geometry rendering).
Most of the problem comes from the complete lack of resources for ZPass capping code - every paper I can find out there uses ZFail.
I remain highly unconvinced that Creative can uphold a patent on all of the published versions of ZFail when just about everyone who has implemented shadows since 99 must have used the technique and Dietricht claims prior art from a presentation he gave at a Creative show. Carmack says he would have had to implement a two pass algorithm and I think that means the patent covers the use of two sided stencil testing - but I can't find the actual patent application, so who really knows.
From what I can tell Creative have not said that they do or do not own a patent on ZFail - they say they own a patent on the algorithm used by Carmack, Carmack agrees (why???). Carmack invented ZFail. Maybe I haven't read the right thing but to me that sound like 1 + 1 = 3 - who knows what other algorithms Carmack invented. All of the ZFail papers from 2001-2003 talk about ZFail as two pass until the general availability of two sided stencil testing - Creative didn't sue any one else, but given the wide availability of the information loads of people must have used the technique.
>> Stop RANT
I am guessing JME will not want to use ZFail so I have been scratching my head about this capping thing.
Shadow Volumes work by extruding the edges that connect triangles facing the light with triangle that don't (and triangles that are unconnected). Extruding basically means creating a quad that uses the vector of the light and the point to push out two further points from a line.
Basically if this line intersects the near clip plane then there is a problem and the ZPass counting gets screwed. My plan is to test every point in each quad to see which side of the near clip plane it is on. If points are on different sides then that volume has the following extra work done to it. (Note: it might not just be extruded edges that clip the plane).
* The following should be done for every "loop" of extruded edges within the shape to cope with non-convex bodies.
I calculate the point of intersection of every quad edge with the near clip plane (in fact just in front of it). I draw a series of triangles that fill the space between each of these points on that plane. This should fill the hole left by clipping.
At the moment this "kindof" works
When I draw the volume I create it looks fine, but something goes wrong when it hits the stencil buffer (I may have the faces pointing the wrong way, I may be overdrawing too many times, I wish I knew).
Anyway - hope that helps. Still getting the code together for Mojo to have a look at / host.
Any suggestions or help on these final bits will be gratefully received!!!!