Cheers for the quick response Will. I may have slightly misinformed here.
TriMesh <-> TriMesh collision works. Its just the physical response that doesn't seem to. In the engine thing I'm putting together primitive collision and response works fine (well, I think) but TriMesh response doesn't actually do anything.. i.e. the collisions are detected but the object falls straight through the other...
I'm not 100% why this could be happening. I imagine it gets pretty complicated with this sort of collision. Anyway... looking through the ODE user guide I'm wondering if we currently cope with this situation:
In order to efficiently resolve collisions, dCollideTTL needs the positions of the colliding trimeshes in the previous timestep. This is used to calculate an estimated velocity of each colliding triangle, which is used to find the direction of impact, contact normals, etc. This requires the user to update these variables at every timestep. This update is performed outside of ODE, so it is not included in ODE itself. The code to do this looks something like this:
const double *DoubleArrayPtr =
(void *) DoubleArrayPtr );
The transformation matrix is the standard 4x4 homogeneous transform matrix, and the "DoubleArray" is the standard flattened array of the 16 matrix values.
If not this might explain why the response doesn't happen as expected since ODE can't caculate velocities for the trimesh?
Thanks again for the swift response,