Java-Gaming.org Hi !
Featured games (91)
games approved by the League of Dukes
Games in Showcase (806)
Games in Android Showcase (239)
games submitted by our members
Games in WIP (868)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  Skeletal Animation  (Read 20768 times)
0 Members and 1 Guest are viewing this topic.
Offline CopyableCougar4
« Posted 2016-06-21 17:22:10 »

So I have been working on skeletal animation and skinning for a few weeks, and I hit a road block recently. I am importing the models from the MD5 format, and I have skinning and rendering meshes working.

Everything works fine until I try to animate the model and my inverse bind pose matrices distort the model.

Here is a screenshot of what is happening. The left is what the static model looks like while the right shows what happens after I load the animation and compute the inverse bind pose matrices.


Here is how I compute the bind pose matrices (I call computeBindPose() and traverse the bone hierarchy):
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
public void computeBindPose() {
      computeBindPose(new Matrix4f(), new Matrix4f(), new Matrix4f());
   }
   
   private void computeBindPose(Matrix4f base, Matrix4f tmp, Matrix4f tmp2) {
      if (bindPose != null)
         bindPose.computeMatrix(tmp);
      base.mul(tmp, bindPoseMatrix);
      bindPoseMatrix.invert(invBindPoseMatrix);
      children.forEach(joint -> joint.computeBindPose(bindPoseMatrix, tmp, tmp2));
   }


Here is my shader for rendering the meshes:
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
#version 150 core

uniform mat4 projectionMatrix;
uniform mat4 viewMatrix;
uniform mat4 modelMatrix;

in vec3 in_Position;
in vec3 in_Normal;
in vec2 in_TextureCoord;
in vec4 in_BoneIndices;
in vec4 in_BoneWeights;

uniform mat4 boneMat[64];

out vec2 pass_TextureCoord;

void main(void) {
   vec4 newVertex = vec4(0.0);
   vec4 newNormal = vec4(0.0);
   
   mat4 boneTransform = boneMat[int(in_BoneIndices.x)] * in_BoneWeights.x + boneMat[int(in_BoneIndices.y)] * in_BoneWeights.y + boneMat[int(in_BoneIndices.z)] * in_BoneWeights.z + boneMat[int(in_BoneIndices.w)] * in_BoneWeights.w;
   
   newVertex = boneTransform * vec4(in_Position, 1.0);
   newNormal = boneTransform * vec4(in_Normal, 0.0);

   gl_Position = (projectionMatrix * viewMatrix * modelMatrix) * vec4(newVertex.xyz, 1.0);
   pass_TextureCoord = in_TextureCoord;
}

Either wandering the forum or programming. Most likely the latter Smiley

Github: http://github.com/CopyableCougar4
Offline theagentd
« Reply #1 - Posted 2016-06-21 19:34:56 »

How do you compute the matrices?

Myomyomyo.
Offline CopyableCougar4
« Reply #2 - Posted 2016-06-21 20:12:32 »

This is my code for computing them:
1  
2  
3  
public void computeMatrix(Matrix4f matrix) {
      matrix.translationRotateScale(offset, rotation, scaling);
   }


I read the offset and rotation from the baseframe section of the MD5 file.

Either wandering the forum or programming. Most likely the latter Smiley

Github: http://github.com/CopyableCougar4
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline theagentd
« Reply #3 - Posted 2016-06-21 21:41:39 »

I'm not awake enough to go through your code, but you're definitely doing something wrong.

Here's my code for this:

1. Precompute bindpose: matrices = translationRotateScale(offset, rotation, scaling).invert(). Make sure that this is transformed by the parent as well:
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
Vector3f translation = new Vector3f(joint.getTranslation());
Quaternionf orientation = new Quaternionf(joint.getOrientation());
Vector3f scale = new Vector3f(joint.getScale());
if(joint.getParent() != null){
   parentOrientation.transform(translation);
   translation.mul(parentScale).add(parentTranslation);
   parentOrientation.mul(orientation, orientation);
   scale.mul(parentScale);
}
matrix[i] = new Matrix4f().translationRotateScale(translation, orientation, scale).invert();


2. When animating, interpolate between two frames, transform by the (animated) parent joint, generate a matrix and multiply it with the precomputed matrix:
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
//Interpolate between translations, orientations and scales. Use lerp() for Vector3fs and slerp/nlerp/nlerpIterative() for quaternions.

Vector3f translation = ...; //lerp()
Quaternionf orientation = ...; //slerp()
Vector3f scale = ...; //lerp()
if(joint.getParent() != null){
   //Note that this is the already lerped/slerped values of the parent.
   parentOrientation.transform(translation);
   translation.mul(parentScale).add(parentTranslation);
   parentOrientation.mul(orientation, orientation);
   scale.mul(parentScale);
}
matrix[i].translationRotateScaleMulAffine(translations[i], orientations[i], scales[i], bindPoseMatrices[i]);

Myomyomyo.
Offline KaiHH

JGO Kernel


Medals: 798



« Reply #4 - Posted 2016-06-21 22:05:44 »

One performance improvement. In
new Matrix4f().translationRotateScale(...).invert();
do
.invertAffine()
instead.
invert() = 94 muls, 49 adds
invertAffine() = 57 muls, 24 adds

Oh, and since you always have access to the individual translation, orientiation and scale parts, you can get rid of the invert() call altogether by applying the inverse of the individual components in the order: scale^-1 * orientation^-1 * translation^-1
where scale^-1 is just 1/scale, orientation^-1 is just orientation.conjugate() and translation^-1 is just translation.negate().
But I guess since the invbindpose matrix is precomputed, this is not a big gain. Smiley
Offline theagentd
« Reply #5 - Posted 2016-06-21 22:18:43 »

It's probably <1000 bones, even for AAA games. It's not worth complicating the code for it.

Myomyomyo.
Offline KaiHH

JGO Kernel


Medals: 798



« Reply #6 - Posted 2016-06-21 22:46:52 »

It would of course be a simple Matrix4f.translationRotateScaleInvert() Smiley
taking the same parameters that translationRotateScale() takes.

EDIT: Latest 1.8.1-SNAPSHOT contains this new method. JMH benchmarking shows a 2x performance boost over translationRotateScale().invertAffine()
Offline CopyableCougar4
« Reply #7 - Posted 2016-06-23 17:21:54 »

I've been looking over my code and trying to get it working, but for the life of me I don't know why it looks so distorted.

What I know:
MD5 animation files use the baseframe data and then replace values each frame (based on bit flags) to represent the orientation and position of each bone, and part of the calculation includes computing the w-component of the quaternions. This data is used (as well as the values for each joint's parent) to compute the bone matrix, which is applied in the shader. Interpolating between frames is done by lerp() for translation and scale and slerp() for orientation.

What I'm doing
This is my code for computing the bone matrix.
computeBoneMatrix()
is called for the root bone and it traverses the hierarchy. The translation, scale, and orientation are instance fields for each joint.
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
public void computeBoneMatrix() {
      computeBoneMatrix(null);
   }
   
   private Vector3f translation = new Vector3f(), scale = new Vector3f();
   
   private Quaternionf orientation = new Quaternionf();
   
   private void computeBoneMatrix(Joint parent) {
      translation.set(animationSequence.getTranslation());
      orientation.set(animationSequence.getOrientation());
      scale.set(animationSequence.getScale());
      if (parent != null) {
         parent.orientation.transform(translation);
         translation.mul(parent.scale).add(parent.translation);
         parent.orientation.mul(orientation, orientation);
         scale.mul(parent.scale);
      }
      bindPoseMatrix.translationRotateScale(translation, orientation, scale).invert(invBindPoseMatrix);
      children.forEach(joint -> joint.computeBoneMatrix(this));
   }

How I compute the w-component of the quaternions:
1  
2  
3  
4  
5  
private Quaternionf convertQuat(MD5Vector3 orientation) {
      float t = 1.0f - (orientation.getX() * orientation.getX() + orientation.getY() * orientation.getY() + orientation.getZ() * orientation.getZ());
      float w = t < 0 ? 0 : (float)-Math.sqrt(t);
      return new Quaternionf(orientation.getX(), orientation.getY(), orientation.getZ(), w);
   }

How I replace data for generating the frames:
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
int j = 0;
         if ((flags & 1) != 0) {
            position.x = frame.getData()[joint.getStartIndex() + j];
            j++;
         }
         if ((flags & 2) != 0) {
            position.y = frame.getData()[joint.getStartIndex() + j];
            j++;
         }
         if ((flags & 4) != 0) {
            position.z = frame.getData()[joint.getStartIndex() + j];
            j++;
         }
         if ((flags & 8) != 0) {
            orientation.x = frame.getData()[joint.getStartIndex() + j];
            j++;
         }
         if ((flags & 16) != 0) {
            orientation.y = frame.getData()[joint.getStartIndex() + j];
            j++;
         }
         if ((flags & 32) != 0) {
            orientation.z = frame.getData()[joint.getStartIndex() + j];
            j++;
         }

How I interpolate between frames:
1  
2  
3  
4  
5  
public void lerp(JointTransform end, float t, JointTransform destination) {
      offset.lerp(end.offset, t, destination.offset);
      rotation.slerp(end.rotation, t, destination.rotation);
      scaling.lerp(end.scaling, t, destination.scaling);
   }


I tried to pick out the relevant parts of code, but I can paste more if necessary. I just simply don't understand why this code isn't working as multiple tutorials (as well as other people's code compiled into games) use the same code and get working skeletal animation.

Either wandering the forum or programming. Most likely the latter Smiley

Github: http://github.com/CopyableCougar4
Offline theagentd
« Reply #8 - Posted 2016-06-24 02:19:27 »

Hmm. That all looks correct.

 - Are you trying to do some kind of conversion from Y-up to Z-up or something like that? It's not easy to do that with quaternions.

 - You've shown the code for generating the bind pose matrix. Where's the code for computing an animated matrix?


The intuitive idea behind skeleton animation is to create a local coordinate system for each bone, which is constructed from a translation and a rotation (and a scale, but ignore that for the intuition). Basically, we want a way to calculate the position of a vertex relative to a bone. This is done the exact same way we compute a view matrix: Create a matrix with the transform of the camera, then invert it.

Example: We have a camera at position (1, 1, 1) and construct a matrix:
matrix.translation(cameraPosition);
is a matrix which simply adds the camera's position to each vertex. If we have a vertex at (0, 0, 0) relative to the camera and want to know where it is in the world, we just add the camera's position to it. Since the vertex is at the same point as the camera in this case (it is at (0, 0, 0) relative to the camera after all), the vertex is at (1, 1, 1) too. Easy to understand. However, we already have vertices in world space, and want to know where they are relative to the camera so we can draw them on a screen. So, we just invert the matrix we made, which in this case is the same as
matrix.translation(-cameraPosition);
since it's a simple transformation. It's clear that if we have a world space vertex at (1, 1, 1) and apply a (-1, -1, -1) translation to it, we end up at (0, 0, 0) relative to the camera again, as we should.

We do the same thing when computing the "inverse bind pose matrix" or whatever you want to call it. We first calculate the default transform matrices of each bone of the bind pose, then invert it to create a matrix that takes us from model space to a coordinate system relative to the bone. Simply put, it allows us to calculate where a given vertex is compared to a bone. Now, why is this useful? By calculating where a vertex is relative to a bone, we can move the bone and calculate a new position of every vertex affected by it easily by simply calculating the relative position of a vertex and then taking it back to model space again using a different (animated) bone matrix. The result is what we call skeleton animation.

What you want to do is simply this:
1  
2  
vec4 localPosition = inverseBindPoseMatrix * modelSpacePosition;
vec4 newModelSpacePosition = animatedBoneMatrix * localPosition;
which can be rearranged like this:
1  
2  
vec4 newModelSpacePosition = animatedBoneMatrix * (inverseBindPoseMatrix * modelSpacePosition);
vec4 newModelSpacePosition = (animatedBoneMatrix * inverseBindPoseMatrix) * modelSpacePosition;

In other words, you can precompute a single bone matrix which takes the vertex from its current model space position directly to its new model space position by precomputing (animatedBoneMatrix*inverseBindPoseMatrix).

Now, bone animation generally uses a weighted average of 4 different bones for each vertex. That just means that we compute the new position that each bone would give us and average together the results.

1  
2  
3  
4  
5  
vec4 newModelSpacePosition = 
        ((animatedBoneMatrix1 * inverseBindPoseMatrix) * modelSpacePosition) * weight1 +
        ((animatedBoneMatrix2 * inverseBindPoseMatrix) * modelSpacePosition) * weight2 +
        ((animatedBoneMatrix3 * inverseBindPoseMatrix) * modelSpacePosition) * weight3 +
        ((animatedBoneMatrix4 * inverseBindPoseMatrix) * modelSpacePosition) * weight4;
which can be rearranged to:
1  
2  
3  
4  
5  
vec4 newModelSpacePosition = 
        (animatedBoneMatrix1 * inverseBindPoseMatrix) * weight1 * modelSpacePosition +
        (animatedBoneMatrix2 * inverseBindPoseMatrix) * weight2 * modelSpacePosition +
        (animatedBoneMatrix3 * inverseBindPoseMatrix) * weight3 * modelSpacePosition +
        (animatedBoneMatrix4 * inverseBindPoseMatrix) * weight4 * modelSpacePosition;
and then to
1  
2  
3  
4  
5  
6  
7  
vec4 newModelSpacePosition = 
        (
            ((animatedBoneMatrix1 * inverseBindPoseMatrix) * weight1) +
            ((animatedBoneMatrix2 * inverseBindPoseMatrix) * weight2) +
            ((animatedBoneMatrix3 * inverseBindPoseMatrix) * weight3) +
            ((animatedBoneMatrix4 * inverseBindPoseMatrix) * weight4)
        ) * modelSpacePosition;
which is the most efficient way of doing it and what you're doing in your shader already. Wink

This may all look complicated at a glance, but it's really just multiplication and addition, just on coordinates and matrices. If you can grasp how this works, then you should be able to debug your code and fix it. There really isn't a shortcut to getting skeleton animation to just work without understanding it, and if you can grasp it you'll be one of the few people in the world who fully understands how this works.

Myomyomyo.
Offline Stranger
« Reply #9 - Posted 2016-06-25 06:31:55 »

Hi.

I worked few years ago on the same problem.
I post fragments of my code. Maybe this will help.

Joint.java

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  
42  
43  
44  
45  
46  
47  
48  
49  
50  
51  
52  
53  
54  
55  
56  
57  
58  
59  
60  
public class Joint implements NamedObject
{
    public final int NULL_PARENT = -1;

    private final short parentIndex;
    private final String name;
    private final Vector3f bindTranslation;
    private final Quaternion4f bindRotation;
    private final Tuple3f bindScale;

    private final Matrix4f invBindPoseMatrix;

    private short index;
    private boolean isAux;

    public final boolean hasParent()
    {
        return ( index > 0 );
    }

    public final void calcInvBindPoseMatrix( Joint parent )
    {
        Matrix4f m = Matrix4f.fromPool();

        MathUtils.compose( bindTranslation, bindRotation, bindScale, m );
        invBindPoseMatrix.invert( m );
        if( parent != null )
        {
            invBindPoseMatrix.mul( invBindPoseMatrix, parent.getInvBindPoseMatrix() );
        }

        Matrix4f.toPool( m );
    }
    public Joint( String name, short index, Joint parent, Transform bindTransform, boolean isAux )
    {
        this.parentIndex = parent == null ? NULL_PARENT : parent.getIndex();
        this.name = name;
        this.index = index;
        bindTranslation = bindTransform.getTranslation( null ).getReadOnly();
        bindRotation = bindTransform.getRotation( null ).getReadOnly();
        bindScale = bindTransform.getScale( null ).getReadOnly();

        invBindPoseMatrix = new Matrix4f();
        calcInvBindPoseMatrix( parent );
        this.isAux=isAux;
    }

    public Joint( Joint joint, Joint parent )
    {
        this.parentIndex = parent == null ? NULL_PARENT : parent.getIndex();
        name = joint.name;
        index = joint.index;

        bindTranslation = joint.bindTranslation;
        bindRotation = joint.bindRotation;
        bindScale = joint.bindScale;

        invBindPoseMatrix = joint.invBindPoseMatrix;
    }
}


SkinnedMeshUpdater.java

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  
42  
43  
44  
45  
46  
47  
48  
49  
50  
51  
52  
53  
54  
55  
56  
57  
58  
59  
60  
61  
62  
63  
64  
65  
66  
67  
68  
69  
70  
71  
72  
73  
74  
75  
76  
77  
78  
79  
80  
81  
82  
83  
84  
85  
86  
87  
88  
89  
90  
91  
92  
93  
94  
95  
96  
97  
98  
99  
100  
101  
102  
103  
104  
105  
106  
107  
108  
109  
110  
111  
112  
113  
114  
115  
116  
117  
118  
119  
120  
121  
122  
123  
124  
125  
126  
127  
128  
129  
130  
131  
132  
133  
134  
135  
136  
137  
138  
139  
140  
141  
142  
143  
144  
145  
146  
147  
148  
149  
150  
151  
152  
153  
154  
155  
156  
157  
158  
159  
160  
161  
162  
163  
164  
165  
166  
167  
168  
169  
170  
171  
172  
173  
174  
175  
176  
177  
178  
179  
180  
181  
182  
183  
184  
185  
186  
187  
188  
189  
190  
191  
192  
193  
194  
195  
196  
197  
198  
199  
200  
201  
202  
203  
204  
205  
206  
207  
208  
209  
210  
211  
212  
213  
214  
215  
216  
217  
218  
219  
220  
221  
222  
223  
224  
225  
226  
227  
228  
229  
230  
231  
232  
233  
234  
235  
236  
237  
238  
239  
240  
241  
242  
243  
244  
245  
246  
247  
248  
249  
250  
251  
252  
253  
254  
255  
256  
257  
258  
259  
260  
261  
262  
263  
264  
265  
266  
267  
268  
269  
270  
271  
272  
273  
274  
275  
276  
277  
278  
279  
280  
281  
282  
283  
public class SkinnedMeshUpdater
{  
    private static Map<Skeleton, SkinnedMeshUpdater> skeletonToUpdater = new HashMap<Skeleton, SkinnedMeshUpdater>();
    private final Skeleton skeleton;
    private final Shape3D[] shapes;

    private final float[][] weights;
    private final short[][] jointIndices;

    private float[][] bindPoseCoords;
    private float[][] bindPoseNormals;

    private final int[] influencesPerVertex;

    private final Vector3f[] worldTranslations;
    private final Quaternion4f[] worldRotations;
    private final Tuple3f[] worldScales;

    private Matrix4f[] matrixPalette;

    public Skeleton getSkeleton()
    {
        return skeleton;
    }

    public Shape3D[] getTarget()
    {
        return shapes;
    }

    public float[][] getWeights()
    {
        return weights;
    }

    public short[][] getJointIndices()
    {
        return jointIndices;
    }

    public int[] getInfluencesPerVertex()
    {
        return influencesPerVertex;
    }

    public static SkinnedMeshUpdater create( Skeleton skeleton,
                                             Shape3D[] target,
                                             float[][] weights,
                                             short[][] jointIndices,
                                             int[] influencesPerVertex )
    {
        SkinnedMeshUpdater smu = skeletonToUpdater.get( skeleton );
        if( smu == null )
        {
            smu = new SkinnedMeshUpdater( skeleton, target, weights, jointIndices, influencesPerVertex );
        }

        return ( smu );
    }

    public void update( IAnimationData data )
    {
        if( data == null )
        {
            return;
        }
        updateSkeleton( data );
        for( int i = 0; i < shapes.length; i++ )
        {
            updateGeometry( i );
        }
    }

    private void updateSkeleton( IAnimationData data )
    {
        SkeletalKeyFrame pose = ( SkeletalKeyFrame ) data;
        Skeleton skeleton = pose.getSkeleton();
        for( int i = 0, jc = skeleton.getJointsCount(); i < jc; i++ )
        {
            Joint joint = skeleton.getJoint( i );
            setAbsolutes( joint, pose );
            short ji = joint.getIndex();
            if( joint.isAux() )
            {
                matrixPalette[ ji ].setIdentity();
            }
            else
            {
                mul( joint.getInvBindPoseMatrix(),
                        worldTranslations[ ji ],
                        worldRotations[ ji ],
                        worldScales[ ji ],

                        matrixPalette[ ji ]
                );
            }
        }
    }

    private void updateGeometry( int shapeIdx )
    {
        Shape3D shape = shapes[ shapeIdx ];
        Geometry geom = shape.getGeometry();
        float[] bindPoseCoords = this.bindPoseCoords[ shapeIdx ];
        float[] bindPoseNormals = this.bindPoseNormals[ shapeIdx ];
        int influencesPerVertex = this.influencesPerVertex[ shapeIdx ];
        float[] weights = this.weights[ shapeIdx ];
        short[] jointIndices = this.jointIndices[ shapeIdx ];

        float x, y, z;
        float nX = 0, nY = 0, nZ = 0;
        float vSumX, vSumY, vSumZ;
        float nSumX, nSumY, nSumZ;
        float tmpX, tmpY, tmpZ;

        for( int i = 0; i < geom.getVertexCount(); i++ )
        {
            x = bindPoseCoords[ i * 3 ];
            y = bindPoseCoords[ i * 3 + 1 ];
            z = bindPoseCoords[ i * 3 + 2 ];
            if( geom.hasNormals() )
            {
                nX = bindPoseNormals[ i * 3 ];
                nY = bindPoseNormals[ i * 3 + 1 ];
                nZ = bindPoseNormals[ i * 3 + 2 ];
            }
            vSumX = 0f;
            vSumY = 0f;
            vSumZ = 0f;

            nSumX = 0f;
            nSumY = 0f;
            nSumZ = 0f;

            float wSum = 0f;
            float weight;
            for( int j = 0; j < influencesPerVertex; j++ )
            {
                if( j != influencesPerVertex - 1 )
                {
                    weight = weights[ i * ( influencesPerVertex - 1 ) + j ];
                    wSum += weight;
                }
                else
                {
                    weight = 1 - wSum;
                }
                if( weight == 0f )
                {
                    continue;
                }
                final int jointIndex = jointIndices[ i * influencesPerVertex + j ];
                if( jointIndex == 0 )
                {
                    JAGTLog.debug( "jindex=0, influencesPerVertex=", influencesPerVertex );
                }
                Matrix4f m = matrixPalette[ jointIndex ];
                //vertices
                tmpX = m.m00() * x + m.m01() * y + m.m02() * z + m.m03();
                tmpY = m.m10() * x + m.m11() * y + m.m12() * z + m.m13();
                tmpZ = m.m20() * x + m.m21() * y + m.m22() * z + m.m23();

                vSumX += tmpX * weight;
                vSumY += tmpY * weight;
                vSumZ += tmpZ * weight;

                //normals
                if( geom.hasNormals() )
                {
                    tmpX = m.m00() * nX + m.m01() * nY + m.m02() * nZ;
                    tmpY = m.m10() * nX + m.m11() * nY + m.m12() * nZ;
                    tmpZ = m.m20() * nX + m.m21() * nY + m.m22() * nZ;

                    nSumX += tmpX * weight;
                    nSumY += tmpY * weight;
                    nSumZ += tmpZ * weight;
                }
            }

            geom.setCoordinateWithoutOpenGlHandling( i, vSumX, vSumY, vSumZ );
            if( geom.hasNormals() )
            {
                geom.setNormalWithoutOpenGlHandling( i, nSumX, nSumY, nSumZ );
            }
        }
        geom.setBoundsDirty();
        geom.getOpenGLReference_DL_GeomData().invalidateNames();
        geom.getOpenGLReference_DL().invalidateNames();

        shape.updateBounds( false );
    }

    private static void mul( Matrix4f matrix, Vector3f translation, Quaternion4f rotation, Tuple3f scale, Matrix4f out )
    {
        //out.set( matrix );
        Matrix4f tmp = Matrix4f.fromPool();

        tmp.setIdentity();
        tmp.setTranslation( translation );
        out.set( tmp );

        tmp.set( rotation );
        out.mul( tmp );

        tmp.setIdentity();
        tmp.m00( scale.getX() );
        tmp.m11( scale.getY() );
        tmp.m22( scale.getZ() );
        out.mul( tmp );

        out.mul( matrix/* , out */ );

        Matrix4f.toPool( tmp );
    }

    //doesn't work  when non-uniform scales
    private void setAbsolutes( Joint joint, SkeletalKeyFrame frame )
    {
        if( joint.hasParent() )
        {
            short parentIndex = joint.getParentIndex();
            short localIndex = joint.getIndex();
            MathUtils.mul(
                    worldTranslations[ parentIndex ],
                    worldRotations[ parentIndex ],
                    worldScales[ parentIndex ],

                    frame.getTranslations()[ localIndex ],
                    frame.getRotations()[ localIndex ],
                    frame.getScales()[ localIndex ],

                    worldTranslations[ localIndex ],
                    worldRotations[ localIndex ],
                    worldScales[ localIndex ]
            );
        }
        else
        {
            short localIndex = joint.getIndex();
            worldTranslations[ localIndex ].set( frame.getTranslations()[ localIndex ] );
            worldRotations[ localIndex ].set( frame.getRotations()[ localIndex ] );
            worldScales[ localIndex ].set( frame.getScales()[ localIndex ] );
        }
    }

    public SkinnedMeshUpdater( Skeleton skeleton, Shape3D[] target, float[][] weights, short[][] jointIndices, int[] influencesPerVertex )
    {
        this.skeleton = skeleton;
        this.shapes = target;
        this.weights = weights;
        this.jointIndices = jointIndices;
        this.influencesPerVertex = influencesPerVertex;

        worldTranslations = new Vector3f[ skeleton.getJointsCount() ];
        worldRotations = new Quaternion4f[ skeleton.getJointsCount() ];
        worldScales = new Tuple3f[ skeleton.getJointsCount() ];
        matrixPalette = new Matrix4f[ skeleton.getJointsCount() ];

        for( int i = 0; i < worldTranslations.length; i++ )
        {
            worldTranslations[ i ] = new Vector3f();
            worldRotations[ i ] = new Quaternion4f();
            worldScales[ i ] = new Tuple3f();
            matrixPalette[ i ] = new Matrix4f();
        }

        bindPoseCoords = new float[ target.length ][];
        bindPoseNormals = new float[ target.length ][];

        for( int i = 0; i < target.length; i++ )
        {
            int numVertices = target[ i ].getGeometry().getVertexCount();
            Geometry geom = target[ i ].getGeometry();
            bindPoseCoords[ i ] = new float[ numVertices * 3 ];
            geom.getCoordinates( 0, bindPoseCoords[ i ] );
            if( geom.hasNormals() )
            {
                bindPoseNormals[ i ] = new float[ numVertices * 3 ];
                geom.getNormals( 0, bindPoseNormals[ i ] );
            }
        }
    }
}


Anton
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline CopyableCougar4
« Reply #10 - Posted 2016-06-25 16:58:42 »

So I realized that I was only factoring in the bind pose matrix and not the animated matrix, but everything is still looking distorted. Since much of the code works fine until the model is animated, I *think* I can narrow down my problem to a few snippets.

They can be found here: http://pastebin.java-gaming.org/cf0c4276d4313

Also, in case this is relevant, this is how I determine the initial positions to send to the shader:
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
for (int idx = 0; idx < submesh.getVertices().length; idx++) {
         com.digiturtle.md5.skeleton.Vertex vertex = submesh.getVertices()[idx];
         AnimatedVertex animatedVertex = new AnimatedVertex();
         Vector3f position = new Vector3f().zero();
         Vector3f normal = new Vector3f().zero();
         Vector2f textureCoords = new Vector2f().zero();
         float[] boneWeightsArray = { 0, 0, 0, 0 };
         float[] boneIndicesArray = { 0, 0, 0, 0 };
         Vector3f tmpPos = new Vector3f(), tmpPos2 = new Vector3f();
         textureCoords.set(vertex.getTextureCoords());
         for (int j = 0; j < 4 && j < vertex.getWeightCount(); j++) {
            // position += (joint_position + (joint_orientation * weight_position)) * weight_bias
            Weight weight = submesh.getWeights().get(vertex.getStartWeight() + j);
            Joint joint = skeleton.getJoints().get(weight.getJointIndex());
            tmpPos.set(weight.getPosition()).rotate(joint.getLocalTransform().getOrientation());
            tmpPos2.set(tmpPos).add(joint.getLocalTransform().getTranslation()).mul(weight.getWeightBias());
            position.add(tmpPos2);
            boneIndicesArray[j] = (float) weight.getJointIndex();
            boneWeightsArray[j] = weight.getWeightBias();
         }
         animatedVertex.setPosition(position.x, position.y, position.z);
         animatedVertex.setNormal(normal.x, normal.y, normal.z);
         animatedVertex.setTextureCoord(textureCoords.x, textureCoords.y);
         animatedVertex.setBoneWeights(boneWeightsArray[0], boneWeightsArray[1], boneWeightsArray[2], boneWeightsArray[3]);
         animatedVertex.setBoneIndices(boneIndicesArray[0], boneIndicesArray[1], boneIndicesArray[2], boneIndicesArray[3]);
         geometry.put(animatedVertex.getData());
      }


Either wandering the forum or programming. Most likely the latter Smiley

Github: http://github.com/CopyableCougar4
Offline ziozio
« Reply #11 - Posted 2016-06-25 17:52:46 »

If everything works fine till animation then the issue must be isolated to the routing that uploads the new bone matrix array to the shader (boneMat). How do you upload the data initially and how do you upload the updated data?
Offline Stranger
« Reply #12 - Posted 2016-06-26 06:02:04 »

I'd recommend you to implement software-based animation system first.
You can use flag (useGPU) to switch between software and GPU implementations.

I also struggled with animation data (I implemented Collada loader at that time) and watched good bind pose and wrong animation that resembled something like devil's dances.

One of the first bugs I found was incorrectly computed inverse bind pose matrix.
Also I recommend you to implement skeleton as the flat array not the tree-structure.

Anton
Offline ShadedVertex
« Reply #13 - Posted 2016-07-17 05:15:51 »

I've got the same problem. My mesh looks like some kind of spiky spaghetti.

CopyableCougar4, have you fixed the problem?
Offline CopyableCougar4
« Reply #14 - Posted 2016-07-17 16:19:53 »

I've been working on it but haven't found a solution yet. My mesh looks fine until I animate it. The instant the matrices (bindpose and animation) are no longer identity matrices, everything gets distorted (i.e. the feet are above the torso). Although it does look better than the original distortions.

Either wandering the forum or programming. Most likely the latter Smiley

Github: http://github.com/CopyableCougar4
Offline thedanisaur

JGO Knight


Medals: 59



« Reply #15 - Posted 2016-07-17 16:41:24 »

In your latest pastebin, I don't see where you are removing the bind pose by using the inverse bind pose, before animating.

Are you doing that?

Every village needs an idiot Cool
Offline ShadedVertex
« Reply #16 - Posted 2016-07-18 11:26:35 »

My guess is that the inverse bind pose matrices aren't being computed correctly. Either that, or the per-vertex bone indices and weights are incorrect.
Offline ziozio
« Reply #17 - Posted 2016-07-18 18:31:50 »

if the identity position works fine then it could be a matrix multiplication order issue where you have A*B rather than B*A
Offline CopyableCougar4
« Reply #18 - Posted 2016-07-18 23:52:08 »

I'm away from my development PC but will post relevant code/descriptions tomorrow.

Either wandering the forum or programming. Most likely the latter Smiley

Github: http://github.com/CopyableCougar4
Offline CopyableCougar4
« Reply #19 - Posted 2016-07-19 20:11:46 »

My matrix multiplication order is boneMatrix * inverseBindPose.

I compute the bindpose matrix with
1  
2  
3  
translation = bindPose.getTranslation();
      bindPoseMatrix.translationRotate(translation.x, translation.y, translation.z, bindPose.getOrientation());
      bindPoseMatrix.invert(invBindPoseMatrix);

for each bone, where translation and orientation are based on the baseframe data (with w computed like shown in a previous post).

I compute the bone matrices with
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
int jointIdx = index[0];
         if (parent == null) {
            frameData[jointIdx + 0] = position.x;
            frameData[jointIdx + 1] = position.y;
            frameData[jointIdx + 2] = position.z;
            frameData[jointIdx + 3] = orientationQuat.x;
            frameData[jointIdx + 4] = orientationQuat.y;
            frameData[jointIdx + 5] = orientationQuat.z;
            frameData[jointIdx + 6] = orientationQuat.w;
         } else {
            int parentJointIdx = startIndices.get(new JointID(parent.getName()));
            parentPos.x = frameData[parentJointIdx + 0];
            parentPos.y = frameData[parentJointIdx + 1];
            parentPos.z = frameData[parentJointIdx + 2];
            parentOrient.x = frameData[parentJointIdx + 3];
            parentOrient.y = frameData[parentJointIdx + 4];
            parentOrient.z = frameData[parentJointIdx + 5];
            parentOrient.w = frameData[parentJointIdx + 6];
            parentOrient.transform(position);
            frameData[jointIdx + 0] = position.x + parentPos.x;
            frameData[jointIdx + 1] = position.y + parentPos.y;
            frameData[jointIdx + 2] = position.z + parentPos.z;
            parentOrient.mul(orientationQuat);
            parentOrient.normalize();
            frameData[jointIdx + 3] = parentOrient.x;
            frameData[jointIdx + 4] = parentOrient.y;
            frameData[jointIdx + 5] = parentOrient.z;
            frameData[jointIdx + 6] = parentOrient.w;
         }
         startIndices.put(id, jointIdx);
         Vector3f pos = new Vector3f();
         pos.x = frameData[jointIdx + 0];
         pos.y = frameData[jointIdx + 1];
         pos.z = frameData[jointIdx + 2];
         Quaternionf orient = new Quaternionf(0, 0, 0, 1);
         orient.x = frameData[jointIdx + 3];
         orient.y = frameData[jointIdx + 4];
         orient.z = frameData[jointIdx + 5];
         orient.w = frameData[jointIdx + 6];

and then use Matrix4f.translationRotate(pos.x, pos.y, pos.z, orient).

The above code basically takes the frame data, replaces baseframe data based on the flags, and then adds that to the parent frame data. (I believe I pulled this code snippet from an early version of LibGDX).

Either wandering the forum or programming. Most likely the latter Smiley

Github: http://github.com/CopyableCougar4
Offline thedanisaur

JGO Knight


Medals: 59



« Reply #20 - Posted 2016-07-19 20:53:06 »

Again, can you please point us to where you are undoing the bindpose? I don't see that in your code.

I personally do it in the shader.

Code has been snipped.
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
#version 400 core

layout (location = 0) in vec4 vert_pos;
layout (location = 5) in float matrix_index;
layout (location = 7) in vec4 joint_indices;
layout (location = 8) in vec4 joint_weights;

layout(std140) uniform model_matrix_buff { uniform mat4 model_matrix[32]; };
layout(std140) uniform anim_joint_buff { uniform mat4 joint_matrix[1024]; };
layout(std140) uniform inv_bind_buff { uniform mat4 inv_bind_matrix[1024]; };

uniform mat4 projection_matrix = mat4(1.0);
uniform mat4 view_matrix = mat4(1.0);

void main()
{
   vec4 t_pos =
       ((vert_pos * inv_bind_matrix[int(joint_indices.x)] * joint_matrix[int(joint_indices.x)]) * joint_weights.x) +
       ((vert_pos * inv_bind_matrix[int(joint_indices.y)] * joint_matrix[int(joint_indices.y)]) * joint_weights.y) +
       ((vert_pos * inv_bind_matrix[int(joint_indices.z)] * joint_matrix[int(joint_indices.z)]) * joint_weights.z) +
       ((vert_pos * inv_bind_matrix[int(joint_indices.w)] * joint_matrix[int(joint_indices.w)]) * joint_weights.w);
       
   gl_Position = projection_matrix * view_matrix * model_matrix[int(matrix_index)] * t_pos;
}


You have to undo the bind position, that's what the inverse bind pose is for, I don't see you actually using it.

Every village needs an idiot Cool
Offline ShadedVertex
« Reply #21 - Posted 2016-07-23 07:45:43 »

I really want to fix this problem, but the hours I've spent scrutinizing my code hasn't paid off one bit. Did you figure anything out on your end?

EDIT: I suggest you check the part where you build the frame skeletons while loading the .md5anim file.
Offline Stranger
« Reply #22 - Posted 2016-07-23 12:36:08 »

Just to ensure we're talking about the same things here:

Skinning equation:

for i to n
v += {[(v * BSM) * IBMi * JMi] * JW}

    n:     The number of joints that influence vertex v
    BSM: Bind-shape matrix
    IBMi: Inverse bind-pose matrix of joint i
    JMi:   Transformation matrix of joint i
    JW:   Weight of the influence of joint i on vertex v

We should precompute (v * BSM) and (IBMi * JMi) since they are constants.

Anton
Offline ShadedVertex
« Reply #23 - Posted 2016-07-24 06:27:17 »

Just to ensure we're talking about the same things here:

Skinning equation:

for i to n
v += {[(v * BSM) * IBMi * JMi] * JW}


Yes, we've already established that.

I really want to find some way to fix this, because skeletal animation is one of the more difficult techniques to understand and it's a lot more difficult to implement. Lots of people who do OGL development and plan to do 3D game development in OGL get discouraged when they get to skeletal animation.
Offline ShadedVertex
« Reply #24 - Posted 2016-07-24 07:14:39 »

I created a GIF of my animation. You can tell it's the Bob model, but it doesn't...look right. Here ya' go:

https://anuj-rao.tinytake.com/sf/ODUxODQxXzM3MDUxMzY

Yeah, sorry, it isn't textured. I just set the colour to be the vertex positions.
Offline basil_

« JGO Bitwise Duke »


Medals: 418
Exp: 13 years



« Reply #25 - Posted 2016-07-24 10:21:50 »

stupid question but ... can this be a matrix column-row major or Y/Z up mix-up ?
Offline Stranger
« Reply #26 - Posted 2016-07-24 10:53:45 »

I forget to say that my code implements the multimesh model animation.

And maybe obsolete  advice from http://www.gamedev.net/topic/665992-collada-animation-leads-to-distorted-model/?view=findpost&p=5212149

Anton
Offline ShadedVertex
« Reply #27 - Posted 2016-07-24 11:43:15 »

stupid question but ... can this be a matrix column-row major or Y/Z up mix-up ?

I'm using JOML, so it's not a column-row major problem. I'm aware that there's a Y/Z up mix-up, I'm fixing that right now.
Pages: [1]
  ignore  |  Print  
 
 

 
Riven (587 views)
2019-09-04 15:33:17

hadezbladez (5528 views)
2018-11-16 13:46:03

hadezbladez (2410 views)
2018-11-16 13:41:33

hadezbladez (5790 views)
2018-11-16 13:35:35

hadezbladez (1232 views)
2018-11-16 13:32:03

EgonOlsen (4669 views)
2018-06-10 19:43:48

EgonOlsen (5687 views)
2018-06-10 19:43:44

EgonOlsen (3205 views)
2018-06-10 19:43:20

DesertCoockie (4104 views)
2018-05-13 18:23:11

nelsongames (5123 views)
2018-04-24 18:15:36
A NON-ideal modular configuration for Eclipse with JavaFX
by philfrei
2019-12-19 19:35:12

Java Gaming Resources
by philfrei
2019-05-14 16:15:13

Deployment and Packaging
by philfrei
2019-05-08 15:15:36

Deployment and Packaging
by philfrei
2019-05-08 15:13:34

Deployment and Packaging
by philfrei
2019-02-17 20:25:53

Deployment and Packaging
by mudlee
2018-08-22 18:09:50

Java Gaming Resources
by gouessej
2018-08-22 08:19:41

Deployment and Packaging
by gouessej
2018-08-22 08:04:08
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!