Ok, I figured it out!!!

I need to find the normal of the two vectors and if the normal points in towards me (i.e. z is negative), then I have the angle I want, if it points away from me (i.e. z is positive) then I need to take 360-angle!!

Here is the updated code:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
| private float getAngle(Point p1, Point p2, Point p3) { float angle = 0, cosAngle = 0; Vector3f A = new Vector3f(p1.x - p2.x, p1.y - p2.y, 0); Vector3f B = new Vector3f(p3.x - p2.x, p3.y - p2.y, 0); Vector3f AxB; cosAngle = Vector3f.dot(A, B) / (A.length() * B.length()); angle = (float)(Math.acos(cosAngle) * (180/Math.PI)); AxB = Vector3f.cross(A, B, null); if(AxB.z > 0) angle = 360f - angle; return angle; } |

Here are some pictures of what it did to different shapes!

This is the original shape that I was having trouble with.

Then I wanted a complicated shape to make sure it wan't going to puke.

And I wanted to make sure that it did something sane on a simple shape.