The issue here is the bit you said I lost you with. That I think is the source of the problem.
The thing is, you are applying a scale of 3x in the x-axis only to the matrix.
However, the matrix is rotated 90 degrees around the y axis. This means the matrixes x axis now lies along the -z axis in world space, and the matrix z axis points along world space x. i.e. the matrix will be (using rows for axis, which is my personal favourite):
x - ( 0, 0, -1 )
y - ( 0, 1, 0 ) = R, a rotation matrix
z - ( 1, 0, 0 )
You then scale the WORLD z-axis by 3 using a matrix:
(1, 0, 0)
(0, 1, 0) = S, a scale matrix
(0, 0, 3)
This is the scale matrix. Multiplying these (R*S) gives:
(0, 0, -3)
(0, 1, 0)
(1, 0, 0) which looks like what you get.
Note that this is not the same as a LOCAL z axis scale (S * R), which would have given:
(0, 0, -1)
(0, 1, 0)
(3, 0, 0) which is what you seem to be after.
The only difference is how the scale is applied. The first way was post-multiplied, the second was pre-multiplied. This can be seen as 'the order which you apply the matrices', or (from a physics viewpoint) the space in which the scale is applied.
Admittedly, I would side with you on the expected behaviour of a function called 'scale', but my previous point was that conventions differ, and there is no real 'right' way. You just have to deal with the conventions used by the library your using.
The best way to ortho-normalise a matrix is this (in pseudo code):
vector x = matrix.getXaxis();
float xl = x.length();
x /= xl; // normalise
vector y = matrix.getYaxis();
float yl = y.length();
y /= yl; // normalise
vector z = matrix.getZaxis();
float zl = z.length();
// Now build a new z-axis that is orthogonal to x and y:
z.crossProduct( x, y ); // z = x ^ y
// z is now a true z-axis, both 'normal', and orthogonal to x and y.
y.crossProduct( z, x ); // y = z ^ x
// Rebuild the matrix, multiplying scale back in:
matrix.setXaxis( x * xl );
matrix.setYaxis( y * yl );
matrxi.setZaxis( z * zl );
This _should_ be what the normalizeCP() is doing, guessing from the name.
Hope this makes more sense