a small probelm with picking

[SamJ]

Hi, I've got a picking algorithm implemented that is essentially the same as the example by Thomas Bladh(http://www.sm.luth.se/~david/classes/smm009/TestPrograms/Picking.java). The only difference is in how I process hits, and the fact that I have to pass an IntBuffer with glSelectBuffer(), instead of an int array. The method for processing the hits reads through the returned IntBuffer and outputs the contents on the screen. It also displays the number of reported hits. The problem appears to be with either accessing the IntBuffer's contents or with glSelectBuffer recording the wrong information to the buffer. When an object is clicked, the console presents the correct number of hits and then prints out strange numbers for the values on the hit list. For example, output for the second object might be something like:
"number of hits: 1
hits: 2  -141016064  -118963712  1  2  0  0  0 0 0 0" with the rest of the intbuffer being empty. I would expect output along the lines of:
"number of hits: 1
hits: 2  0  0  0  0  0  0  0 0 0 0"
The hit list gets even weirder when multiple hits are recorded.
Is there something obvious that I'm doing wrong? I think it might have something to do with unsigned ints but I really have no idea.
I'm running all of this on mac os x with a JOGL  release from over the summer.
Any help would be greatly appreciated.

[Ken Russell]

The strange-looking values are the scaled Z-coordinates of your hits expressed as unsigned integer values. Thomas Bladh's example converts the hits back to floating-point depth values, but I'm not 100% sure the conversion is correct. You might need to do something like this:

1	float depth = ((float) (buffer.get(offset) & 0x00000000FFFFFFFFL) / 0x00000000FFFFFFFFL));

[SamJ]

Thank you, that was very helpful.