Mapping a 2d point into 3d space

[~Spaceaholic]

Hi everyone, I am taking the summer to try and develop a game for the android operating systems... I am not entirely new to programming, I just finished my bachelors in computer science, but I still have much learning to do...
My Problem, I want to map user touches on the screen of my phone into my 3d world drawn with opengl... I have a plane that lies perpendicular to the y axis which is my floor and a bunch of planes perpendicular to the floor which represent buildings. i need to determine if the user touched the floor or a building, if they touched the floor then they are going to be panning through the world, but if they clicked on a building they get an action menu for that building menu... the menu and panning are not the issue but determining where they have touched is...

the screen registers touches as an x and y coordinate increasing x across to the right and increasing y down towards the bottom of the screen...

Thanks for your time and help,
~Spaceaholic

[Riven]

The easiest way is color coding, which gives you pixel-perfect accuracy (at the expense of fillrate):

1. Clear the framebuffer
2. Render every object in another color:
      Render the floor blue
      Render building #1 red
      Render building #2 green
      Render building #3 yellow
      ... etc, etc (there are enough colors for a megacity)
3. Grab the pixel from the framebuffer at (xMouse,yMouse) and look at its color, now you know which object the player clicked on
4. Clear the framebuffer
5. Draw the scene like usual
6. Update the screen

(nobody ever saw the colored objects)


It does not address your specific question at all ('mapping a 2d point into 3d space') but it solves your problem.

[lhkbob]

A more complicated way that requires more knowledge of how 3D works is:

1. There is a projection frustum that represents the camera and creates the perspective transformation. This has a near and far plane that are perpendicular to the direction the camera looks at.  A user's press on the touch screen can be mapped to a point on the near plane.
2. Create a ray from the camera's location to the point on the near plane.
3. Intersect this ray with the scene to see what it intersects with.

This is definitely harder, but has the benefit of giving you the depth coordinate of their click/press.

Games published by our own members! Check 'em out!

[~Spaceaholic]

Option number 1, rendering every object in a different color sounds like it would take more time, I am running on a phone with limited processing power.. option number two creating a ray out of the coordinates and the viewing frustum sounds more like what I want(assuming its less processing time)....

So how would one take the coordinate from the touch and map it to the frustum? the touch events coordinates are on a coordinate axis with x increasing left to right and y increasing top to bottom, my coordinate space, at least as I am familiar with it has a y increasing bottom to top and has negative values...

also how would I go about making this ray and intersecting it with objects within the space? could you point me in a direction where I could see examples?

Thanks for your help,
~Spaceaholic

[Eli Delventhal]

Option number 1, rendering every object in a different color sounds like it would take more time, I am running on a phone with limited processing power.. option number two creating a ray out of the coordinates and the viewing frustum sounds more like what I want(assuming its less processing time)....

So how would one take the coordinate from the touch and map it to the frustum? the touch events coordinates are on a coordinate axis with x increasing left to right and y increasing top to bottom, my coordinate space, at least as I am familiar with it has a y increasing bottom to top and has negative values...

also how would I go about making this ray and intersecting it with objects within the space? could you point me in a direction where I could see examples?

Thanks for your help,
~Spaceaholic

Huh, cool idea, Riven.

If everything you have is simply shaped then option 1 should be fine for you. Rendering to an FBO is pretty cheap if you never render it to the screen. If you're doing a whole lot of rendering and pushing things to the limit then this may not be the case, but you can do things like color render bounding boxes to keep it simple and relatively accurate. Remember there is a reasonable overhead to raycasting as well, and it's a massive amount of work to do properly. Also remember the very important fact that you don't need to care at all about the entire screen, you only need to care about the single pixel where the user is touching. That means you can make a really really tiny FBO (even like 8x8 or something), then use bounding boxes to decide what to draw onto the FBO, and this approach will be lightning fast.

I also know of a very high-profile incredibly successful iPhone game (not one of my company's) that renders every single character to an FBO before drawing it to the screen. That's like 20 or 30 128x128 FBOs per frame sometimes, and these are trashed and recreated repeatedly.

I'm not advocating for Riven's method, I'm just saying don't rule it out for performance reasons. In 99% of cases it should be just fine, as long as you're not trying to deploy on iPhone 3G or the Hero or some other old shitty devices. And if you're doing that then your game is going to choke anyway.

[~Spaceaholic]

Ok so sounds like option one wins the day, I have a tendency to over complicate things...
The only clickable options to the world are the floors for moving around and the buildings, which like i said get a menu... I was entertaining the possibility of placing a highlight point on the floor were the user was currently putting there finger, but that is trivial at this point...

So what I have is a floor made up of a single plane at the moment but soon to be multiple tile...
and a number of buildings that are like I said perpendicular to it.
Then there are 3 or four other characters that get drawn to the screen many times per frame, but since they are not clickable they dont have to be rendered in the colors..

How would I would I go about implementing the multi colored render?
Thanks,
            ~Spaceaholic

[Riven]

How would I would I go about implementing the multi colored render?

Give all your items an ID (integer), you can assign it, using a static counter in the constructor of some superclass, start with value 1.

1   2   3   4   5   6   7   8   9   10   11

public class RenderItem {      private static int ID_COUNTER = 1;      private final int id;      public RenderItem()      {          this.id = ID_COUNTER++;      } }

To render each item, split the ID into RGB channels:

1   2   3

int id = item.getID(); glColor3ub((id >> 16) & 0xFF, (id >> 8) & 0xFF, (id >> 0) & 0xFF); item.render();

The reason you start with 0, is because that's black, and will be the color the framebuffer is cleared with, which is an indication nothing was found under that pixel.

[Eli Delventhal]

Riven, if you have anti-aliasing or something like, is there the possibility of getting in between colors with this method, or half-alpha colors?

[~Spaceaholic]

Just had a thought, in my game the user is going to be transferring men from building to building a lot, I want to be able to have a user touchdown a specified building and drag towards other buildings, animating an arrow to the closest building of there current touch, this will allow for smoother game play as the user wont get frustrated that they have to touch up on the exact building but can just get close....
Is there any way I could implement something like this using the colored render?
Thanks
~Spaceaholic