Yes it sounds like you are on the right track.
Assuming you are using java sockets, the player acting as the server will create a ServerSocket on the desired port. The player acting as the client then connects using a Socket to the server's IP and port.
If the player acting as a server is sitting behind a router however, they will more than likely have to enable port-forwarding for the particular port that your game is running on. Although this would be up to the player to sort out, as the method varies for different routers. I'm not sure if there is any way around this; if there is perhaps someone more experienced than me can fill you in.
EDIT: (I always edit my posts...

) Just food for thought, if you are still learning the ropes when it comes to networking, perhaps you could try create a simple chat application or similar for starters? Much of the networking theory would be the same, well at least in terms of clients connecting to servers etc. That way you can focus on the networking side of things, instead of trying to build a game and learn networking at the same time. I don't have any idea of your experience or anything, so sorry if it sounds like I'm trying to teach you to suck eggs!!

nerb.