two-way UDP from behind a NAT firewall

[William Denniss]

OK, so TCP works very nicely with NAT.  You simply forward the ports on the server, and your clients can all establish outgoing connections fine.

With my game, I have both TCP and two-way UDP communication.

My idea is to connect with TCP, then open a UDP port on each end, exchanging the port's via TCP.  Then each computer would try to send UDP datagrams to the other.

This works great in a LAN.

But, I am having problems when trying to comminicate from one NAT'd LAN to another NAT'd LAN (over the internet).  I had hoped that the NAT router would see the outgoing UDP packet and allow incoming connections on the same port.  But this doesn't seem to hold up.

How then can things like DNS queries work?  I send a request from inside a LAN behind a NAT firewall to a DNS server on port 53.  Presumably, it replies with a datagram to the port I used, and the NAT router knows where to send the packet.

I am dearly hoping not to have to tell my users to start forwarding ports

Does anyone have experiance dealing with this?  I would love to know how you did it.

Thanks,

Will.

[kevglass]

The method you described worked fine for me, sending out a UDP packet causing the router to then accept packets and forward on the way back in, when I tried it a while ago.

However, the routers did seem to timeout pretty quickly so I had to keep a ping UDP packet going to keep the forwarding open.

Kev

[William Denniss]

Kev,

Thanks for shaing your experiance, I'm glad you found that it did work.  I shall have to investigate further.

Do you know how long approximately the firewall kept the window open?

Thanks,

Will.

Games published by our own members! Check 'em out!

[kevglass]

Sorry, can't remember the details it was such a long time ago (with my memory, it could have been yesterday). Endolf might remember more.

Kev

[Jeff]

The problem here is that such behavior is highly fire-wall dependant.

About the only generalization is that newer firewall tend to be better at such inutitive port-forwarding, older fire-walls won't do it at all :/

If this is for a game I strongly suggest you include a readme with "ports to map" instructions in case their firewall is too conservative.  Its not a great solution but its what the industry seems to be doing right now.

[colesbury]

I use the method you describe in my game and it seems to work with all the routers I have tested it with.  Some firewalls will block UDP, but there is not much you can do about that.

One thing you have to remember is that when the server in the middle sends the WAN address when the clients have different WAN addresses (ie. they are behind different routers) and the LAN address when they share the same WAN address.  Otherwise you run into problems.

I'd send a packet about once a minute (or 30 seconds to be on the safe side) to keep the entries in the router's table alive.

The biggest pain I have had with this method is creating a TCP like protocol over UDP so that the clients can exchange data reliably without sending it through the central server.

-Sam

[Jeff]

The biggest pain I have had with this method is creating a TCP like protocol over UDP so that the clients can exchange data reliably without sending it through the central server.

-Sam

Wheveer I read somnethign like this I find myself blinking twice to make sure I read it right.

Why go through all the trouble of re-inventing TCP if thats what you need.  There are all kinds of good reasons not to and i have never seen a defensible reason why one should.

With one exception.  IF your data stream is entirely predictable into the future, AND you want to experiment with foward-error-correction, there might be a reason to build your own protocol.  But I know of no game that meets those criteria.

[colesbury]

Why go through all the trouble of re-inventing TCP if thats what you need.  There are all kinds of good reasons not to and i have never seen a defensible reason why one should.

I had very little bandwidth on the server I was using and needed to exchange information directly between the clients.  Many internet users are behind routers and the only way for NAT'd computers to directly communicate (as far as I know) is by NAT traversal w/ UDP

It was not for an increase in speed or decrease in overhead, as my own method is slower and less efficient than TCP.

-Sam

[William Denniss]

Interesting.  I read about that technique here as well:  http://alumnus.caltech.edu/~dank/peer-nat.html

I have a feeling I know what's going on.  I'm using the UDP port numbers reported by java running on the local machine.  It is highly likely that the NAT router is actually changing those port numbers.

The way around this is that the server must operate on a fixed UDP port (either forwarded, or not behind NAT).  This is what I will try next.  My first solution was more elegant, but couldn't cope with NAT.

In my mind, it has to work, otherwise UDP communication would simply not be possible from behind NAT!  I really don't want to tell users who are simply joining games that they have to forward ports.  People creating games will have to do this, but that's OK.

Will.

[tom]

I'm planing on implementing nat punch-through, but have not done it yet. Here are some links I've found:
http://www.mindcontrol.org/~hplus/nat-punch.html
http://www.codewhore.com/ (links at the bottom of page)

Games published by our own members! Check 'em out!

[kevglass]

Reading the pages it sounds like the exact same through only with a "punchier name" <-- (see what I did there ).

Kev

[blahblahblahh]

I'm planing on implementing nat punch-through, but have not done it yet. Here are some links I've found:
http://www.mindcontrol.org/~hplus/nat-punch.html

Of the top of my head, that's the one that he cleaned up, embellished, and put in the networking section of Game Programming Gems 5.

By co-incidence, I read got around to reading it just two days ago , and I strongly recommend getting hold of a copy of GPG5 for this - it's well explained (although you'll want to skip the first 4 pages as trivial) and covers the main points. Although towards the end I detected author fatigue, as he comes close to glossing over the final questions where he *could* have added some more detail.

But I'm pretty confident it says everything you ever need to know to get this working.

EDIT: the author-fatigue comment isn't a negative on the article, BTW. It's a good article. Read it!

[kevglass]

Someone managed to write *more* than 4 pages about this subject!?

* Lookup Server
* Send UDP both ways until signalled reception
* Keep alive.

I'm tempted to buy the book just to see how it was stretched out?

Kev

EDIT: Big headed/Conceit check, present.. noted.

[blahblahblahh]

Someone managed to write *more* than 4 pages about this subject!?

Off the top of my head:
4 pages: explain "this internet thing" and IP addresses (!)
2 pages: explain "clients and servers have to talk"
2 pages: firewalls and NAT, and the addressing problem (though it doesn't use the official name, IIRC, which I can't remember right now anyway )
3 pages: explain what you try but doesn't work + why
2 pages: explain what does work + why

does NOT go into detail about routers that are bitches. Leaves you with "and some routers will just not work at all, you have to use your server as a router". What would have made it excellent IMHO would have been to , at the very end, give some useful detail on doing efficient / effective routing via your server.

If I were being cynical, I'd say the author was leaving that open so that he has an easy "followup" gem to write next year . But I'm sure that's not the case.

I'm tempted to buy the book just to see how it was stretched out?

you mean you haven't bought it already to read Secure by Design?

[kevglass]

Not too keen on books, its the weight of all that paper

Kev

[Jeff]

In my mind, it has to work, otherwise UDP communication would simply not be possible from behind NAT!

Sorry to break it to you but in many cases it isnt.

Thats why just about every online game has a list of ports to map in case your firewall is blocking things.

[William Denniss]

Thanks everyone for your helpful replies.

Jeff thanks, in those cases then they will have to foward ports, nothing I can do about that.  Fortunately I think they are the minority.  

I did get it working.  My suspicion was correct, the NAT firewall was changing the ports.  Why they would do this unless the port was already in use, I don't know.  I was communicating the results from DatagramChannel.socket().getLocalPort() down the TCP channel.  Now I have an open UDP port on the server (either not behind nat, or fowarded though the NAT) so I can get packets from the client to discover the real client port.  This is a compromise as I thought it was nicer to have one UDP port per connected client, but it can't be helped.

I noticed that some newer routers can have their port forwarding configured by third-party softwre using UPnP.  Having this abillity would be great because even game host's wouldn't need to forward ports.

Has anyone here had experiance with that or know of some good articles?  I know Azureus does it, perhaps I can review their source.

Cheers,

Will.

[blahblahblahh]

Jeff thanks, in those cases then they will have to foward ports, nothing I can do about that.  Fortunately I think they are the minority.  

Or you act as a 3rd party remote packet router...

I did get it working.  My suspicion was correct, the NAT firewall was changing the ports.  Why they would do this unless the port was already in use, I don't know.

I can think of several practical reasons to do with simplicity of implementation. As noted, the GPG5 gem glosses over these issues at the end, merely claiming "it's obviously easier to keep the same port, so everyone ought to do it" - which isn't true; it *probably* is easier, dependending upon your implementation. And cheap routers aren't known for having large dev teams with lots of time and expertise .

I noticed that some newer routers can have their port forwarding configured by third-party softwre using UPnP.  Having this abillity would be great because even game host's wouldn't need to forward ports.

Has anyone here had experiance with that or know of some good articles?  I know Azureus does it, perhaps I can review their source.

try the linux routing/firewall docs resources (howtos, manpages) and the linux kernel stuff. In order to explain to admins how to use and control it, you get quite a lot of info that's helpful to understand how it works.

Or look for docs from the higher-cost router manufacturers, free on their websites?

[Jeff]

I noticed that some newer routers can have their port forwarding configured by third-party softwre using UPnP.  Having this abillity would be great because even game host's wouldn't need to forward ports.

AIUI Most people at all concerned with security disable UPnP on their router.  Apparently (suroprise surprise) Micrsoft didnt think the security model through very well.