Combining TCP & UDP

[bitshit]

As most games have good use for both reliable and unreliable packet sending, why not opening a TCP socket for sending reliable messages and using UDP for sending quick frequent position updates etc?

Has anyone expiriented with this? Are there good reasons not to do this?

[Riven]

Clients with a LAN have to manually setup the router to do port-forwarding or setup virtual servers for UDP to come through.
The clients can send UDP packets to the server, but the server can't reach the client behind a router.

That's my main problem with UDP, unless somebody figured out how to work around this problem... anyone?

[kevglass]

Yeah, isn't there a technique called UDP punch through where you first send a UDP packet to the destination. This sets up on "most" routers a forwarding path which allows packets recieved from the destination address to get back through to the source address. If you want to go direct peer to peer you set up a lobby server in the middle that you communicate with using TCP to establish the UDP ports between the players.

Actually, TCP is the real problem if you wanted to be able to allow hosting on client machines. The host has to set up a forward through from the outside world to the internal machine. At least its only the game host who has to worry I suppose.

Kev

Games published by our own members! Check 'em out!

[OverKill]

Currently working on this myself.

I had hoped to open connections similar to how we do it with sockets but have had no success.
(is it even possible)

My biggest problem is how to send data to the client without the client having to open up ports.
Since the client will only recieve stuff.

[bitshit]

Clients with a LAN have to manually setup the router to do port-forwarding or setup virtual servers for UDP to come through.
The clients can send UDP packets to the server, but the server can't reach the client behind a router.

I don't see why this should be a problem, assuming that the unreliable messages don't need to get ack'ed... But maybe there are other pitfalls?

[Riven]

Eh... the router doesn't know to who (of the clients on the LAN) to send the UDP packet, and drops it.

But maybe, as Kev pointed out we've got smart routers these days. I'll give it a try.

[bitshit]

I wasn't talking about p2p games perse, for a client/server model this wouldn't be a issue i think?

[Riven]

It is an issue. P2P only makes the problem worse, that doesn't mean there is no problem in a server/client setup.

Unless Kev is right, ofcourse, but I already said that.

[bitshit]

Ah yes you're talking about games within the same LAN... I was thinking about a client/server model over the internet... that wouldn't cause any problems sending your UDP packets to.

[kevglass]

Either way data has to get back and forth between client computers. Its all very well a client being able to get data from the client through the router to the server to set up a game, but without being able to route UDP back through to the client the server can't tell client A that client B has connected  (or for that matter, moved, fired, jumped up and down like a wild thing).

Kev

Games published by our own members! Check 'em out!

[Riven]

Ah yes you're talking about games within the same LAN... I was thinking about a client/server model over the internet... that wouldn't cause any problems sending your UDP packets to.

No no no.

No.


Kev gave an excellent explaination and I hope you understand it, otherwise, try to re-read this thread, and make sure you understand every single word.



We are talking about this situation:

  client A  --->  [   LAN   ]
  client B  --->  [      -     ]   -----> [   ISP   ]  -----> .....  ------> [ SERVER ]
  client C  --->  [  router  ]

Client A can send UDP to server without problems, but server can't reach client A because the LAN-router doesn't know who to send the packet: client A, B or C, as the server only sees the public IP (of the LAN-router) and not the local IP of client A.

[bitshit]

I stand corrected.

But back to my original question, when this UDP punch through works well enough, would it be a good idea to listen / send reliable messages with TCP and send/receive quick/unreliable messages with UDP?

Or another alternative, has anyone have expirience using jEnet in their game?

[tom]

Using both TCP and UDP will work fine. Not done it myselft but I'm it has been done by many games in the past.

Riven: I think you are wrong. The router/NAT will remember messages fram client to server and correcly rout the messages back again. The only problem is that it is impossible to connect to a client from outside.

[kevglass]

I think you'll find jEnet uses TCP and/or UDP underneath.

But back to my original question, when this UDP punch through works well enough, would it be a good idea to listen / send reliable messages with TCP and send/receive quick/unreliable messages with UDP?

Yeah, if you know that the messages have to be reliable and in-order its best to use TCP. In this case you can just get your clients to connect out through the router to the server to the ServerSocket running there. Once that stream is established you can send data back and forth on it.

Kev

[bitshit]

Yeah, if you know that the messages have to be reliable and in-order its best to use TCP. In this case you can just get your clients to connect out through the router to the server to the ServerSocket running there. Once that stream is established you can send data back and forth on it.

Actually I was thinking about using a TCP stream and UDP datagrams concurently. So certain messages could be send with UDP and other critical messages with TCP

Using both TCP and UDP will work fine. Not done it myselft but I'm it has been done by many games in the past.

Yes, but I remember reading about some potential problems you could run into... somewhere in a forum post on gamedev.net

I think you'll find jEnet uses TCP and/or UDP underneath.

jEnet is the port of Enet to Java and is a reliable protocol on top of UDP. This isn't the same as TCP, because now you can tell certain packets to be develivered reliable, in order or unreliable depending the need. The advantage of jEnet would be to stick to one protocol (udp) and being able to send all type of messages with it (unreliable, reliable and in order).

But it doesn't seemed to be used in any game yet, so I thought maybe there's a reason for that as the jEnet port sounds prety complete...

[Riven]

Riven: I think you are wrong. The router/NAT will remember messages fram client to server and correcly rout the messages back again. The only problem is that it is impossible to connect to a client from outside.

Well, not very wrong I said it would be the case, unless Kev was right and routers had become smart. Seems they did. Hurray

[Jeff]

On a totally different subjec relating to TCP and UDPt...

You can actually user UDP to reduce TCP latencies.  It was a trick we came up with abck at TEN.  (Well Bill Lipa came up with it, to give proepr credit.)  What causes latency spikes in the TCP flow is when you lose a TCP packet.  The entire stream has to stop and wait for the resend of that apcket because TCP gaurantees in-order delivery.

UDP however has no garauntees.  Thus it never "jams up" this way.  What we did was to peridically send a "frame" of the last N TCP packets over a secondary UDP channel.  These could "fill in" for the missing TCP packets while TCP was figuring itself out just so long as the UDP packets got through.

It drastically reduced our fequency of latency spikes.

Its the best of both worlds for reliable delivery IF you can afford the extra bandwidth.

[Mr_Light]

and with the current lines, f**k bandwidth. 

[blahblahblahh]

Using both TCP and UDP will work fine. Not done it myselft but I'm it has been done by many games in the past.

Yes, but I remember reading about some potential problems you could run into... somewhere in a forum post on gamedev.net

Generally, if you need UDP, you sooner or later want TCP-without-the-"always-on"-in-order-delivery, at which point you write that on UDP (or use a library, if you're smart), and suddenly you have no need for TCP at all.

With a hybrid UDP/TCP you often get problems with synching between the two layers, get the performance of the worst performing, and voila you have a completely useless setup. If you are absolutely sure there is no interrelationship between the layers (e.g. one
is an out of game chat channel) then you are fine.

[Jeff]

Using both TCP and UDP will work fine. Not done it myselft but I'm it has been done by many games in the past.

Yes, but I remember reading about some potential problems you could run into... somewhere in a forum post on gamedev.net

Generally, if you need UDP, you sooner or later want TCP-without-the-"always-on"-in-order-delivery, at which point you write that on UDP (or use a library, if you're smart), and suddenly you have no need for TCP at all.

I would disagree on two grounds:

(1) If you are going to connect over analog lines, thanks to PPP TCP compresses across the critical last mile much tiger. (2 bytes per packet TCP overhead as opposed to 30 per UDP packet.)

(2) The entire infrastructure of the net "undersatnds" TCP on the system level. You really can't accomplish the same exact thing as efficiently in user space.

With a hybrid UDP/TCP you often get problems with synching between the two layers, get the performance of the worst performing, and voila you have a completely useless setup. If you are absolutely sure there is no interrelationship between the layers (e.g. one
is an out of game chat channel) then you are fine.

[chaosdeathfish]

  client A  --->  [   LAN   ]
  client B  --->  [      -     ]   -----> [   ISP   ]  -----> .....  ------> [ SERVER ]
  client C  --->  [  router  ]

Client A can send UDP to server without problems, but server can't reach client A because the LAN-router doesn't know who to send the packet: client A, B or C, as the server only sees the public IP (of the LAN-router) and not the local IP of client A.

Unless you use uPnP, which most routers I've come across recently support.

Speaking of which, does anyone know of a decent uPnP library for Java? (Just the Internet Gateway Device profile, not the rest of it). I was thinking of writing one, but the uPnP specs seemed designed to confuse me.

[swpalmer]

Speaking of which, does anyone know of a decent uPnP library for Java? (Just the Internet Gateway Device profile, not the rest of it). I was thinking of writing one, but the uPnP specs seemed designed to confuse me.

At one point I was toying with the idea of ripping the uPnP support from the Azureus bittorrent client... it is at least a reference implementation that you might learn the protocol from.

[chaosdeathfish]

I've actually found the relevant part of the uPnP spec now! Unfortunately I haven't got the time to do a proper IGD implementation. I've looked at cyberlink uPnP for Java (which is BSD license), which provides the framework but no profiles. It uses its own HTTP implementation, which seems a little odd - although it is ported from a C++ version I think.

Anyone feel like writing an IGD implementation?

[Jeff]

Problem with UPnP is there are a significant number of people (I'm one of them) who see it as a MAJOR security risk and disable it on their routers.

A secure, controllable way for software to explain to rouerts thier needs woudl be great.  Unfortunately UPnP is neither.

[chaosdeathfish]

OK, I had a sudden urge to write a UPnP control point in Java. It still needs some work, but it can do port forwarding. And in theory it can control almost any other UPnP device, but I've written a simple wrapper that makes port forwarding easy.

Hopefully I can get it a bit more polished and release it for general consumption sometime in the next few days!