Don't use blocking IO; use non-blocking Channels along with a Selector.
It should be just as simple to just use a thread-per-client model with blocking IO... you might want to give your reasons for the advice.
My suggestion was responding to this request by the OP.
The simplest solution would be to split the packet-input thread into a TCP-input and UDP-input thread, but I'd want to avoid this if possible.
If he doesn't want blocking, and doesn't want lots of threads, non-blocking Channels accessed through a Selector is the logical suggestion.