1) two threads per connection (reader / writer)
2) use the selection api (nio)
I'd pair both solutions with a queue in a producer/consumer pattern.
I heard nio was really difficult (especially as I never programmed anything in the network layer before) but I will have a look at it and see if it can solve the question in one way or another.
I guess I'll have to go down the path of two threads indeed and pass some identifier with the connection, It'll take a wrapper class though and some rethinking of my data structure but it'll save me opening two sockets (which still feels like a good alternative to me :-P
If anyone has anything else to throw in I'd appreciate the input, otherwise it's time for me to start coding

Thanks to Matzon and Nate!

Kind regards,
Mike