First you need to realize everything you send and receive over a connection are in bytes. Secondly you must realize that if everything anyone ever sends to each other is in bytes, i.e. one's and zero's, how can anyone ever make sense of what is being sent? How do we turn these bytes into text or commands etc? This is where "protocols" come in. A protocol is a fancy name for a set of specific rules on how 2 people/programs/parties will interpret the bytes they are sending to each other so that they can make sense of them.
e.g. a protocol might say that "The first 4 bytes will determine the length of the message in bytes, all the other bytes are interpreted as ASCII text until you reach the end of the message and the message ends.".
So in order to make an IRC client - you need to adhere to the IRC protocol. These are often called RFC's with a number after them. "RFC" stands for "Request For Comments". The name comes from when in the early days of network programming - there were no largely used protocols. So people made their own protocols and asked other programmers to comment (suggestions etc).
IRC RFC 1459 http://www.irchelp.org/irchelp/rfc/rfc.html
Here's a key quote from the RFC above to let you know what you're dealing with.
The IRC protocol is a text-based protocol, with the simplest client being any socket program capable of connecting to the server.
>I'm not seeing a problem here...