Understanding the physical properties is just the beginning. TCP/IP is a many-layered beast -- 7 layers to be precise, by the most common (but not neccessarily most useful) reckoning, called the OSI layers.
The first thing to understand is that both Ethernet and IP are packet-based protocols, not byte-based. That is, the bytes get packaged up into bundles called packets, a header is tacked onto the beginning of each packet, and the packet moves as one piece across the network. Of course, at lower levels, packets are just sequences of bytes, and the physical Ethernet transport takes care of pushing bits.
Looking at it from the top down, if you want to send a single byte over a TCP/IP connection (and we'll assume the connection is already established, which is another thing entirely), that byte first gets wrapped into a TCP packet, which has a TCP header, and outside that there is an IP header, and there you have an IP packet. Then that packet gets wrapped inside an Ethernet packet, which has its own header, and the Ethernet packet is sent, as a whole string of bytes, over the physical Ethernet interface. Because of the fact that Ethernet is a shared medium, with many transmitters, the packet might need to be retransmitted once or twice if there were other nodes trying to transmit at the same time (this is called a collision). Eventually, the packet will get out, and be reconstructed on the other end by the destination node, which will unwrap the Ethernet packet to get at the IP packet to get at the TCP packet to get at your byte. It will then send an acknowledgement packet (Again, TCP wrapped in IP wrapped in Ethernet) to let you know that the packet arrived safe and sound.
(Technically, TCP packets are not really wrapped in IP packets -- it is only one packet, but it has both an IP header and a TCP header. But the TCP/IP packet is truly wrapped inside the Ethernet packet, which knows nothing about IP and just shuffles the IP packet around as a string of bytes.)
That explanation should serve as an indication that you need to do some serious reading if you're going to wrap your mind around all this. There are many books on the subject -- TCP/IP Illustrated comes to mind. Google might also turn some stuff up, and try wikipedia.org too.
A quick fix would be to get one of the many serial-to-IP (aka serial-to-Ethernet) converters out there, which look just like a serial port to your microcontroller, but they wrap your bytes up in packets and send them over a TCP/IP connection, so that they can be reconstituted on the other end by a similar device, or by software running on a PC which emulates a serial port. The Xport
and Digi Connect ME
are two examples (weighing in at about $50/unit, depending on quantity), but there are many more.
It should also be noted that you can use Ethernet without IP, but that's generally only done for certain specialized applications.
Telnet transfers the key hits and display characters with a TCP connection. There are also some control bytes in the Telnet protocol itself (0xf0 - 0xff) which are used to control the transfer. For details, read the relevant RFCs.
A key hit is often transferred alone in a TCP segment. This needs at least 64 bytes on the Ethernet cable. There is good sense to buffer the display output into larger chunks.
One is Jeremy Bentham's TCP/IP Lean project, see .
First, let's get some terminology straight. Ethernet has frames. You put an IP packet (sometimes called a datagram) in an Ethernet frame. You put a TCP segment inside an IP packet. You put chunks of your data inside the TCP segments. Of course if your application is using UDP instead of then you have UPD datagrams inside the IP packet, and your data inside the UDP datagram.
The terminology is not that big a deal, and the words frame and packet are quite commonly misused. Once you get a handle on this stuff you'll pretty automatically figure out which one is really meant, but as your just getting started, knowing the real terms will probably help.
Second, don't try to understand TCP/IP in terms of the seven-layer OSI model. TCP predates the OSI model, and despite repeated attempts to mash it into that model, fits rather poorly. Nor do any of the specs actually match up with the OSI model. In short, TCP/IP has about three layers in the protocol, plus your application on top, plus the physical layer underneath.
where an IP packet is somehow moved across a communications link. Each type of LAN (Ethernet, token-ring, FDDI, etc.) for example, has a specification for how that takes place. In the case of Ethernet, the specification spells out how to set up the Ethernet headers, maximum frame sizes, and how one IP host can find another on the same LAN. A quite similar specification exists for Token-Ring LANs (although there are some differences). When you use a dial-up connection, the link-layer protocol is usually something called PPP - which is the specification of how to move an IP packet across a serial link. Obviously and two nodes that want to talk across some communications link need to use the same mapping.
Now below the link layer is the physical layer. Somewhat loosely, your application will care little about the physical layer, except, of course, that the correct hardware has to be there. For example, Ethernet comes in 1*, 10, 100, 1000, and 10,000Mbps versions, and runs over several types of coaxial cable, twisted pair, several varieties of optical fiber, and even various wireless links at times. The layer exists, but doesn't usually impact your software stack very much.
The link layer in a real implementation is usually split into an upper and lower half. The upper half does the specified mapping (say IP packet into an Ethernet frame), and the lower part drives the physical I/O device (the Ethernet chip, for example). An OS like Windows or Linux typically separates the two pieces (eg. the IP-to-Ethernet layer comes from Microsoft, and the lower part is the "device driver" you get from the network card manufacturer). An embedded system often has those two pieces closely coupled.
The middle layer is the IP packet. The IP packet has a relatively simple header that most notably specifies the source and destination IP addresses, as well as the type of protocol contained within the IP packet. While there are some complexities (options and fragmentation for example), most of those can be ignored and then there's really darn little in the IP header. All TCP/IP hosts (any machine with a TCP/IP stack is a "host") exchange IP packets across the Internet. That *all* they do. All data communications breaks down to a host building an IP packet, and then sending to either a directly connected neighbor, or a directly connected router. It never sees it again.
The upper layer is the transport protocol. In TCP/IP the two most common transport protocols are TCP and UDP. UDP is a simple datagram based protocol, which provides no delivery guarantees. In fact, UPD does little but add a couple of port numbers in the UDP header so that traffic to multiple applications on a single host can be sorted out. UDP is very simple. You put some number of bytes (up to about 64KB) in a UDP packet, toss it out the back door, and it may show up at the destination application. Usually UDP applications limit themselves to much smaller chucks of data (512 bytes is a good starting limit). If your application can tolerate packet loss, or has some sort of built in retry mechanism (for example DNS), then UDP is quite simple to use.
If, OTOH, you need reliability, and want a connection oriented view of the world, TCP is usually a better choice. TCP again lives inside IP packets, but the two applications set up a connection (virtual circuit) between them by sending some packets containing TCP handshaking and negotiation stuff back and forth (of course the TCP software in the OS does all the real work here). Once established, the applications send two streams of bytes over the link, one in each direction. The two streams are largely unrelated to each other, and TCPs job is to package chunks of those streams into TCP segments and send those off in IP packets. TCP is also responsible for making sure the stream arrives in the same condition it was sent, and it handles missing packets, duplicated packets, reordered packet, etc., so the receiving application just sees the same stream of bytes as the sending application sent. The exact details of how the application data is packaged is largely hidden from the application, and the TCP software might send a stream of 10,000 bytes in 10,000 individual Ethernet frames containing one byte of application data each, or might send seven ~1500 byte frames. Typically you can trust TCP to send large (and more efficient) segments whenever possible.
Now on top of that is the application. That's sometime called a layer, and sometimes not (BTW, folks who count both the physical and application layers usually call TCP/IP a five layer protocol). This is where the end user stars to see stuff. For example, HTTP (the protocol use for transferring pages between a web server and a web browser) plus HTML (how those pages are coded) make up what we all know as the web. FTP is another common application, as is Telnet.
Telnet is a good example for your application. In short Telnet is a specification of how you stuff a serial terminal datastream inside a TCP connection. There's some negotiation (the terminal tells the host what type of terminal it is), and some minor encapsulation (escapes for the telnet commands), but one the session is running, you're pretty much looking at the conventional terminal datastream running over the TCP connection, essentially identical to what you'd see over a serial attached terminal. In fact if you're running a TTY style device, you can skip (or just say "no") to most of the negotiations.
Assuming, for example, that your device supports attaching a VT-100 terminal (obviously emulated these days) via a serial port for whatever control purposes you have, making your device a Telnet server might be just the ticket. Then anyone with a Telnet client (that supports VT-100), will be able to access your device a do whatever it is they'd do with your device.
On the flip side, if your device is generating some data which you're sending over a serial link to some PC that's processing it in some way, then it might make more sense to just set up a TCP connection, and then just send the bytes as-is. That's not all that much different than what a Telnet connection would do, but you'd get to avoid some of the negotiation that's required in Telnet.
But the critical point is that that you can treat a TCP connection like a stream of bytes, very, very similar to what you'd see between two devices connected via their serial ports. What happens underneath is mostly invisible to your application.
On the other hand, TCP is a good sized chuck of code, and perhaps stuffing your data into UDP datagrams would work. It's especially attractive if you have messages of at least several bytes (one byte at a time is possible, but somewhat painful), and you can tolerate losing messages on occasion. Then you'd just set up the target PC to listen for incoming UDP datagrams on some predefined port, and as you got them, you'd process the data (messages) in them.
If you want to get more modern about it, you might want to set up your device as a web server. As an aside, if you're doing this over the Internet, most people's firewall policies make it easiest to use HTTP (web service), and TCP connections are easier than UDP.
OK, so now how do you get started? First, get either the TCP/IP "Illustrated" series (first two volumes) by Stevens, or the "Internetworking with TCP/IP" (three volumes) by Comer and Stevens. No need to get both. That'll teach you everything you need to know to both write basic applications using TCP/IP, and enough about the internals of a TCP/IP implementation, so that you'd probably be able to dig into the RFCs and actually implement a stack if you were so moved.
Second, get "TCP/IP Lean" by Bentham. He's got an entire TCP/IP stack, including a web server, running on a PIC!
Third, get a network monitor type (sometimes called sniffers**) package and *watch* some of this traffic. Fire up your web browser, your FTP client, Telnet to a host. There are several shareware and demo versions available for Windows (if you've got Windows Server, MS's "Network Monitor" comes with it), and several network monitors are included in your favorite Linux distribution.
I cannot emphasize this enough - get a network monitor. Not only is it a wonderful learning tool, it's an invaluable debugging tool for when you're trying to figure out what your device is actually doing when your PC application isn't seeing anything. All of these will do a nice job decoding the Ethernet and TCP/IP headers, so it's pretty easy to follow what's going on.
Finally, look at some of the packaged solutions. If you already have a serial port, there are several serial-to-TCP/IP converters on the market. You establish a TCP connection from the PC to that (some are set up as Telnet servers), and you're all set. The down-side is that it's an extra part. Most of the embedded CPU vendors also have versions of their CPUs with embedded Ethernet controllers, and have available TCP/IP stacks so you don't have to write your own.
To give you an idea of the scope, you can probably do a functional UDP based device, from bare metal (eg. the Ethernet controller chip) on up, in about 3000 lines of C code. A full TCP/IP stack is probably
*OK, so no one's seen StarNet in a couple of decades...
**"Sniffer" is actually a trademark of a high end network monitor that's been around for a very long time.
Since you are curious about how it all works, first go here and get a free packet analyzer:
It will both capture the traffic, as well as decode it for easier viewing. Then telnet to route-views.routeviews.org (or your friendly neighborhood Unix system) and see what the traffic looks like as you type each character.
To get an idea of a minimal TCP stack, check out the book TCP/IP Lean by Jeremy Bentham. There are many others online, such as lwIP and uIP.
You'll find that the programming side of TCP is huge, and something running on TCP is usually what you'll want to use. If your goal is just to IP-enable your serial port, there is a whole class of turnkey products targeted at exactly this application.
Digi offers packaged terminal server devices (turns the serial port into a telnet device), as well as the Digi ME embedded product which does that can much more. Lantronix X-Port is another embedded product, as are larger (some cheaper) products by RabbitSemiconductor, NetBurner, and many others.