I have a serial device that is using MCU's UART (rs232) for data transfer, every time my device is suppose to deliver 20 bytes making it a single transaction. We have a VB and C# app using MSCOMM and it works perfectly. Now I have added an Ethernet converter to convert serial data to TCP/ IP packets. It works atleast for half of the time. For the rest of the problematic half the behavior is very strange, yes, my received packets are broken in parts. For example I'll receive 20 bytes in tow transactions like 10 byte for each time. I tried to look at the MCU UART signal via oscilloscope but the timing was quite consistent. Can't understand why packets are broken/scattered, any idea?
You cannot expect TCP to give you pakets on the remote end the way you feed them into the system. That's just by design. TCP is a streaming protocol, and it's up to the senders TCP stack implementation to fill segments to what ever degree seems good to get a good performance. That said, the sending part will most likely try to combine your 20 bytes chunks of data into as few segments as possible cause sending out a segment over the wire is timewise expensive. The Nagle algorithm will delay sending out segments a tad bit to see if more data arrives and if so will concatenate your pakets into a single bigger TCP segment.
So your reciveing end simply must be written so as it keeps on reading from the TCP stream up until you have 20 bytes, and store excess bytes it may have read for the next call. This can easily be implemented with a ring buffer structure and as many calls to recv() as needed.
If you try to circumvent this, you are fighting against TCP and potential oddities/requierements of the unterlaying network along the comunication path.
Hey Markus, Thanks for follow up, yes you are right as far as TCP/IP packet transfer is concern. But i wonder why it works for half of the time? is there any way i can increase the success probability ?
On MCU side i'm not using interrupts to push the data over serial line. Do you think using interrupts can improve the success rate? AFAIK interrupt are more likely non-preemptive .
If you are using TCP, there is NO GUARANTEE that the data is delivered in same size of chunks you feed it in. There's also no guarantee that they are not packed into similar size segments - the segment size is totally at the mercy of the transport layer code, and you should not even attempt to change it. The only guarantee you have with TCP is that all the bytes fed in will eventually arrive in the same order.
If you need to preserve the record boundaries, you can use UDP, but then it's up to you to handle packet drops in the network.
Another possibility is to include packet boundary markers into the TCP data stream and split the incoming flow at these markers.
If you wait long enough after sending your 20 bytes, it's likely that the TCP stack on the sender side get's bored enough to send out the data. But there is absolutely no guarantee that the 20 bytes are still there on the recivers end. That said a gateway along the path could reasemble segments or even more fragment them because the underlaying transport mechanism requiers it. Ok, with only 20 bytes that's quite unlikely but relying on such stuff would be an extremly bad idea.
Only if the timeing gets changed (see above). You could try to turn off Nagle and use IOCTLs to set the PUSH flag but once more there is no guarantee for sucess by doing so.
It's really easy. Call recv() in a loop up until you have 20 bytes (asuming your pakets are always 20 bytes in size, else introduce a marker or lenght information). Store in excess recived bytes for the next call where you would want the next paket and return to the caller.
lost its TCP header, but IP header (with protocol type set to TCP) and TCP payload are still there... We started to see thousands of packets just like this one somewhere in the middle of 2000, coming from many locations in Poland.
Markus! Thanks for valuable input and it was indeed helpful. I really can't understand your Nagle point though i tried to search my hardware vendor provided tcpip library but no luck so far. Anyways, i've tried following approach to get by with broken packets but need to test it tomorrow .
say buffer is char array wiht 128 size.
so the seriali2ethernet s/w should do:
if (serialData < 20) // as desired data is with length of 20 bytes while(buffer != 128 ) { fetch every single byte from serial port and store it in buffer array. and flush the buffer once sentinel (buffer==128) is true. } else send data to connected socket.
i think it would be worth discussing my app architecture as well so here i go:
I have developed a VB demo app which is calling a C dll (dynamic link library in windows world) for tcp/ip connectivity. The dll is mulithreaded asynchronous tcp/ip server. Basically dll is no more than a forwarder module that is getting data from one tcp/ip client and making data available on windows messaging system for registered (proc hooking stuff) apps. (Using windows messaging system for passing data through dll to VB app) .
The converter (serial to ethernet and ethernet to serial) software is acting just like a pipe while converting serial requests to tcp/ip packets and tcp/ip to serial data.
No converter will break up a 20 byte burst into multiple packets. If this is happeneing its the worst converter I have ever seen. If the packets were
2,000 bytes then maybe, but I have never seen a converter send while data is still being recieved. I suspect the user has set a setting wrong on the device to only allow 10 byte buffer or something. We have 300byte packets and have NEVER seen on brocken out of hundreds of thousands (I programmed it to log these and it has never happened).
10 bytes was just an example as some times i do receive 40 (consisting on 2 complete packets) bytes too, the point was packet breaking. I guess that its because of TCP/IP behaviour so there is nothing we can do unless we switch to some other protocol. Anyway, i've fixed the issue while making a buffer on converter say for 128 or 256 bytes and it works fine though technically its not that much real time now. But i think it should be Ok as TCP/IP is never meant for real time anyway.
ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here.
All logos and trade names are the property of their respective owners.