Slightly unmatched UART frequencies

When you say "your" UART, is this a design you did yourself in an FPGA? If so you may not have designed the logic correctly. In order for the receiver to synchronize to a continuous data stream, it has to sample the stop bit in what it thinks is the center and then *immediately* start looking for the next start bit. This will allow a mismatch in speed of almost a half bit minus whatever slack there is for the sample clock rate. BTW, you are sampling at at least 8x the bit rate, right?

The max mismatch is not 10%, but a bit less that 5%. In the field I find that 2 to 3% mismatch is reliable, but any more and you can start getting errors. I guess the difference in theory and practice is perhaps skew caused by the drivers.

Does this make sense?

--

Rick "rickman" Collins

rick.collins@XYarius.com
Ignore the reply address. To email me use the above address with the XY
removed.

Arius - A Signal Processing Solutions Company
Specializing in DSP and FPGA design      URL http://www.arius.com
4 King Ave                               301-682-7772 Voice
Frederick, MD 21701-3110                 301-682-7666 FAX

Yes, you're right I have my design runs on CPLD. However, the qestion is more in logic rather than implementation. The value 10% I have got from

's forum where "Software Uart" is a hot topic.

I use 16x oversampling and check input values at middle of a bit (SampleCnt = 7). You suggest exactly what I have done. I think receiver part will work under any condition. But I need to know what should I do with transmitter module. As I attempted to explain, this half-bit solution cannot be used to synchronize transmitters. It is a bad idea to start transmitting next byte at the middle of the stop bit. It may fail listening device with slow clock as it reaches center of stop bit when start bit of next byte is being transmitted. On the other hand, if data is coming slightly faster transmitter should do something, otherwise I face buffer overrun condition. I understand that I can ignore the problem with transmitter module, it is receiver that should synchronize with transmitter. However, I had got buffer overrun problem until I used a trick described in my message (early transmit). It is defenetely not the problem with receiver because I have solved it right before got problem with transmitter's buffer overrun. And I want to know how should function correct logic; there should be a solution as commertial UARTs work without any problems. My UART is the first one where I've realized that it is at all possible to get a problem with slowly transmitting uart. Is now the problem become clearer?

Valentin,

You bright up a good subject, and you're absolutely correct that if you continuously send data from one serial port at 9600.01bps to a receiver at

9600, sooner or later there must be a buffer overflow. There's no way around this -- but keep in mind that RS-232 (or most any protocol, for that matter) isn't designed to send a truly continuous stream for days, months, or years at a time without a break! With a typical RS-232 device, there are MANY breaks, and keep in mind that something like a PC often has a pretty generous software buffer (many kilobytes) backing up the hardware so that it would take a 'long' time to create an overflow. I can't explain your observation that an, e.g., 8031-based data forwarder -- supposedly -- works other than to say I suspect that perhaps you didn't really do the type of torture test that could produce an overrun. (I.e., did you look with a scope or logic analyzer to make sure there was NEVER a idle bit time that might have allowed the receiver to 'catch up'?)

One solution that you can use for protocols such as 8B/10B -- where you get a bazillion data bytes interspersed with an occasional 'comma' character -- is to use a form of compression where you assign 9 bits to ever received byte and 'swallow' any comma you get by setting the high bit. You can then sit down and work out how often you need to insert a comma into your bitstream to avoid buffer overflow. We had a gigabit fiber interface that used this approach, and with a 16 byte FIFO for buffering and a +/-100PPM clock at 1.0625Gbps, the numbers worked out to many thousands of bytes before overflow would be a concern.

To build a data repeater that never suffers from potential buffer overflow under any circumstances whatsoever, I don't think you have many options other than locking the re-transmit bit rate to that of the received data using, e.g., a PLL. You could try this in software as well, I suppose, if you have a bit rate generator that's 'finely tunable' -- most in microcontrollers aren't!

A few other comments:

'Permissible,' yeah, I suppose, although 10% wouldn't be anything to write home about!

Nothing wrong with 16x oversampling (it will definitely help -- a little), but keep in mind that you _can_ get away with no oversampling at all and get quite reasonable results if you position the sample point at the middle of each bit interval.

---Joel Kolstad

Valentin, apparently you are trying to resolve the clock difference by cutting the stop bit in order to achieve a higher transmission rate. That is a very nice idea and it is completely wrong. In commercial (and all other) Uarts, it is the receiver that compensates for the clock difference. The rule is that the transmitter sends 10 bits (Start

• 8 Data + Stop), but the receiver only requires 9.5 bits (1 Start + 8 Data +

0.5 Stop). It is this 0.5 bit difference which compensates the clock difference (and which also gives you the 5% that rick mentioned). So far, your design seems correct. But then you try to speed up the transmitter as well by sending less than 10 bits (actually 9.5 bits). The net effect is that you have changed the usual "transmit 10 & receive 9.5" scheme to a "transmit 9.5 & receive 9.5" scheme, which is as bad as a "transmit 10 & receive 10" scheme when the clocks are different. By doing this you will neccessarily lose single bits long before your buffer overruns. You stole the 0.5 bits from the receiver that the receiver desperately needs to compensate for the clock differences. In other words, your attempt to avoid buffer overflows (which cannot occur since the receiver takes care of the clock frequencies) has actually created the problem you are describing. The solution is simple: don't touch the transmitter. BTW., you should check if your 10% refers to clock jitter (moving of the clock edges around a fixed reference point) rather than to a difference in clock frequency. /// Juergen

I think you are missing a key idea. The receiver has to make sure that it will tolerate early start bits. That is the receiver has to start looking for a new start-of-start-bit right after it has checked the middle of the stop bit rather than wait unitl the end of the stop bit to start looking.

[Interesting thread. Thanks. I didn't know about that trick.]

--
The suespammers.org mail server is located in California.  So are all my
other mailboxes.  Please do not send unsolicited bulk e-mail or unsolicited
commercial e-mail to my suespammers.org address or any of my other addresses.
These are my opinions, not necessarily my employer's.  I hate spam.

at

Correct. If you really must tolerate continuous streams, with no data loss, and only a single stop bit, then you must actually be able to 'pass on' data at whatever average rate it comes in at. Some UARTS have the idea of fractional TX stop bits to allow this, most just choose

2 stop bits to give margin. Receive should ALWAYS start looking for START at the middle of STOP. ( but chips do not always get this right :)

The OP's idea of a full half bit is a coarse example, that can force an error, as it jumps right to the limit

Since this was using 16x BAUD clocking, you can quantize to 1/16 of bit time, or in appx 0.625% steps. 4 of these is 2.5%, or 1/4 bit, that tolerates to

9840 Baud If you don't want to use up all the error budget at one end, that's about the limit. ( 1/4 bit at each end, or a tad less, if you use 3 slot vote sampling )

This is why most uC with trimmed on-chip OSCs specify 2.5% or 2% precision.

Since this a PLD device, you could watch for TX buffer phase, and nominally send a full STOP bit, but if the phase indicates margin problems (incomming faster than outgoing) you can decrement the STOP bit in 1/16 fractions - or you could force 15/16 wide STOP bit and use a crystal, and keep the logic simpler. (tolerates to 9660 Baud, 100% traffic )

A purists design would also extend STOP bits fractionally, (17/16 at ip

MAXIMUM error is .5 bit over one frame. In your case frame = 10 bits. .5/10 = 5%

And if the sender is 2% too slow and the receiver is 2% too fast, you have

4% error which is just below the 5% error tolerated.

--
Best Regards
Ulf at atmel dot com
These comments are intended to be my own opinion and they
may, or may not be shared by my employer, Atmel Sweden.

juergen sauermann wrote:

And Philip writes: Modifying the local transmited character to be a non-standard length by changing the length of the stop bit on the fly as buffer over-run is about to occur is not a good idea, as you don't know the details of how the receiver that is listening to it was designed, and it may not be very happy to see the next start bit before it is finished with what it expects is a full length stop bit, but it is not.

The underlying problem is that you are potentially sending very long streams of data through a protocol that was designed for asynchronous transmission. That is why there are start bits and stop bits. In real systems, there is flow control, typically implemented one of 3 ways:

1) Hardware flow control: CTS/RTS 2) Character based flow control: XON/XOFF (ctrl-q/ctrl-s) 3) Upper layer flow control: packet based transfers with acknowledge packets used to pace transmissions.

The "real-time-ness" (new word I just invented) of the flow control depends on the size of the receive buffer. With only

1 byte, you need (1), and even this may not be good enough, you may need at least 2 bytes of buffer. As the buffer gets bigger (say 8 to 100 bytes) then (2) is workable, and can even tolerate some operating system delay. When the buffers get to be multiple packets in size, then (3) may be appropriate.

juergen sauermann also wrote:

Well up to a point this is correct. The receiver can cetainly declare that the character has arrived after the sample is taken in the middle of the stop bit (at 9.5 bit times into the received character).

BUT this is not a solution to the original poster's problem! The problem still exists because the remaining .5 bit is still going to arrive, the data is being sent with a slightly faster clock than the transmitter is able to retransmit the character. If there is no line idle time between the end of the inbound stop bit and the next inbound start bit, the system will eventually have an over-run problem, no matter how big the input buffer. The closer the two clock rates, and the bigger the buffer, the longer it takes to happen, but it will happen.

Do the math:

Let the far end transmitter be running at 1% faster clock rate than the local transmitter that is going to retransmit the character.

Here are some easy to work with numbers: Perfect 9600 baud is 104.1666666 microseconds per bit

1 character time (1 Start,8 Data,1 Stop) is 1.041666666 ms

After 1 character has arrived, we start to retransmit it. It doesnt matter if we start at the 9.5 or 10 bit time, it will take us 1.010101 times longer to send it than it took to receive.

If we have a multibyte buffer, after 100 characters arrive at a far-end transmit rate that is 1% too fast, we have the following:

.99 * 100 * 1.041666666 = 103.1249999 ms

If our local transmitter is right >It is this 0.5 bit difference which compensates the clock

As you can see above, I disagree. This is not a solution.

Actually, he hasn't changed the receiver to receive 9.5, because the far end transmitter is still sending 10 bits. ignoring the last .5 bit does not solve the problem, as it is accumulative.

Nope. This does not work.

Nope. This does not work.

The following solutions can be made to work:

A) Use one of the 3 described flow control systems above, with a suitable length buffer, or some other flow control system with similar effect.

B) Deliberately force some idle time between characters at the far end transmitter. If your system is designed for a worst case of 5% difference in clock frequencies, forcing an idle between the stop bit and the next start bit of .6 bit time will achieve this (with some minor safety margin). You will still need some buffer though between your receiver and transmitter.

Another version of this is to just add some idle time every N characters, such as "every 100 characters, let the go to sleep for 2 character time".

C) Use a PLL to derive a local clock that is phase locked to the received data, and use this for transmit.

D) At the far end transmitter, add some pad characters at regular times to the data stream, that can be thrown away at the receiver.

E) run a clock line from the far end transmitter to your system and use that for your transmit clock (hardly an async system any more)

F) Be sneaky. Most UARTs can be set for 1 , 1.5 , or 2 stop bits. Set the far end transmitter for 8N2 (1 start, 8 data, 2 stop). Set your receiver and transmitter for 8N1 (1 start, 8 data, 1 stop). This works, because stop bits look just like line-idle. This effectively implements (B), but is localized to the initialization code for the far end transmitter.

Philip Freidin

Philip Freidin Fliptronics

Philip, after thinking about the problem once more, I hate to admit that, yes, you are right. I still do not believe, though, that inserting idle time one way or the other (including cutting the transmitter's stop bit) is a solution. Consider the following: Left side: Slow (9600 Baud) Right side: Fast (9700 Baud) Both sides use e.g. 8N2 for Tx and 8N1 for Rx. At some point in time, Left see's its buffer filling up and hence skips a few stop bits here and there (using 8N1) in order to compensate this. Left is now faster that Right, despite of the clock rates. As a consequence, Right sees its buffer filling up and skips stop bits (using

8N1) as well. This continues until both sides transmit with 8N1 all the time; at this time Left will loose data. Thus, there must be some kind of understanding between Left and Right, which of the two is the "clock master", that ultimately controls the transmission speed. Unfortunately this is sometimes not possible, for instance in symmetric configurations. /// Juergen

G) copy the scheme used in the TI MSP430, effectively a DDS. Using this arrangement gets you a lot more speed, and you can take the TI documentation as a spec and save a ton of time...

If you do it, please forward the results to Xilinx for a writeup as an App Note ;-)

Unless your (slightly slower) transmitter also has the capability of producing shortened start (or stop) bits, how those this approach 'fix' the problem? If the date rates are, say, 9601 received BPS and 9600 transmitted BPS, detecting early start bits just buys you one extra bit interval before your overrun your buffers, doesn't it?

---Joel

(snip)

As far as I know, asynchronous transmission was intended to be between two devices, such as a terminal and a computer, though more likely two terminals in the early days.

The two stop bits were required by machines that mechanically decided the bits. (The Teletype (R) ASR33, for example.) Using stop bits as flow control seems unusual to me.

Electronic UARTs (no comment on mechanical ones) sample the bit at the center of each bit time. For a character with no transitions (X'00' or X'FF') timing error can accumulate for the duration of the character. The STOP bit is the receivers chance to adjust the timing, and start over with the new START bit.

With a 5% timing error, which is very large for a crystal controlled clock, the stop bit could start 0.45 bit times early, but the receiver will still detect it at the right time, and be ready to start the next character.

The timing for each character is from the leading edge of the START bit.

This allows for difference in the bit clock rate between the transmitter and receiver. It is unrelated to any buffering or buffer overflow problems that may occur.

-- glen

"Philip Freidin" wrote

UARTs look for the START edge, from the _middle_ of the STOP bit. With x16 clocking, typically that gives 8 possible time slots for earlier start.

I would agree that a half-bit jump in STOP, as the OP first suggested, is NOT a good idea, but fractional (1/16 quantized ) STOP changes are valid and safe.

Yes, true if the stop bit is 'whole bit' quantized. CAN be avoided if the TX can move the START edge as needed, both left and right, in 1/16 steps. Something like

+/-4 sixteenths would leave design margin.

Yes, by far the simplest, and most practical solution.

However, this is comp.arch.fpga, and here we can design any UART we like, including one that can handle 100% traffic loading, single stop bits, and

1-2% region clock skews. ! :)

To illustrate this, look at the SC28C94 UART data, this from info on their 16 possible STOP BIT options:

MR2[3..0] = Stop Bit Length

0 = 0.563 1 = 0.625 2 = 0.688 3 = 0.750 4 = 0.813 5 = 0.875 6 = 0.938 7 = 1.000 8 = 1.563 9 = 1.625 A = 1.688 B = 1.750 C = 1.813 C = 1.875 E = 1.938 F = 2.000

-jg

Here is my simple analysis: There are two very different situations:

If the transmitter clocks slower than the receiver, there is no problem on the receive end, as long as the error inside the word does not exceed half a bit time.

If the transmitter clocks faster than the receiver, the receiver has to be able to resynchr>

ITU-T Recommendation V.110 (a.k.a. I.463) explains how to do synchronous to asynchronous conversion (to get async serial signals across ISDN). This includes things like stop bit shaving.

Regards, Allan.

Originally developed by Emille Baudot (google) for telex. Hence Baud. Buffer overflows are irrelevant in the modern world since the terminals will be handling the data far faster than the transmission rate. All that is required is the receiving uart detect the start bit reliably. Longer stop bits help.

Well, this is not correct. Analyze the situation yourself and see at what point the receiver will not sample the data at the correct time. You will find the allowed slip is half a bit over the whole word which is about 5% give or take.

That assumption is where you are wrong. There is nothing the transmitter can do to change the way it works unless you *always* transmit at a rate slower than your receiver.

The transmitter can be either faster or slower than the receiver. If the transmitter is slower than the receiver, then the start bit detection will simply happen a few clocks after the end of the stop bit rather than right at the end. If the transmitter is faster than the receiver, then the start bit detection has to occur before the end of the timing of the stop bit. If the transmitter starts the start bit early you make the problem worse. If the transmitter delays the start bit, then this will prevent any issues. So you can eliminate the problem by sending two stop bits and only looking for one at the receiver, but clearly this is not how commercial units work.

Commercial receivers start looking for a start bit leading edge as soon as the middle of the stop bit has been timed and checked. You need to do the same thing. You don't care when the end of the stop bit happens. You only care about the leading edge of the start bit which may be a bit earlier than the end of the stop bit as timed by the receiver.

Others have posted in this thread by this point all the info you should need. If you still don't understand what is wrong with your understanding of the problem, let us know and we will try to make it more clear.

To us, yes. Is it more clear to you as well?

--

Rick "rickman" Collins

rick.collins@XYarius.com
Ignore the reply address. To email me use the above address with the XY
removed.

Arius - A Signal Processing Solutions Company
Specializing in DSP and FPGA design      URL http://www.arius.com
4 King Ave                               301-682-7772 Voice
Frederick, MD 21701-3110                 301-682-7666 FAX

No, by looking for the start of next word before the last word is complete, the receiver can receive data faster than you would calculate given the clock speed and the bit count. The receiver can receive data in 9.5 bit times per byte rather than the 10 bit times per byte the transmitter takes to send them. That is where the 0.5 bit or about 5% rate mismatch numbers come from.

--

Rick "rickman" Collins

rick.collins@XYarius.com
Ignore the reply address. To email me use the above address with the XY
removed.

Arius - A Signal Processing Solutions Company
Specializing in DSP and FPGA design      URL http://www.arius.com
4 King Ave                               301-682-7772 Voice
Frederick, MD 21701-3110                 301-682-7666 FAX