LPC quad UART garbling 8th bit in TX data - difficult to reproduce - any ideas?

Dear Everyone,

apologies for cross-posting, I'm wondering if some of the veterans of

10-15 years ago are still reading these USENET groups... I'm sending this problem description just in case the weird symptoms ring a bell with someone - if someone happens to have a past experience that might resemble my symptoms. I will deliberately not name the vendors involved - we are not sure exactly where the problem is.

Strange as it may seem to the majority PC user nowadays, in our industrial process control practice we're still using RS232 and its flavours RS485, RS422. The problem at hand occurs in a [brand censored] panel PC (that's an LCD with an embedded motherboard in a flat wallmount box), that's based on an Intel Atom, coupled with the i945GSE chipset (including some ICH7 flavour for a south bridge). The PC doesn't seem to contain a full-fledged SuperIO chip (no floppy, no PS2, no LPT/joystick, just USB) and its three serial ports are implemented using a [brand censored] quad UART chip, that connects to the ICH via an LPC bus. The quad UART chip has some configurable "industrial" features, such as TEMT bit exported to an outer modem control signal (for HW-based RS485 RX/TX steering) and something called "9bit mode".

Under some circumstances, for an unknown reason, the quad UART tends to produce garbled TX data. You open the HyperTerminal, select a COM port, configure it for 9600 8N1, without flow control. If you send a string of e.g. the ASCII character "a", the 8th bit in every *second* character is inverted. Bit 8 in the RS232 character = the last before the stop bit = the MSB. One character okay, another character garbled. In HyperTerminal, it looks like a string of good characters mixed with non-ascii garbage. An interesting feature seems to be, that it does this with *any* character you can type on the keyboard - any letters or numbers. If you type different characters in a sequence, the garbage doesn't occur. If you keep sending a single character, it starts producing garbage. It doesn't matter if you type the characters isolated (so that the FIFO is empty all the time) or if you send a string via the clipboard, so that the characters get "clocked out" back to back (and buffered in the UART's 16byte FIFO in the process).

The waveform is fairly clear, either the bit is there nice and clear, or it's completely absent - there are no glitches or weird malformations. There's no doubt that the garbage is coming out of the UART's "trasmitter shift register". Observed with an oscilloscope straight at the UART's TTL level output, before RS232/485 drivers, with nothing attached to the RS232/485 ports (no load on the drivers). Observed on port 1 and 3 of the quad-UART chip. Unfortunately I don't have an LPC bus analyzer to check if perhaps the data is coming garbled from the ICH. Makes me wonder how come that everything else works just fine, all the addresses and data on the LPC bus - just that bit #8 gets garbled. Otherwise it might be attributed to interference from WiFi or BlueTooth (both in the box).

Another problem is that the symptom tends to go away if you start looking too hard or poking around. On some machines it seems to appear after a cold boot, after a period of inactivity. On other machines, it starts after some flawless production runtime. It definitely seems to go away if you switch the baud rate away from 9600 and back to 9600. And it doesn't come back after the next reboot...

An interesting aspect is, that it only occurs on Windows XP Embedded. If the problem occurs, and I reboot to Linux (via Etherboot), I cannot reproduce it with Minicom or other serial TTY software that I have. If I reboot back to XPe on a CF card, you bet it's there. This is a known- clean system with only the basic drivers: a stock Windows serial.sys, and the next closest driver to RS232 is a Penmount touchscreen driver (never a problem with that one, sitting fixed on its own port). I've tried with a pristine system, before the visualization app gets installed (the production app that works with the COM ports), and I cannot imagine a filter driver sitting on the COM ports - in the first place I haven't installed any, in the second place I don't know what purpose it might have.

I'd really love to know where the gremlins are hiding this time :-)

If you've read this far, thanks a lot for your attention.

Frank Rysanek

Reply to
Frantisek.Rysanek
Loading thread data ...

On Mon, 21 Dec 2009 07:53:53 -0800 (PST), " snipped-for-privacy@post.cz" put finger to keyboard and composed:

What happens if you send a string of genuine 8-bit characters, eg extended ASCII? This will force Windows to "think about" the 8th bit, whereas anything you type at the keyboard will usually be 7 bits.

- Franc Zabkar

--
Please remove one 'i' from my address when replying by email.
Reply to
Franc Zabkar

... using a [brand censored] quad UART chip, that connects to

Well, there's two hints: first, it doesn't happen in Linux. Second, it 'comes and goes'.

Alas, that leaves two possibilities: a bit of background software that reinitializes the UART registers and is unaware of the FOUR ports that are there, which inadvertently changes your '8n1' settings but which changes it back to the correct value later. It would be rational to suspect other serial processes (IRDA as well as COM ports). You've already looked there, though.

And, perhaps there's a bad bypass capacitor near the UART, and it allows the power to warble enough to change a logic threshold. That would be heat-related, you might be able to use a hot-air gun and create the symptom.

Reply to
whit3rd

You speak of opening "the HyperTerminal" -- is that the terminal app that's part of Win XP? If so, be aware that it itself is flawed, at least in my experience: when I was using it to capture data from an RS-232 serial line communications link, it would habitually and consistently buffer its capture contents so badly that scrolling back to earlier parts of the capture, that *had* looked OK at first, showed very badly garbled contents.

My solution was to replace it with some other Terminal app, coming from the public domain, that turned out *not* to manifest such issues.

No guarantee that that's where *your* issues arise, of course, but as the issue arises under XP and not under Linux, it's a thought ... .

HTH; and cheers, -- tlvp

--
Avant de repondre, jeter la poubelle, SVP
Reply to
tlvp

Is is possible the send or receive is set to 7 bits + parity and the other set to 8 bits? that would mess up the 8th bit on 50% of the possible characters.

G=B2

Reply to
stratus46

You have a "stock Windows serial system"? No Windows updates, no driver updates?

I'd suspect a problem right there. Go ahead and try *ALL* the updates. And since serial support is not Windows strong point, and especially since Hyperterminal is an incredibly limited, try installing and using the 'Putty' freeware, which has decent serial support and vastly better terminal emulation.

Given the concatenation of somewhat oddball chipsets in this system, and the claim that it works properly under Linux, I'd suspect driver issues, particularly flow control handling. Are your serial devices actually wired correctly for hardware flow control? Many well-meaning but clueless engineers get the RS-232 signals for different devices miswired, and never notice because they've mishandled flow control and don't use it much.

You should also definitely state which Windows and Linux operating systems you're using.

Reply to
Nico Kadel-Garcia

In hyperterminal there is an option to treat incoming messages as 7 bit ascii. It sounds like you are getting 7 bits. Have you tried this?

Reply to
Andy

" snipped-for-privacy@post.cz" wrote in news: snipped-for-privacy@c3g2000yqd.googlegroups.com:

Hyperterminal is known to garble *RECEIVED* characters under certain circumstances if the first three characters recived are identical. This fault, once triggered, persists until the application is exited. I wonder if it is possible for the same bug to also affect transmit?

Please try a 3rd party terminal and see if the problem goes away...

Also censoring the uart chip brand and number will not lead you to any answers...

--
Ian Malcolm.   London, ENGLAND.  (NEWSGROUP REPLY PREFERRED) 
ianm[at]the[dash]malcolms[dot]freeserve[dot]co[dot]uk 
 Click to see the full signature
Reply to
Ian Malcolm

My thoughts exactly.. Its kinda like saying I've got a broken monitor and I want to know whats wrong with it.. It doesn't power on...

Well my first question is what make and model is it.. Often certain hardware suffers the same problmes.

- Mike

Reply to
Michael Kennedy

It sounds to me like the problem is an interaction between the standard serial driver & the extra features of the UART. Try contacting the manufacturer of the UART, & see if they know about this 'bug' & have a workaround, or an updated serial driver. What UART is it?

--
    W
  . | ,. w ,   "Some people are alive only because
 Click to see the full signature
Reply to
Bob Larter

Dear gentlemen,

many thanks to all of you who responded :-)

I'm delighted to see that the USENET is still alive, and that it's sometimes the same pe>

s

And, maybe special thanks to Ian Malcolm - his bug description is the closest to mine :-) If I exit Hyperterminal and start it again, the problem is typically still there. It survives even a power-cycle :-/

[suggested by several polite people]

Yes, this is definitely something I should do. I've seen one more occurrence of the problem, but didn't have a chance to test this - I probably inadvertently reset the UART and the problem went away before I tried a different terminal. I have Putty and CRT on my USB flash drive just in case.

I know :-( It's a matter of not biting the hand that feeds you. After some early rookie experience in that way, I prefer to be cautious. After all, I'm not sure exactly where the problem is, and this decription could cast a shade on innocent hardware, which otherwise looks pretty good...

right, tried that - made a file containing a megabyte of some character with the MSb=3Dlog.1, and the problem was still there. Back to 7bit characters, and still it was there.

Ha, I haven't checked for IRDA. Makes me wonder if the IRDA modes possibly could use this kind of hardware feature. The UART's can work in some IRDA modes, but the hardware doesn't have a physical IR port, and as far as I know, the IR modes are not even offered in the BIOS setup for the COM ports. I can only hope they're properly disabled. Anyway I believe that Windows don't detect an IR port. Have to check to make sure - thanks for that note. Still it makes me wonder. If the UART device was seized by some IRDA service, the production app running on the PC would fail to open it as a COM port. Sounds weird... I was wondering if some Windows-internal ISA PnP stuff or generic SuperIO support could be messing with the SuperIO chip's config registers at random. Those are a special bank of registers with multiplexed access, separate from the classic ISA UART footprints in IO space. Perhaps more likely the BIOS could have some bugs in that direction (ACPI/SMI handling and whatnot). Wish I had an LPC bus probe :-)

No. It works just fine most of the time. The peers are set allright against each other. Seen such misconfigurations before. This is different. Most important it garbles every *other* character in a string. Thanks for the suggestion though :-)

It's a cut of XP Embedded based on XP SP3. I've checked the versions of serial.sys and serenum.sys - they're the same as on my laptop, with Windows Update active.

no they're not :-) You _don't_ use flow control on RS485. Anyway a flow control misconfiguration behaves very different. You don't get anything transmitted (the write() syscall blocks), or you get whole characters lost at the other end (FIFO overruns).

[chuckle] don't tell me about well-meaning engineers :-) What the Taiwanese HW designers can come up with... Such as auto-loopback relays on Ethernet ports, in hardware intended for firewall applications... or they use Linux in an embedded box, are soo very secretive about it (except that you can see the version strings in the firmware images available for download), and they can't seem to get *default routing* right, and they have bugs in serial port handling on a great 16C950 UART (those are two areas where Linux has been okay for decades now, in my experience).

the production OS is XPe, derived from XP SP3 (Czech locale I believe).

The Linux that I'm using for tests, is an in-house mini-distro assembled by hand (scripted file copy) based on Fedora 5, with a 2.6.28.6 kernel (custom config). Booting via PXE and working just fine with my tools :-)

--
The lasting problem is that I keep getting pieces of the suspected
hardware
 Click to see the full signature
Reply to
Frantisek.Rysanek

his

er

n

I had similar issues when I was writing a user interface for a machine that had a windows pc interfacing with a PLC. We were using win98 if I remember correctly. I think the uart and windows had an issue because of the way windows would multitask. Windows would lock the application just long enough to put it out of sync with the uart, and cause garbled data, or a frozen application. I think there were also problems with keeping other applications from accessing the port, even though it should have been locked. At first, I would have to restart windows when the app crashed. But then I rewrote the app, so I could just re-initiate communications when it failed. I have a binder around here somewhere, with our notes and data sheets and the like. I don't know if they'll apply to you, but I'll post a better explanation if I can find them.

I must be getting old when I remember fixing something, but don't remember how the hell I did it. Then tell a long drawn out story with no useful information whatsoever.

-Lazers

Reply to
Sansui Samari

his

er

n

You might try posting here.

formatting link

Reply to
Sansui Samari

=A0This

nder

y
.

ion

e
p

And maybe searching through here...

formatting link

Reply to
Sansui Samari

Frank,

I saw a similar problem but it was running at approx 1Mb/s. I had problems with clock skew in the devices. The clock in the send device was not the same as the receive device. Hence the clocks would slowly drift over time. I usually destroyed the 8th bit. Sometimes it would be just fine. I saw the part about every other character which made me think if there is no character pacing, it might take a full character time to resync. Just an idea..

-Greg p.s. I liked your ground loop discussion and graphs. Good info!

many thanks to all of you who responded :-)

I'm delighted to see that the USENET is still alive, and that it's sometimes the same pe>

And, maybe special thanks to Ian Malcolm - his bug description is the closest to mine :-) If I exit Hyperterminal and start it again, the problem is typically still there. It survives even a power-cycle :-/

[suggested by several polite people]

Yes, this is definitely something I should do. I've seen one more occurrence of the problem, but didn't have a chance to test this - I probably inadvertently reset the UART and the problem went away before I tried a different terminal. I have Putty and CRT on my USB flash drive just in case.

I know :-( It's a matter of not biting the hand that feeds you. After some early rookie experience in that way, I prefer to be cautious. After all, I'm not sure exactly where the problem is, and this decription could cast a shade on innocent hardware, which otherwise looks pretty good...

right, tried that - made a file containing a megabyte of some character with the MSb=log.1, and the problem was still there. Back to 7bit characters, and still it was there.

Ha, I haven't checked for IRDA. Makes me wonder if the IRDA modes possibly could use this kind of hardware feature. The UART's can work in some IRDA modes, but the hardware doesn't have a physical IR port, and as far as I know, the IR modes are not even offered in the BIOS setup for the COM ports. I can only hope they're properly disabled. Anyway I believe that Windows don't detect an IR port. Have to check to make sure - thanks for that note. Still it makes me wonder. If the UART device was seized by some IRDA service, the production app running on the PC would fail to open it as a COM port. Sounds weird... I was wondering if some Windows-internal ISA PnP stuff or generic SuperIO support could be messing with the SuperIO chip's config registers at random. Those are a special bank of registers with multiplexed access, separate from the classic ISA UART footprints in IO space. Perhaps more likely the BIOS could have some bugs in that direction (ACPI/SMI handling and whatnot). Wish I had an LPC bus probe :-)

No. It works just fine most of the time. The peers are set allright against each other. Seen such misconfigurations before. This is different. Most important it garbles every *other* character in a string. Thanks for the suggestion though :-)

It's a cut of XP Embedded based on XP SP3. I've checked the versions of serial.sys and serenum.sys - they're the same as on my laptop, with Windows Update active.

no they're not :-) You _don't_ use flow control on RS485. Anyway a flow control misconfiguration behaves very different. You don't get anything transmitted (the write() syscall blocks), or you get whole characters lost at the other end (FIFO overruns).

[chuckle] don't tell me about well-meaning engineers :-) What the Taiwanese HW designers can come up with... Such as auto-loopback relays on Ethernet ports, in hardware intended for firewall applications... or they use Linux in an embedded box, are soo very secretive about it (except that you can see the version strings in the firmware images available for download), and they can't seem to get *default routing* right, and they have bugs in serial port handling on a great 16C950 UART (those are two areas where Linux has been okay for decades now, in my experience).

the production OS is XPe, derived from XP SP3 (Czech locale I believe).

The Linux that I'm using for tests, is an in-house mini-distro assembled by hand (scripted file copy) based on Fedora 5, with a 2.6.28.6 kernel (custom config). Booting via PXE and working just fine with my tools :-)

--
The lasting problem is that I keep getting pieces of the suspected
hardware
 Click to see the full signature
Reply to
G B

ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.