I have a client who needs to send 10 bit greyscale video data over a serial link from a PC to an ARM Cortex uP. Right now just using a KISS implementation, constant-overhead-byte-stuffing protocol, two bytes per value, with a simple CRC-type checksum appended to each scanline the link needs to be around 384kbps to support the amount of data required to transmit.
the hardware is on-paper capable of that but in practice the lash up is unstable, there's a bottleneck somewhere, most likely on the PC side as it's not a "real" hardware serial port but virtualized via USB and it has to go thru all the multiple layers of abstraction from the application until it finally gets spit out; the application code isn't even complied it's from a scripting language.
Using compiled C/C++ that can do system calls directly will probably help but I'm also looking for ways to compress the serial stream bandwidth. For 10 bit greyscale data I was thinking that just something like u-law or A-law encoding could significantly reduce the bandwidth without resorting to more compute-intensive bespoke video compression algorithms. Primary requirement is that the link needs to be very fault-tolerant and reliable for long term operation
What does "unstable" mean? Do you loose data or are there simply variable delays? Do you buffer up data in the CPU for display? I would think you'd want to understand the problem before you try to find a cure.
I don't think the eye has the same sensitivity to light as the ear does to sound. uLaw works by approximating a log relationship which matches human sensitivity to sound. More resolution is given to soft sounds extending th e effective number of bits than what is transmitted.
If you are convinced the problem is too high bandwidth, I agree with Hul, y ou are adding 6 bits of overhead by having empty bits on each word. With H ul's packing you can shove the CRC and a sync bit into a 32 bit word and ge t a 60% bandwidth increase. If you need to, do the data reshuffling and CR C check in a small FPGA and present the CPU a FIFO interface via SPI like s ignals where the overhead isn't important.
- Get a 1,000 miles of free Supercharging
The KISS communication implementation works perfectly fine for a while and then both the uP and PC app lock and it fails utterly. transmission stops. annoying problem to diagnose. Logging the data on the link doesn't show anything unusual just prior to it happening.
About all I can say at this point is that it doesn't happen (at least not over the course of a day's runtime) at lower data rates. And that since the transmission is only one-way I don't see any good reason for the PC application to lock up other than the issue originating on that side of things.
all human senses are more-or-less logarithmic in response. the difference in actual brightness between walking outside into a bright spring morning like today, from say an office lit in a usual office-way with ceiling fluorescents is like, a million times brighter outside (sorry not really an optics-guy I don't know the appropriate unit to use, here.) but we don't go blind
In any case it was just an idea on how two bytes of data could be stuffed into one, in fact since the data is only 10 bits I'd first just try right shifting 2 on the way out then left shifting on reception.
It may be that the re-write of the transmission code in C solves the immediate problem so some of my request is just for my reference if further optimization should be necessary. Byte-packing, at the very least, doesn't seem like premature optimization regardless.
I've been doing a CRC for each line rather than each frame as the client doesn't need pixel-by-pixel correctness, if a line is wrong then just dump it and wait for the next frame.
Do you have timeout settings? Sounds like at high data rates part of the packet is getting lost on the PC and the PC is waiting for the rest of the packet and the uC is waiting for a ACK/NAK to continue sending the next frame.
I had somewhat-similar problems when trying to run a Bitscope from a PC... it would work for a while and then the communication would freeze up.
The problem appeared to be the USB adapter I was using. It was one of the inexpensive "translucent blue" adapters, that claim to have a Prologix PL-2303 serial-to-USB bridge inside... but very probably had a clone/counterfeit chip instead. I've heard other reports of these counterfeit chips locking up under heavy load... timing or chip-firmware problem, I suspect.
I swapped the adapter for a known-good FTDI serial adapter (Mouser and Digi-Key are both authorized FTDI distributors, so you can be pretty confident that they aren't selling counterfeits) and the problem went away.
As to compression: A simple logarithmic-encoding scheme (akin to A-law or u-law), followed by a run-length-encoding, might do what you want, without introducing too many artifacts (although there will be some). Another approach would be a 16-to-8-bit quantization with Floyd-Steinberg error diffusion... this will give you a fixed 2:1 compression with (I think) fairly decent visual quality, and it would be quite fast to implement.
USB is a serial bus. It's not either electrically- or protocol-compatible with a "serial port" in the sense that most people have used the term historically (an asynchonous serial port, using either RS-232 voltage levels or a TTL-voltage sort-of-approximation of same). Instead, a device plugged into a USB bus adopts a device address, and then presents the host with a flexible set of "USB endpoints" which can receive, and send, various sorts of packetized data.
"putty" and similar terminal emulators assume the existence of a kernel device interface which "looks like" a PC serial port interface at the software level... that is, it supports arbitrary-length reads and writes, it has a concept of "baud rate", it implements (or pretends to implement) normal RS-232 handshaking, and so forth. A "real" serial port will present such an interface as (e.g.) /dev/ttyS0. The Linux console emulates such a device as (e.g.) /dev/tty0, /dev/tty1, and so forth.
If you try to tell putty to (e.g.) "open up and talk to whatever is plugged into USB port 2 on the front of my machine" it will have no idea what you're talking about. It can't talk to USB ports per se, nor to arbitrary USB devices. It can only talk to thinks that "look like" RS-232 serial ports at the programming level.
USB serial adapters have one or more data endpoints, that can be recognized by their device drivers. The device driver then presents a programming interface such as (e.g.) /dev/ttyUSB0. The driver translates between the kernel's normal "serial port" programming API, and the necessary packet-oriented communication with the USB serial adapter (and this packetized communication is usually manufacturer- specific and often device-specific).
These days, when you hook a PC to a microcontrollers via a "serial" connection, it's common to use such a USB serial adapter (and the necessary driver on the PC side). On the microcontrollers side, the communication is often "TTL serial", using TTL or CMOS voltage levels, and inverted in polarity from what RS-232 requires. Most microprocessors which have "serial port" pins, expect this sort of signaling. Some have the necessary USB-to-serial chip on the micro board (it's often called a "USB bridge" in this case), and these micros can be cabled directly to a USB port on a computer. The USB-to-serial adapter is still present (you just don't see it as a separate piece of hardware) and you still need a serial-port- emulation driver on the PC.
There are a few microcontrollers which provide their own "native USB" communication interface and endpoint... usually using a newer standard called ACM ("asynchronous communication module" I believe... could be wrong). Linux will show these as (e.g.) /dev/ttyACM0, /dev/ttyACM1, and so forth. You can think of these as "built-in USB-to-serial-like protocol, which provides buffered bidirectional communication at USB speeds, doesn't have the concept of 'baud rate', and doesn't pretend to implement RS-232 hardware handshaking or flow control pins." You still need a driver on the PC side for such devices, but Linux (for one) provides a standard driver which should work with any such device.
putty and similar programs will work with ACMs just as they work with RS-232 serial ports, Linux consoles, and vendor-proprietary USB-to-serial adapters.
There are also "USB gadgets", which is a whole different level of complexity. The BeagleBone Black board takes on the role of a network-interface-like gadget when you plug it into a PC - the kernel sees what appears to be an Ethernet-like interface and can run normal networking protocols over the cable.
email@example.com (Dave Platt) wrote in news: firstname.lastname@example.org:
Very nice and succinct. Thanks for that.
I have a small computer that has a USB tty interface for attaching a keyboard to I think. So I think the USB spec has some kind of proviso for adopting one of the ports in some sort of psuedo addressable port. But I definitely do not know.
I think it is the same thing the USB to serial dongle guys use.
You're a bit mislead: EIA RS-232 is an electrical specification for connection of a data terminal equipment (computer/terminal) and a data communication equipment (modem etc), originally for asynchronous transfers up to 20000 bits/s, using voltage level signals.
The character transmission in start-stop mode is a different story, as is further the character coding (usually a variant of the ISO 646 standard, like ISO 10646).
It is a common mistake to include the asynchronous octet transfer and character coding into 'RS-232'.
Windoze is nothing like an RTOS and has always been dodgy at fast serial data since forever. My money is on a FIFO buffer overflow whilst the PC is off doing something "more important" - screen updating or disk IO.
OK the first thing to try is drop the most significant bytes entirely and live with bullseye effects on the highlights to see if as a proof of concept halving the volume of data will bring it back under control.
If that works then you have two basic KISS choices:
Transform the 10 bit data into 8 bit data using a lookup table
y' = INT(8*sqrt(y)+0.5) would do for a start
Or since you know you must have an average step of 4 and have 1024 levels to transform into 256 output levels choose something like
if (y About all I can say at this point is that it doesn't happen (at least
How locked up it is? Is it just the user application that is unresponsive or has the thing really gone AWOL inside a driver?
How fast are you running the serial link?
I think making the transmission code go even faster will almost certainly make things worse if the problem is due to buffer overflow in the PC end serial receiver.
Quick and dirty test is send only the least significant byte and see if that ever locks up. At least then you have a working baseline again.
The other trick might be to run the PC side in a debugger and hope to catch the trap inside the errant driver. There may be options on the fake serial implementation driver to make its IO buffers deeper.
Good serial drivers are possible on a PC but not in Windoze. The OS/2 serial driver was capable of implementing full 16550 hardware FIFO behaviour on the crudest bare metal original stone age serial ports.