CMOS camera w/ USB2 -- crazy?

Hi,

I'm a firmware guy pulled into a project well out of my area of expertise. My boss wants to build (essentially) a digital camera using an image sensor chip (1600x1200) and output it's data "as fast as possible" using USB2.0. His initial concept, being that I'm a firmware guy, was to use a "really fast micro." Normally the company is involved with small 8-bitter micro projects, so you can see I'm well out of my normal bounds.

Now this seems like a pretty big stretch to me... and I don't see how it can be done without the speed of hardware (the image chips all seem to have clock speeds in the tens of MHz and the USB2 transfer rate is 480Mbps (theor.). Do aspects of this project require an FPGA to keep the data "as fast as possible?" If we use and FPGA for camera-to-RAM and then use a micro for the USB2 part, what kind of fast micros can move data at that rate?

Also, this is something that I am sure we will have to contract out, so if you have any past experience with this, please let me know your thoughts (and if you are available).

Thanks!

Reply to
GB
Loading thread data ...

I don't think the micro should be in the data path. A micro can be used for control and management, but the data path should be pure hardware for maximum throughput. If you look around, you will find that there are USB chips with integrated micros (many are 8051s). But the USB 2.0 chips typically have a DMA engine for the data path.

We have some background in this area. If you are interested, please contact me at the email address below or at sales (at) arius.com.

--
Rick "rickman" Collins

rick.collins@XYarius.com
 Click to see the full signature
Reply to
rickman

Reply to
hamilton

Sounds like a good project for an FPGA. Micros don't keep up with this kind of data rates, standard parts do what everyone else is doing (and this doesn't sound standard), and ASICS can do this, if you have the volume to make it worthwhile, but it would be wise to prototype the design in a FPGA first.

I've done several FPGA designs to solve similar problems. I'm not available, but I'm fairly sure I know of someone fairly good that is available, and if you send mail to (attbi.com phil-hays) I'll get you in contact with him.

--
Phil Hays
Reply to
Phil Hays

That's another undecided at this time, but CMOS most likely.

GB

Reply to
GB

From what I understand, the image size is 1-2 MB (only need 8-bit depth) large and the USB packets are quite small in comparison. So there needs to be some smarts for setting up what part of the data gets DMA'd to the USB endpoint buffers for any given transfer? Or does the FPGA implementation just point to the image data stored in RAM (assuming image frame captured to some local RAM buffer) and never move the data from image_buffer -> to_endpoint -> to_USB (essentially skipping the middle man by just using a pointer of sorts)?

It's just a still picture (not a movie). Doesn't this seem like a somewhat common function -- meaning wouldn't there be standard chips or implementations that do this?

Thanks again, GB

Reply to
GB

Aha, that's an important fact as it greatly reduces the requirements. Depending on what "as fast as possible" (in terms of latency) means, a DSP or a better RISC microprocessor might be a better solution for you. At least you won't have to introduce yourself to the new (for you) area of FPGAs.

I don't know how the interface of the CCD chip you are planning to use is working, but I can imagine that it uses some standard interface that can be found on higher-class processors.

So I suggest you to check some DSPs first (Analog Devices, Texas Instruments, Motorola, ...). Although you might not need the computing power they provide, perhaps you can use their hardware interfaces. DSPs are often used for image processing and therefore have a close relationship to CCDs.

Perhaps you could also ask for this issue in comp.dsp.

Regards, Mario

--
----------------------------------------------------------------------
Digital Force / Mario Trams      Mario.Trams@informatik.tu-chemnitz.de
 Click to see the full signature
Reply to
Mario Trams

Yes, Data-> USB 2.0 is common, and a good example of a std chip is the Cypress chip someone has already mentioned.

You may need a CPLD or small FPGA to run the CMOS sensor clocks, and help the transfer, but the idea is to do as much in the standard uC.USB2.0 package as you can.

I would also nail down just what "as fast as possible" really means. Tell your boss there is a speed/price/development time tradeoff, and find what speed the product really NEEDS.

Also find out, early on, if any form of compression is envisioned, or if a simple 'frame dump' is fine.

-jg

Reply to
Jim Granville

Until recently I was under the impression that CMOS sensors were junk and were of low resolution & quality typically 320.240 used in $0-50 cameras. I have an old Connectix webcam device thats is all green that demonstrates that.

Recently though a Micron engineer persuaded me that CMOS is looking very much up and showed me some upcoming sensors for both high resolution & higher quality still not quite as good as CCD but far cheaper to make & potentially far easy to integrate with externl interfaces. If the quality is there, I would expect USB2/FW to end up on chip.

I will wait and see if they show up in the mid range cameras coming soon.

But check out their imaging division for specs if you care.

Reply to
john jakson

One suggestion is to use a fairly decent "microcontroller" such as Intel Xscale (low power per MIPS) device.

USB means stack. An USB stack couldd be bought from SoftConnex, for instance. A fitting device might be found from Transdimension.

Homann (Yes a relation exists)

--
Magnus Homann, M.Sc. CS & E
d0asta@dtek.chalmers.se
Reply to
Magnus Homann

I believe all present digtal cameras use CMOS. That includes monsters in the 16Mpixel range.

--
	Sander

+++ Out of cheese error +++
Reply to
Sander Vesik

Nope. CMOS sensors are found in cheap cameras up to about 1Mpixel and up to about $100. Above that it is all CCD sensors.

-- Neil Franklin, snipped-for-privacy@franklin.ch.remove

formatting link
Hacker, Unix Guru, El Eng HTL/BSc, Programmer, Archer, Blacksmith

- hardware runs the world, software controls the hardware code generates the software, have you coded today?

Reply to
Neil Franklin

"GB" wrote in message news:ZMJ6b.2615$ snipped-for-privacy@newsread3.news.pas.earthlink.net...

Which means that you may be able, after all, to do the whole thing with a micro - if you can tolerate very long image readout times (~2 second).

CCDs *can* output their pixels very fast - they *must* do so for video applications, because you have to be able to read out all 2M-ish pixels in one frame time - but they don't *have* to. The image sensor manufacturer probably specs some stupid-fast minimum pixel rate, but you don't need to believe that. They simply don't want the hassle of testing the things at low data rates. Very, very low data rates will degrade the image because of leakage of charge in the CCD transfer registers, but only if you go to stupid-long readout times. So here's the deal:

Using your micro.... (1) Pulse the frame transfer clock just once, like it says in the data sheet for the beginning of a frame. This transfers all the accumulated photo-optical charge from imaging cells (pixels) to transfer cells in the CCD. The now-empty imaging cells immediately start to accumulate photo-charge, but you can suppress this process by using the CCD's "electronic shutter" feature - typically, this involves pulling the anti-blooming gates to a special voltage so that any new photo-charge is spilled off into the anti-blooming drain. (2) Wiggle the vertical AND horizontal transfer clocks TOGETHER using your micro. Yes, I know the manufacturer's data doesn't tell you to do this. All you're trying to do here is to clean out the transfer registers as quickly as you can. The video output will be garbage during this process. Keep going until you have clocked the vertical transfer at least twice as many times as there are video lines; remember that your camera has been exposing for a very long time before all this business began, so there's a lot of photo-charge sloshing around that needs to be cleared out thoroughly. Anyway, you need only around two thousand cycles of the clock, and you don't need to DO anything on those cycles - just keep on clocking! (?) It's probably a good idea to do (1) and (2) once more, particularly if the camera has been idling in a high light level for a long time. (3) Stop all the clocks, to keep the camera nice and quiet. Open the electronic shutter so that exposure can begin. (If you prefer, you could have opened the shutter at any time during (2). This makes the process a bit more complicated, but quicker). Time your exposure, then: (4) Hit the frame transfer to get your desired image into the transfer registers. Then you can close the electronic shutter again, so that any further illumination does no harm. (5) Now you need to get the image out of the camera. You can do this at leisure, but you should follow the camera manufacturer's *sequencing* of vertical and horizontal clocks to the letter. Because you're in no hurry, software can do this. Digitise the resulting outgoing video, capture it through the micro, and you're done. As slowly as you like.

To reduce shutter latency, it's better to loop around steps (1) and (2) continuously so that the camera is always ready for a new exposure in no more than the time it takes to do (2).

Details will vary dramatically between cameras, but this is do-able with any interline transfer CCD and it greatly reduces the hardware requirements - at terrible cost of image capture rate, of course. You'll be doing well if you can get an image out within a second.

It's probably no use to you, but it's worth a thought if you are hardware-averse.

-- Jonathan Bromley, Consultant

DOULOS - Developing Design Know-how VHDL * Verilog * SystemC * Perl * Tcl/Tk * Verification * Project Services

Doulos Ltd. Church Hatch, 22 Market Place, Ringwood, Hampshire, BH24 1AW, UK Tel: +44 (0)1425 471223 mail: snipped-for-privacy@doulos.com Fax: +44 (0)1425 471573 Web:

formatting link

The contents of this message may contain personal views which are not the views of Doulos Ltd., unless specifically stated.

Reply to
Jonathan Bromley

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

"CMOS" sensors are usually 1 megapixel or less and have most of the timing generation integrated on board. They are less flexible than CCD imagers which are CMOS technology.

james

Reply to
james

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

1600x 1200 is essentially a 2 megapixel camera!

1) First step is to determine what the camera is going to be used for!

Terrestial or astronomical or video photography

2) Pick an imager! Either Sony, TI, Kodak, or Panasonic to name a few.

3) From the Imager specs you can derive how fast the data can be clocked out of the imager. Most imagers will transfer the image area into a serial register one line at a time. How fast this is depends on how fast you can clock the serial register. Transfer speeds differ from vendor to vendor.

4) Then build the circuitry around the imager based on its ability to transfer the full image as fast as you want and that meet your cost goals.

Again depending on what you determine as reasonably fast will effect the cost of the imager along with its size. Another consideration will be the speed of the ADC. That can slow things down also. Even if you can clock the serial register of the imager out at 20Mhz rate, if the ADC sasmple rate is 10 MHz that is as fast as you can get the pixel data out.

IF your imager's max clock frequency for the serial register is 20 MHz., you can clock a 1600 pixel row out in about 80 uS. Or the whole image area out of the chip in about 100 mS. So your microC or FPGA will have to read the ADC once every 50 nS or so during the readout period.

There are some CPU cores as well as USB cores that can drop into an FPGA. You can build a large FIFO or add onboard flash to store the picture.

It is not crazy at all in fact it is quite doable. The two key items in a digital camera is the imager and the ADC. All the rest is digital hardware that is well suited for an ASIC or FPGA.

james

Reply to
james
090403060309010104060907 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit

Use the micro to set up the packets in an FGPA/ASIC under isocronous control, and stream them out from there, if you can deal with the low data/frame rates - I used to build large format (4 x 5 and hasseblad) camera digital inserts using big CCD's - USB (even at 12Mbs is too slow for this stuff ( a 1Mpixel at 8 bit will require 2/3 sec + overhead to empty one frame) - tell you boss it sounds cool, but you need 1394 (firewire) or SCSI to do this worth a DS. If you simply want to capture one frame and empty it - consider dumping it to RAM and then out from there. Leaving stuff on a sensor - CCD or CMOS (yes CCD's are CMOS, both n and p type are built) results in large dark currents that make them unusable. At 2/3 second, well depth will be a large consideration here - electrons like to move around.

Andrew

james wrote:

Reply to
Andrew Paule

This is not my project. It's GB's project.

Still you can not build hardware around an unknown sensor. You have to pick an imager and then build the hardware/firmware around it.

james

Reply to
james

Sorry - no offense meant.

Andrew

james wrote:

Reply to
Andrew Paule

According to the Subject: line he is intending to use USB2.

That is 480MBit/s, according to

formatting link
.

Should be fast enough.

-- Neil Franklin, snipped-for-privacy@franklin.ch.remove

formatting link
Hacker, Unix Guru, El Eng HTL/BSc, Programmer, Archer, Blacksmith

- hardware runs the world, software controls the hardware code generates the software, have you coded today?

Reply to
Neil Franklin

We have a CMOS (1.3 mpixel) camera based on TI TMS320DM642 DSP (4000 MIPS) with USB 2.0 connectivity. Changing it to your sensor should be straightforward, since the sensor is on its own board.

Contact me at snipped-for-privacy@XXapollo-image.com; take out the "XX" and I will give you the password to our web site.

Reply to
lichau

ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.