CMOS camera w/ USB2 -- crazy?

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
Hi,

I'm a firmware guy pulled into a project well out of my area of
expertise.  My boss wants to build (essentially) a digital camera
using an image sensor chip (1600x1200) and output it's data
"as fast as possible" using USB2.0.  His initial concept, being
that I'm a firmware guy, was to use a "really fast micro."
Normally the company is involved with small 8-bitter micro
projects, so you can see I'm well out of my normal bounds.

Now this seems like a pretty big stretch to me... and I don't see
how it can be done without the speed of hardware (the image
chips all seem to have clock speeds in the tens of MHz and the
USB2 transfer rate is 480Mbps (theor.).  Do aspects of this
project require an FPGA to keep the data "as fast as possible?"
If we use and FPGA for camera-to-RAM and then use a
 micro for the USB2 part, what kind of fast micros can
move data at that rate?

Also, this is something that I am sure we will have to contract
out, so if you have any past experience with this, please let
me know your thoughts (and if you are available).

Thanks!



Re: CMOS camera w/ USB2 -- crazy?
Quoted text here. Click to load it

I don't think the micro should be in the data path.  A micro can be used
for control and management, but the data path should be pure hardware
for maximum throughput.  If you look around, you will find that there
are USB chips with integrated micros (many are 8051s).  But the USB 2.0
chips typically have a DMA engine for the data path.  

Quoted text here. Click to load it

We have some background in this area.  If you are interested, please
contact me at the email address below or at sales (at) arius.com.  

--

Rick "rickman" Collins

snipped-for-privacy@XYarius.com
We've slightly trimmed the long signature. Click to see the full one.
Re: CMOS camera w/ USB2 -- crazy?

Quoted text here. Click to load it

From what I understand, the image size is 1-2 MB (only need 8-bit depth)
large and the USB packets are quite small in comparison.  So there needs to
be some smarts for setting up what part of the data gets DMA'd to the
USB endpoint buffers for any given transfer?  Or does the FPGA
implementation just point to the image data stored in RAM (assuming image
frame captured to some local RAM buffer) and never move the data from
image_buffer -> to_endpoint -> to_USB (essentially skipping the middle
man by just using a pointer of sorts)?

It's just a still picture (not a movie).  Doesn't this seem like a somewhat
common function -- meaning wouldn't there be standard chips or
implementations that do this?

Thanks again,
GB



Re: CMOS camera w/ USB2 -- crazy?

Quoted text here. Click to load it

Aha, that's an important fact as it greatly reduces the requirements.
Depending on what "as fast as possible" (in terms of latency) means,
a DSP or a better RISC microprocessor might be a better solution
for you. At least you won't have to introduce yourself to the new
(for you) area of FPGAs.

I don't know how the interface of the CCD chip you are planning to
use is working, but I can imagine that it uses some standard interface
that can be found on higher-class processors.

So I suggest you to check some DSPs first (Analog Devices, Texas
Instruments, Motorola, ...). Although you might not need the computing
power they provide, perhaps you can use their hardware interfaces.
DSPs are often used for image processing and therefore have a close
relationship to CCDs.

Perhaps you could also ask for this issue in comp.dsp.

Regards,
Mario  

--
----------------------------------------------------------------------
Digital Force / Mario Trams       snipped-for-privacy@informatik.tu-chemnitz.de
We've slightly trimmed the long signature. Click to see the full one.
Re: CMOS camera w/ USB2 -- crazy?
Quoted text here. Click to load it

 Yes, Data-> USB 2.0 is common, and a good example of a std chip
is the Cypress chip someone has already mentioned.

 You may need a CPLD or small FPGA to run the CMOS sensor
clocks, and help the transfer, but the idea is to do as much in
the standard  uC.USB2.0 package as you can.

 I would also nail down just what "as fast as possible" really
means. Tell your boss there is a speed/price/development time
tradeoff, and find what speed the product really NEEDS.

 Also find out, early on, if any form of compression is
envisioned, or if a simple 'frame dump' is fine.

-jg

Re: CMOS camera w/ USB2 -- crazy?

Quoted text here. Click to load it

One suggestion is to use a fairly decent "microcontroller" such as
Intel Xscale (low power per MIPS) device.

USB means stack. An USB stack couldd be bought from SoftConnex, for
instance. A fitting device might be found from Transdimension.

Homann
(Yes a relation exists)
--
Magnus Homann, M.Sc. CS & E
snipped-for-privacy@dtek.chalmers.se

Re: CMOS camera w/ USB2 -- crazy?

Quoted text here. Click to load it

Which means that you may be able, after all, to do the
whole thing with a micro - if you can tolerate very
long image readout times (~2 second).

CCDs *can* output their pixels very fast - they *must*
do so for video applications, because you have to be
able to read out all 2M-ish pixels in one frame time -
but they don't *have* to.  The image sensor manufacturer
probably specs some stupid-fast minimum pixel rate, but you
don't need to believe that.  They simply don't want the
hassle of testing the things at low data rates.  Very, very
low data rates will degrade the image because of leakage
of charge in the CCD transfer registers, but only if you
go to stupid-long readout times.  So here's the deal:

Using your micro....
(1) Pulse the frame transfer clock just once, like it
says in the data sheet for the beginning of a frame.
This transfers all the accumulated photo-optical charge
from imaging cells (pixels) to transfer cells in the CCD.
The now-empty imaging cells immediately start to accumulate
photo-charge, but you can suppress this process by using the
CCD's "electronic shutter" feature - typically, this involves
pulling the anti-blooming gates to a special voltage so that
any new photo-charge is spilled off into the anti-blooming
drain.
(2) Wiggle the vertical AND horizontal transfer clocks
TOGETHER using your micro.  Yes, I know the manufacturer's
data doesn't tell you to do this.  All you're trying to
do here is to clean out the transfer registers as quickly
as you can.  The video output will be garbage during this
process.  Keep going until you have clocked the vertical
transfer at least twice as many times as there are
video lines;  remember that your camera has been
exposing for a very long time before all this business
began, so there's a lot of photo-charge sloshing around
that needs to be cleared out thoroughly.  Anyway, you
need only around two thousand cycles of the clock, and
you don't need to DO anything on those cycles - just
keep on clocking!
(?) It's probably a good idea to do (1) and (2) once more,
particularly if the camera has been idling in a high light
level for a long time.
(3) Stop all the clocks, to keep the camera nice and quiet.
Open the electronic shutter so that exposure can begin.
(If you prefer, you could have opened the shutter at any
time during (2).  This makes the process a bit more
complicated, but quicker).  Time your exposure, then:
(4) Hit the frame transfer to get your desired image
into the transfer registers.  Then you can close the
electronic shutter again, so that any further illumination
does no harm.
(5) Now you need to get the image out of the camera.  You
can do this at leisure, but you should follow the camera
manufacturer's *sequencing* of vertical and horizontal clocks
to the letter.  Because you're in no hurry, software can do
this.  Digitise the resulting outgoing video, capture it
through the micro, and you're done.  As slowly as you like.

To reduce shutter latency, it's better to loop around steps
(1) and (2) continuously so that the camera is always ready
for a new exposure in no more than the time it takes to do (2).

Details will vary dramatically between cameras, but this is
do-able with any interline transfer CCD and it greatly reduces
the hardware requirements - at terrible cost of image capture
rate, of course.  You'll be doing well if you can get an image
out within a second.

It's probably no use to you, but it's worth a thought if you
are hardware-averse.
--
Jonathan Bromley, Consultant

DOULOS - Developing Design Know-how
We've slightly trimmed the long signature. Click to see the full one.
Re: CMOS camera w/ USB2 -- crazy?
What image sensor chip are you looking at ???


Quoted text here. Click to load it


Re: CMOS camera w/ USB2 -- crazy?


Quoted text here. Click to load it

That's another undecided at this time, but CMOS most likely.

GB



Re: CMOS camera w/ USB2 -- crazy?
Quoted text here. Click to load it

Until recently I was under the impression that CMOS sensors were junk
and were of low resolution & quality typically 320.240 used in $0-50
cameras. I have an old Connectix webcam device thats is all green that
demonstrates that.

Recently though a Micron engineer persuaded me that CMOS is looking
very much up and showed me some upcoming sensors for both high
resolution & higher quality still not quite as good as CCD but far
cheaper to make & potentially far easy to integrate with externl
interfaces. If the quality is there, I would expect USB2/FW to end up
on chip.

I will wait and see if they show up in the mid range cameras coming
soon.

But check out their imaging division for specs if you care.

Re: CMOS camera w/ USB2 -- crazy?
Quoted text here. Click to load it

I believe all present digtal cameras use CMOS. That includes monsters
in the 16Mpixel range.

--
    Sander

+++ Out of cheese error +++

Re: CMOS camera w/ USB2 -- crazy?

Quoted text here. Click to load it

Nope. CMOS sensors are found in cheap cameras up to about 1Mpixel and
up to about $100. Above that it is all CCD sensors.


--
Neil Franklin, snipped-for-privacy@franklin.ch.remove http://neil.franklin.ch /
Hacker, Unix Guru, El Eng HTL/BSc, Programmer, Archer, Blacksmith
We've slightly trimmed the long signature. Click to see the full one.
Re: CMOS camera w/ USB2 -- crazy?
On Tue, 9 Sep 2003 01:42:27 +0000 (UTC), Sander Vesik

Quoted text here. Click to load it
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

"CMOS" sensors are usually 1 megapixel or less and have most of the
timing generation integrated on board. They are less flexible than CCD
imagers which are CMOS technology.  

james


Re: CMOS camera w/ USB2 -- crazy?

Quoted text here. Click to load it

Sounds like a good project for an FPGA.  Micros don't keep up with this kind of
data rates, standard parts do what everyone else is doing (and this doesn't
sound standard), and ASICS can do this, if you have the volume to make it
worthwhile, but it would be wise to prototype the design in a FPGA first.

I've done several FPGA designs to solve similar problems.  I'm not available,
but I'm fairly sure I know of someone fairly good that is available, and if you
send mail to (attbi.com phil-hays) I'll get you in contact with him.


--
Phil Hays

Re: CMOS camera w/ USB2 -- crazy?

Quoted text here. Click to load it
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1600x 1200 is essentially a 2 megapixel camera!

1) First step is to determine what the camera is going to be used for!

Terrestial or astronomical or video photography  

2) Pick an imager! Either Sony, TI, Kodak, or Panasonic to name a few.

3) From the Imager specs you can derive how fast the data can be
clocked out of the imager. Most imagers will transfer the image area
into a serial register one line at a time. How fast this is depends on
how fast you can clock the serial register. Transfer speeds differ
from vendor to vendor.

4) Then build the circuitry around the imager based on its ability to
transfer the full image as fast as you want and that meet your cost
goals.  

Again depending on what you determine as reasonably fast will effect
the cost of the imager along with its size. Another consideration will
be the speed of the ADC. That can slow things down also. Even if you
can clock the serial register of the imager out at 20Mhz rate, if the
ADC sasmple rate is 10 MHz  that is as fast as you can get the pixel
data out.

IF your imager's max clock frequency for the serial register is 20
MHz., you can clock a 1600 pixel row out in about 80 uS. Or the whole
image area out of the chip in about 100 mS.  So your microC or FPGA
will have to read the ADC once every 50 nS or so during the readout
period.

There are some CPU cores as well as USB cores that can drop into an
FPGA. You can build a large FIFO or add onboard flash to store the
picture.

It is not crazy at all in fact it is quite doable. The two key items
in a digital camera is the imager and the ADC. All the rest is digital
hardware that is well suited for an ASIC or FPGA.  


james


Re: CMOS camera w/ USB2 -- crazy?
This is a multi-part message in MIME format.
--------------090403060309010104060907
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

Use the micro to set up the packets in an FGPA/ASIC under isocronous
control, and stream them out from there, if you can deal with the low
data/frame rates - I used to build large format (4 x 5 and hasseblad)
camera digital inserts using big CCD's - USB (even at 12Mbs is too slow
for this stuff  ( a 1Mpixel at 8 bit will require 2/3 sec + overhead to
empty one frame) - tell you boss it sounds cool, but you need 1394
(firewire) or SCSI to do this worth a DS.  If you simply want to capture
one frame and empty it - consider dumping it to RAM and then out  from
there.  Leaving stuff on a sensor - CCD or CMOS (yes CCD's are CMOS,
both n and p  type are built)  results in large dark currents that make
them unusable.  At 2/3 second, well depth will be a large consideration
here - electrons like to move around.

Andrew

james wrote:

Quoted text here. Click to load it

--------------090403060309010104060907
Content-Type: text/html; charset=us-ascii
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1">
  <title></title>
</head>
<body text="#000000" bgcolor="#ffffff">
Use the micro to set up the packets in an FGPA/ASIC under isocronous
control, and stream them out from there, if you can deal with the low
data/frame rates - I used to build large format (4 x 5 and hasseblad)
camera digital inserts using big CCD's - USB (even at 12Mbs is too slow
for this stuff&nbsp; ( a 1Mpixel at 8 bit will require 2/3 sec + overhead to
empty one frame) - tell you boss it sounds cool, but you need 1394
(firewire) or SCSI to do this worth a DS.&nbsp; If you simply want to
capture one frame and empty it - consider dumping it to RAM and then
out&nbsp; from there.&nbsp; Leaving stuff on a sensor - CCD or CMOS (yes CCD's
are CMOS, both n and p&nbsp; type are built)&nbsp; results in large dark currents
that make them unusable.&nbsp; At 2/3 second, well depth will be a large
consideration here - electrons like to move around.<br>
<br>
Andrew<br>
<br>
james wrote:<br>
<blockquote type="cite"
 cite=" snipped-for-privacy@4ax.com">
  <pre wrap="">On Sun, 07 Sep 2003 15:03:39 GMT, "GB" <a
class="moz-txt-link-rfc2396E"
wrote:

  </pre>
  <blockquote type="cite">
    <pre wrap="">Hi,

I'm a firmware guy pulled into a project well out of my area of
expertise.  My boss wants to build (essentially) a digital camera
using an image sensor chip (1600x1200) and output it's data
"as fast as possible" using USB2.0.  His initial concept, being
that I'm a firmware guy, was to use a "really fast micro."
Normally the company is involved with small 8-bitter micro
projects, so you can see I'm well out of my normal bounds.

Now this seems like a pretty big stretch to me... and I don't see
how it can be done without the speed of hardware (the image
chips all seem to have clock speeds in the tens of MHz and the
USB2 transfer rate is 480Mbps (theor.).  Do aspects of this
project require an FPGA to keep the data "as fast as possible?"
If we use and FPGA for camera-to-RAM and then use a
micro for the USB2 part, what kind of fast micros can
move data at that rate?

Also, this is something that I am sure we will have to contract
out, so if you have any past experience with this, please let
me know your thoughts (and if you are available).

Thanks!

    </pre>
  </blockquote>
  <pre wrap=""><!---->^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1600x 1200 is essentially a 2 megapixel camera!

1) First step is to determine what the camera is going to be used for!

Terrestial or astronomical or video photography  

2) Pick an imager! Either Sony, TI, Kodak, or Panasonic to name a few.

3) From the Imager specs you can derive how fast the data can be
clocked out of the imager. Most imagers will transfer the image area
into a serial register one line at a time. How fast this is depends on
how fast you can clock the serial register. Transfer speeds differ
from vendor to vendor.

4) Then build the circuitry around the imager based on its ability to
transfer the full image as fast as you want and that meet your cost
goals.  

Again depending on what you determine as reasonably fast will effect
the cost of the imager along with its size. Another consideration will
be the speed of the ADC. That can slow things down also. Even if you
can clock the serial register of the imager out at 20Mhz rate, if the
ADC sasmple rate is 10 MHz  that is as fast as you can get the pixel
data out.

IF your imager's max clock frequency for the serial register is 20
MHz., you can clock a 1600 pixel row out in about 80 uS. Or the whole
image area out of the chip in about 100 mS.  So your microC or FPGA
will have to read the ADC once every 50 nS or so during the readout
period.

There are some CPU cores as well as USB cores that can drop into an
FPGA. You can build a large FIFO or add onboard flash to store the
picture.

It is not crazy at all in fact it is quite doable. The two key items
in a digital camera is the imager and the ADC. All the rest is digital
hardware that is well suited for an ASIC or FPGA.  


james

  </pre>
</blockquote>
</body>
</html>

--------------090403060309010104060907--


Re: CMOS camera w/ USB2 -- crazy?
This is not my project. It's GB's project.

Still you can not build hardware around an unknown sensor. You have to
pick an imager and then build the hardware/firmware around it.

james


wrote:

Quoted text here. Click to load it


Re: CMOS camera w/ USB2 -- crazy?
Sorry - no offense meant.

Andrew

james wrote:

Quoted text here. Click to load it


Re: CMOS camera w/ USB2 -- crazy?
Andrew

No problem.

james



wrote:

Quoted text here. Click to load it


Re: CMOS camera w/ USB2 -- crazy?

Quoted text here. Click to load it

According to the Subject: line he is intending to use USB2.

That is 480MBit/s, according to http://www.usb.org/faq/ans2#q1 .

Should be fast enough.


--
Neil Franklin, snipped-for-privacy@franklin.ch.remove http://neil.franklin.ch /
Hacker, Unix Guru, El Eng HTL/BSc, Programmer, Archer, Blacksmith
We've slightly trimmed the long signature. Click to see the full one.

Site Timeline