Tablet/phone cameras

Op Thu, 16 Jun 2011 16:54:47 +0200 schreef Don Y :

Do be wary of the fact that there are two different kind of lenses used in camera phones:

• "narrow focus": a focus light is used to determine the appropriate position of the lens

• "wide/fixed focus": no moving parts or focus light needed

Wide focus is used in cheap forward-facing lenses (for capturing your surroundings), fixed focus is used in cheap backward-facing lenses (for capturing yourself).

All phone camera's I've seen are fast enough to show captured images on the main LCD at several frames per second. Compressing and storing the data somewhere is a different matter.

Without knowing the answer for any specific phone, I will go out on a limb and say: it depends.

For my phone, movie mode is lower quality for a few reasons:

1. data has to be compressed on-the-fly (with a video codec!)
2. compressed data has to be stored on a slow SD card
3. in lower quality, it doesn't matter whether the focus is exactly right
4. audio is also captured, compressed and stored

Taking a picture is more than just grabbing a frame of data from the CCD, but also:

• adjusting light sensitivity of the CCD (which might take many frames)

• focussing the lens (which might take many frames) And in the 'default' setting also:

• deciding whether to use the flash

• loading the software that controls the camera (including any "cool effects" that might be provided in real-time, like sepia, negative, solarise, etc.)

• charging the flash capacitor

Since at the very least, light sensitivity adjustment will need to be made at a certain rate, frames will probably be grabbed at a fixed rate when the camera is active. I guess it is the choice of the application to skip as many frames as it needs to.

Can you say "battery life" ? What if I'm sleeping? What if the phone is in my pocket? Also, you may not even have complete control over that; my camera phone has a sliding lens cap that triggers camera mode.

It would be a great advantage if the phone had an accelerometer. Then you can grab more frames if the phone is held still while viewing a suitably bright object.

My comments are mostly for Android camera phones (i.e. LG VS740).

Phones cameras are more geared for 2D barcodes, but i guess old fashion barcodes are workable.

n =A0

=A0

My phone has only auto focused forward-facing lens.

Motion video are captured and processed in SDRAM (256M in my case) first. Single hi-res (upto 2000x1500) takes couple of seconds. The jpeg file can be several megabytes compressed.

=A0

=A0

You can always write your custom camera app: Java for Android.

b =A0

Yes, you have to keep charging it.

=A0

Build-in app only do single shot or video (fixed 15 FPS, i believed). Of course, you can write your custom apps.

e =A0

=A0

kip =A0

You will have to write your custom apps.

e is =A0

=A0

you =A0

=A0

I would go for a rooted Android. Unfortunately, theVS740 is hard to root.

om/mail/

Gack! You've confused me.

OK, my understanding was that the *original* cameras in phones (so far, we haven't talked about tablets) were on the "back" of the phone. I.e., while viewing the screen (the "main" screen -- since some phones have more than one), you would be looking in the same direction that the camera was "pointed". (so, the camera would be imaging what you see

I guess this is what you call the "forward-facing" lens?

Some phones appear to have a little adjustment "lever" on this lens (rotate it 90 degrees about the center of the lens to cause ____________).

You mentioned narrow and wide/fixed. Then, went on to describe wide and fixed. Are wide/fixed synonyms? In which case, what about "narrow" (is this what would be called "macro" on a digital camera?)

But, the display in most (?) phones seems to be lower quality than the stated camera capabilities. Is this real-time display deliberately discarding data? I.e., it exists solely to help the user get an idea of what the camera is shooting? Like a "preview" mode in a page scanner?

Does the application have to do anything to make this data available to the display (i.e., is the application copying data from camera to display or is there a special data path that is enabled to make this happen "for free")

OK. None of those allow you to conclude that the phone is incapable of *processing* data at that rate (i.e., and discarding it)

It would be very "unfriendly" if the device had to keep taking flash photos. So, I will assume there will be enough ambient light to capture the images required.

Adjusting light sensitivity can be ongoing -- because the camera is "on always". I guess this is the dimming/brightening effect you tend to see when the camera is first "enabled".

Is the camera focused mechanically? What role does the application software take in doing that? Does the application *know* when the scene is "in focus" or do you rely on the user delaying "snapping" the picture until the displayed image "looks good"?

So, the camera isn't just "turned on and off" but, rather, treated as an application in its own right.

This suggests that the *camera* does this adjustment? Or, is it in the "camera application"? I.e., do I have any control over it?

This will be used in an industrial setting. I.e., turned off at the end of a shift. Possibly returned to a "charging base" frequently during the day (if folks learn that it is prudent to do so).

If its in your pocket, then it won't see anything (and will just waste battery life).

Note that the application will obviously idle the "phone" (which could be a tablet, recall) when it notices that it isn't being actively used.

Ah! That's an excellent idea! If the camera is "in motion", then, chances are, the *image* won't be "in focus". And, regardless, the user will not be "focussed" on a particular barcoded object.

So, you could almost idle the camera -- just doing brightness compensation and, possibly, some crude scene processing to augment the accelerometer's idea of whether or not the device is "moving".

Thanks! There's a fair bit to chew on...

You expectorate a hairball when confused?

Yes.

Hmm, I didn't even think about hand-operated lenses.

No, wide and fixed are not synonyms. Wide focus means the image is sharp over a wide range (like in a disposable camera), fixed focus means the image is only sharp at a fixed distance. Analogously, narrow focus means the image is sharp only in a narrow (but not fixed) distance range. IIRC "macro" is for short distances only, and involves dynamic mechanical zoom (which most phone cameras don't provide).

But, I was just trying to illustrate that some lenses need time (and energy) to focus on the object of interest.

It has no choice but to create a scaled version of the image. What's wrong with that?

Yes.

Dunno.

Yes.

Depends on the lens type, as mentioned above.

My phone camera has a "dual action" button like in regular cameras. Press it softly and it will try to focus (which may fail), press it all the way to capture the image. If you press it all the way directly, it will first try to focus. The user is notified when the camera thinks that the image is focused. But, it will only know after trying to focus. So it is an interactive process.

Yes. In order to conserve resources, I presume.

Dunno. But why would you want to mess with it?

This is highly variable. To run off a few possibilities - some phones/ tablets have the camera on an internal USB interface. Some have the image sensor connected directly to a dedicated port on the SoC (this would almost always be the case for cameras). To acquire images, you have to power up the sensor, enable a DMA channel (for the dedicated camera port type solution), maybe instruct the GPU to set up an overlay window to perform YUV->RGB conversion. Exposure and white balance are typically adjusted on the fly by host-side software but this is certainly not universally true, especially for the USB solutions. The frame rate will typically be some divisor of 60Hz, for some resolution which might be anything from 160x120 up to full HD resolution depending on the camera. Some cameras, especially higher resolution cameras, have mechanical autofocus.

Notice how I'm using words like "variable" "typical" etc. These systems can range from a single-core 400MHz ARM9 to a dual-core 1.2GHz Cortex with a separate DSP and/or other coprocessor(s), so there is absolutely no one size fits all answer.

These sensors also typically require fairly long exposure times, so simply waving the camera about won't acquire a good image - you need to select a subject and hold the sensor still while acquiring the image. Usually, the live-view mode is significantly lower resolution than the sensor's actual max resolution.

Sure, because you have powered up the sensor, moved the CPU core into a higher power mode, possibly activated a separate DSP core to handle MPEG compression, possibly powered up additional segments of the GPU (if any) to handle the overlay window, increased RAM bandwidth usage, forced the screen to remain on with 100% duty cycle, ... ... ...

And for this you should start by looking at the APIs offered by the OS in the device you intend to use, because life's too short for hacking below the OS level in this case - especially if you want to insulate yourself against supply chain changes.

Yes -- hence my mention of QR (below). It's conceivable that you could decode linear ("classic") barcodes but they would probably need to be printed at exaggerated scales to get the proper discrimination between wide/narrow bars/spaces.

So, if there is enough horsepower (or, clever enough recognizer), the image could be examined -- and discarded -- relatively quickly (i.e., no need to compress and/or store it)

Every 3 minutes? 2 hours? etc.? You have to "keep charging" a cell phone *regardless*... the point is how much of an impact it makes on battery life, relatively speaking. (e.g., does phone w/ radio off, camera on consume as much/more/less than phone w/ radio on, camera off, etc.?

Yes, I plan on writing most everything "from scratch" as I suspect it will be *more* work to try to coerce a device made for one type of use into a very different type of usage (see comments re: software reuse thread :> )

Thanks, I'll look and see what's around with my basic needs (incl tablets).

Ah, OK. "depth of field".

I'm not claiming there is anything *wrong* with it! :> Rather, I am wondering how much of a "fortunate circumstance" this is for the phone implementor? I.e., knowing he only has to *display* ~1/4 of the data that he's taking in?

But, there is nothing that prevents a piece of code from making that decision (albeit possibly "less well")?

Just rying to understand what's involved, what's "canned", what's likely to be "proprietary", what's clonable, etc.

But these aren't (as) important when processing monochromatic video (e.g., barcode symbols).

Understood.

So, "motion video" is, of necessity, pretty poor quality?

Is the issue related to luminance, too (instead of just chrominance)?

As expected. But, many of these things I can live without or have to deal with already. E.g., I don't have to worry about compression because I'm not saving images. How the video is displayed is entirely up to me (if it is displayed at all!). CPU will already tend to be running at a higher duty cycle (imagine being *on* the phone all the time it is powered up). Etc.

Supply chain won't be a problem in the quantities we're looking at. In the short term (prototyping), I suspect trying to fit some "middleware" above the OS to make it behave the way the application would *want* it to behave would be more work than gutting the device completely -- especially as it will need to be done *eventually*. Annoying to have a successful prototype and then have to "start over" to come up with a

I took 10 pictures and the battery drop from 66% to 65%. So, i guess you can take 1000 pictures on a full charge.

larwe: USB sensors are impractical beyond VGA. Need lots of memory and bandwidth on chip to do so. The 3M pixel sensor of VS740 has parallel data bus into the system DSP. Pictures are first captured in SDRAM, then write to SD card. However, in Linux, you can probably memory map the SDRAM for processing, but you need rooted Android. The camera software library is available in C++ (unsupported), but wrapper in Java (supported) available.

None --- people do expect those images to look good shown on other displays, including ones considerably better than the phone's own one. And the image display has zooming capability, too, so you don't get away with all that much nonsense even before the image is uploaded somewhere.

This mismatch between sensor resolution and on-camera display resolution has been with us since just about day 1 of digital photography. There is a reason most digicams still have viewfinders of some design.

My point was concerned with the fact that the camera (software) need not have to deal with *processing* all that data for *display* in real-time. Presumably, this reduces the demands on that process ("fortunate circumstance" for the phone implementor)

These sensors don't have a heck of a lot of dynamic range, so it's easy to wash out even a barcode. I think you really would be well served by buying a couple - for example go to

The augmented reality apps that work on a printed orientation glyph do almost exactly what you're talking about, but they don't need to read much information out of the shape, and they also incorporate other sensor input to make inferences about how the shape is moving around in the FOV.

Varies :) For the low end, absolutely. At the extreme low end, even the resolution is very constrained.

You don't care about it, but the APIs available to you may implicitly turn on those features. Getting fancy behind the scenes by poking at the power management features of the system will likely get the drivers/OS very confused.

(choosing words with some care here) With a similar though not identical problem to solve, a large corporation with which I'm familiar has decided to approach this problem entirely at the app layer, and qualify individual devices.

I'm not sure if that extrapolation necesssarily follows from the empirical evidence. :> If that were the case, you could take 60 seconds of 15FPS video (?)

Pointer to a URL?

The pictures were taken over 20 to 30 seconds. So, you can probably run the phone for about an hour (pictures or video). The processors (600MHz CPU + 400MHz DSP) are the main battery killer anyway.

google "android camera programming" will give you plenty to read.

Dunno. I've never been particularly interested in the camera aspects of these devices before. I noticed that the cameras were "slow to start" and would typically take three "repaints" to get the brightness (?) sorted out in low light conditions (I would usually play with these in the evening hours so light wasn't overly abundant). I should take one outdoors and turn it on to see how that fares...

I have several (older) smartphones that I've accumulated over the years. I used an HTC wizard for a few months while traveling as a "portable email (wifi) client". And, I know I have an assotment of Treos (kept because they have "keyboards" -- as does the wizard).

But, since I don't *use* cell phones, I confess I've never played with anything other than the built-in applications...

I have the luxury of being able to bastardize the code for my own purposes -- though being able to read "real" QR codes might have some advantages (and DISadvantages).

We'll either work on bare iron or contract the development of our own drivers to give us the features we want/need. We're not trying to be a "typical phone/tablet" so doubt solutions intended for those sorts of markets would fit well.

We want to be sure the devices are *not* usable for any purpose other than our own. And, we will have other I/O's that you just can't expect to get in a phone/tablet. So, rather than design yet another set of "peripherals" and figure out how to marry them to , it makes more sense to roll all the development into one set of devices. This should also greatly simplify the maintenance of the system as well as the software.