Choosing a processor for a run-time image-processing based navigation system

Hi. I am currently working on an image processing application at my place of work.

The overview is: We are working on an AGV (automated guided vehicle) for warehousing application. For navigation purposes, we were looking for a system that provides 10mm accuracy. We have chosen image processing for the same. 4 overhead cameras will be taking images of the entire area. Every AGV in the area will have a distinct color marker on top of it. Each camera sends the image wirelessly to the AGV. The circuitry on the AGV shall process the image to identify its own position on the ground.

Further development, once this will be implemented is that the AGv circuitry shall also generate an optimal path to its destination.

Can someone please guide me in choosing the processor. Assuming a color image , the size of each received image shall be ~300 kB for a 640 X 480 pixel image. The same for an image with resolution 1024 X 768 pixels will be ~768 kB. It is desirable to have the system performance as close to run-time as possible. How is the memory used for such an application? What are the other various considerations while choosing a processor for image processing?

Also, there will be other systems like the drive control and the safety system (continuous monitoring of obstacle sensor status) running simultaneously.

Can a single processor handle this kind of an application? What should be the considerations while choosing my processor?

--------------------------------------- Posted through

formatting link

Reply to
sckulkarni246
Loading thread data ...

You need an automated system for a 1300sq ft warehouse?! (Which is the maximum size implied by the specified number of cameras and resolution).

Reply to
Robert Wessel

Without a size specifier, that's a pretty big leap.

If you assume not too much overlap, that's 1200 x 900 pixels. If you assume that the pixels are 2.4", that's 120 x 90 feet, which works out to over 10000 square feet. That's a lot better...

--
My liberal friends think I'm a conservative kook.
My conservative friends think I'm a liberal kook.
 Click to see the full signature
Reply to
Tim Wescott

Yes, but it'll be busy-busy, as well as your comm link. And everything depends on a host of other factors.

The smallest amount of memory that you'll get away with will be a frame buffer that's big enough to hold the image of the thing in the last known position, plus enough pixels around it to accommodate any unexpected motion and any obstacles you want to detect. Doing this "minumum memory" approach will cost you big time in ease of programming, system robustness, and speed.

The most memory that you'll really need would be one frame from each of the four cameras, so that you can do a bone-headed exhaustive search in each frame.

Processing power varies widely for whatever method you choose, too.

I would do the following:

1: Have one processor per _camera_, that tracks all of the AGVs and obstacles in its field of view. These won't have be designed to be small, vibration-proof, low battery drain, or a host of other things that you'd need to pay attention to if they're on the vehicles. Then have each camera's processor broadcast the AGV position -- this will take much less bandwidth than having the AGVs do it themselves, it'll make it easier to have a central supervisory computer (to answer questions like "Car #4, where are you?" when something breaks down), and it'll make it easier to expand the system to a larger space, because the heaviest-duty processing will be expanded along with the number of cameras.

2: Use industrial PC hardware. It's cheap, it should be more than capable of this task, there should be off-the-shelf vision software that'll do most of it for you, and the development will be quick.

3: Then put some small processor on the AGV, like any of the host of ARM parts, or whatever you please, and have fun.
--
My liberal friends think I'm a conservative kook.
My conservative friends think I'm a liberal kook.
 Click to see the full signature
Reply to
Tim Wescott

First of all work out all your lighting, distance, lenses, Field of View calculations, to be sure you get your 10mm resolution,

Work out ALL your algorithms and trial them with MEASUREMENTS.

Look at what lenses you are using as barrel/pin/chrmoatic distortion will screw up your calculations.

How do you envisage

Installation set up

Calibration of system

Dealing with changes in lighting levels - do any roofs have skylights, this will cause sunlight flooding at known tmes of day per set days of the year) - Is the warehouse only light by internal lighting, how do you cope with flickering light or blown light

Dirty colour marker recoginition.

I do assume your marker has shape for direction recognition what happens when marker is obscured or partly.

Do NOT say "oh the software will sort this out", if you do not know what you are trying to do for ALL these things you do not even have a atart point to even work out your software and systems requirements.

You currently do not seem to have the basis of calculting anything yet.

--
Paul Carpenter          | paul@pcserviceselectronics.co.uk
    PC Services
 Click to see the full signature
Reply to
Paul

The OP mentioned 10mm accuracy is required, so assuming you need a single pixel to achieve the required accuracy the maximum size of the warehouse would be 12.8 x 9.6 meters (~1300 sq ft). That is assuming no overlap and perfect pixel aligned, distortion-less, camera's.

Reply to
Dombo

Whoops -- I missed that.

In theory you could do some fancy interpolation -- but I wouldn't count on it.

--
My liberal friends think I'm a conservative kook.
My conservative friends think I'm a liberal kook.
 Click to see the full signature
Reply to
Tim Wescott

This does seem to be an odd way to achieve the goal -- I wonder what's motivating the OP (or maybe the whole thing is being run by the PHB, who did this for a Master's thesis 16 years ago and it worked fine in the space-constrained lab).

--
My liberal friends think I'm a conservative kook.
My conservative friends think I'm a liberal kook.
 Click to see the full signature
Reply to
Tim Wescott

Yes, there seems to be a problem with the OP numbers, if the warehouse is a realistic size, it is not going to work.

I would think the obvious solution is to put the camera on the robot, and markers on the ceiling. Add some dead reckoning and the robot should know where it is quite accurately.

Unfortunately this idea is considered sufficiently considered non-obvious to warrant granting a patent. Perhaps it is only obvious to an outsider, and not a "competent practioner in the field"?

Anyway, you can buy a system off the shelf, it might work out cheaper.

Reply to
Bob

Alternatively, the OP could do a proper analysis of his requirements, complete with a risk assessment for when there are failures in the lighting, camera system or links to the AGV's. The amount of information that would need to be communicated to the AGV's might also be an issue.

I would look at putting the cameras on the AGV's (cheaper ones should be OK) and adding processing to the cameras to determine where it is. I am looking at this aspect as a fun project myself (self guided robot). My solution would be using the GA144 chip but you could do something like Canny Edge Detection with other processors. This would be much simpler than keeping good cameras, adequate lighting, and a large amount of data communication going all the time. With many more cameras and independent processing in place, links to a central command station would be simplified to just the information required to give each AGV its designated task.

--
********************************************************************
Paul E. Bennett...............
 Click to see the full signature
Reply to
Paul E. Bennett

OTOH, there should be a rich field of prior art -- there was a ton of university research done in robotics & autonomous vehicles.

Patterned markers on the ceiling would let the robot get position and orientation, and would certainly solve a whole bunch of issues with fixed- mount cameras.

--
My liberal friends think I'm a conservative kook.
My conservative friends think I'm a liberal kook.
 Click to see the full signature
Reply to
Tim Wescott

It does, and one thing that kept my initial response short was a bit of a feeling that this might be a homework question.

But using cameras in realistic quantities and of realistic resolution (why, for heaven's sake, would you limit yourself to four VGA-resolution cameras, when you can buy OTS 8MP* cameras for a pittance?), and then doing a bit of coarse/fine positioning (the coarse image could obviously be synthesized out of the entire field of high resolution cameras), could work.

Obvious issues like lighting, interference, optical issues, and whatnot, have been mentioned elsewhere.

As mentioned elsewhere, most systems of this type put the imaging on the AGVs, and ceiling mounted markers are clearly one option. Marks on the floor, and/or on fixed vertical surfaces, might be a better option - always easier to measure something close by. And the 10mm resolution seems intended to be fine enough for the robot to stop and do a pick - a local alignment system would almost certainly be much more reliable.

But a combined system is actually kind-of interesting. The ceiling mounted cameras providing a backup position sensing system. If the AGV gets lost, the external system can shut it down (or help it recover). A good backup system might be valuable in a system where humans may be roaming the same warehouse as the AGVs** - you certainly don't want your picking robot to get lost and go screaming down one of the reserved-for-humans aisles (you might allow your AGVs in the human aisles, but you'd want to restrict their speed to something in the vicinity of 1m/s).

*50 8MP cameras could cover a 200,000sq ft warehouse with 100% overlap. **There have been a number of cases where humans have been injured or killed in that sort of scenario.
Reply to
Robert Wessel

First of all - I forgot to mention the dimensions of the warehouse - it is a typical warehouse - about 50 m by 70 m.

Also, the warehouse is internally lit with no openings of any kind.

I know this (image processing) is an odd way - the motivating factor for such an approach is cost-effectiveness. We considered LiDAR and UWB-based position tracking too for the purpose (since they give very accurate results). But, the cost was blowing up. Hence, we are considering alternative options. This is also the reason why we are not going for a 'system off the shelf'.

As for placing the camera on the robot and markers on the ceiling - more times than not, the camera would be facing the sources of light. I understand that when the camera faces the source of light, other objects in its vicinity appear darker. Will that not create problems while processing and identifying markers? Especially if the markers are color markers.

As for the cases when the AGV gets obstructed/lost - we have a redundant system for coarse (~1.5 m) location of the AGV based on RSSI fingerprinting of Zigbee nodes suspended from the ceiling with the receiver on the AGV. After coarse positioning, the AGV's movements shall be forced to the last visible position and re-calculate the path.

As I read your ansewers, I was more and more convinced that placing the markers on the ceiling and camera on the AGV is a much viable solution. As for the image correction algorithms, we are working on it and hopefully design a generic algorithm that can handle the various noise issues.

As for the processor (on which surprisingly there was no comment) - now that you all know the background, can you suggest some processors for the AGV?

After considerable research, we zeroed in on the BeagleBoard. Cost-effective, a large number of documented projects, 60+ GPIO, good image processing capabilities. Can it do the job?

What about the Blackfin series by Analog Devices?

--------------------------------------- Posted through

formatting link

Reply to
sckulkarni246

The reason there hasn't been much comment on the processor at this point is because the specifications are still pretty vague. Until you have a good idea of the image quality you're going to get, you can't do a good job with knowing the processor requirements.

Were I in your shoes I'd start looking for camera possibilities -- you are correct that fighting the lighting is going to be a significant part of the task; getting a camera that's going to stay out of your way on this is certainly going to be part of the fight.

Depending on what lights are being used, you may be able to implement the on-the-ceiling patterns with more or less wisdom -- using the narrowest- band LEDs you can find, aggressively filtering the light to cut down on the illumination-for-humans as much as possible, and choosing a camera that lets you control the gain while not letting bright sources bloom to much are all things that should help.

--
Tim Wescott
Control system and signal processing consulting
 Click to see the full signature
Reply to
Tim Wescott

You are nowhere near choosing a processor yet. Get the thing working on a PC using whatever hardware is at hand. (Go for a nice big FAST one.). If you ever get it to work then you can do the cost/benefit analysis of getting the algorithms you have developed to run on other hardware. My hunch is that it will never be cost effective to move away from the PC.

Michael Kellett

Reply to
MK

Precsiely the 10mm accuracy, depends on so many optical factors and max speed of the AGV as to how many images a second have to be taken and then how fast they have to be processed to not fall behind.

If possible I would do this is monochrome to reduce the amount of processing by a third and look for shapes/barcodes in image. probably a multitude of known items in more than one view from AGV to get 10mm accuracy. Reading barcodes is useful in someways as many things in a warehouse will have barcodes on already, so for coarse positioning it helps simplify things.

There are so many optical, aspect, keystone and myriad of image problems let alone environment and safety problems to deal with first.

--
Paul Carpenter          | paul@pcserviceselectronics.co.uk
    PC Services
 Click to see the full signature
Reply to
Paul

So you imply you have some knowledge about the field. Yet you seem to be completely ignoring the question that should result from that knowledge: _Why_ would it be odd? I.e. why is everyone else not doing it that way? Is it really so implausible that this might be because everyone else already found out that your approach is unfeasible?

That reasoning is flawed. If a system like the one you're trying to design actually were cheaper than the established ones, then for that precise reason it _would_ be available off-the-shelf. Or did you really think all other users of such systems just liked spending more than they have to?

Not even to mention that developing a new system from scratch for a one-off, when there are already established systems available in the market, is hardly ever the most cost-effective way. Those other systems have their development costs amortized across a multitude of installations already --- you'll have to pay for that entirely yourself. That alone will drive your costs out of the park.

Hmm... but why would there be any overhead lights at all, in a fully automated warehouse?

Reply to
Hans-Bernhard Bröker

... on a 50 m by 70 m floor, that would require 5000 x 7000 pixels as a bare minimum. Your idea of "4 SVGA cameras" is aiming about 2 orders of magnitudes below the target. Oh, and the warehouse would have to be essentially empty for a mere 4 cameras to cover every square centimeter like that. But empty warehouses can't afford automation.

Reply to
Hans-Bernhard Bröker

Well, to an extent _that_ reasoning is flawed. It's an excellent argument against trying to invent something like the Internet, or the PC, or the microprocessor, or integrated circuits, or transistors, or penicillin, or airplanes, or tanks, or cars, or steam engines, etc., etc., etc.

New things don't get invented because someone looks at the world and can't imagine it with that new thing in it, and sometimes new things get invented (or commercialized) simply because they've been around for 50 years waiting for the enabling hardware to get cheap enough -- and now it is.

So I wouldn't stomp on the OP completely.

Although I do think he's leaving a lot out of his analysis, and doing much of it backwards.

--
Tim Wescott
Control system and signal processing consulting
 Click to see the full signature
Reply to
Tim Wescott

No it isn't. Because none of those inventions were trying to beat an existing market simply by being cheaper.

It's one thing to invent something actually new, and a totally different one to re-invent the wheel and hope to come up with a cheaper one, particularly if the latter is based on no underlying idea that could be actually new.

Reply to
Hans-Bernhard Bröker

ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.