pob-eye2, CMUcam3 or blackfin for lego nxt interfacing

[photon-1]

Hi,

I am looking to choose a camera (considering pob-eye II, CMUcam3 & blackfin) that will allow me to do the following (not at the same time but in different projects):

1. object recognition (a known type of object) using just the chosen camera setup and the object.

QUESTION: I imagine it would be easier to do this by just connecting the camera direct to a PC and and using C++ or matlab to do the image processing.
-IS THIS POSSIBLE WITH THE ABOVE CAMERAS?

2. object recognition (a known type of object) using the chosen camera setup interfaced with an nxt.

QUESTION: I imagine I would need to interface the camera with a microcontroller and then interface the microcontroller with the NXT...

-WHICH OF THE ABOVE CAMERAS IS IT EASIER TO DO THIS WITH? (AND HAS MANY ACCESSIBLE EXAMPLES WITH DETAILS & SOURCE CODE)

-I IMAGINE THE GENERAL IDEA IS THAT I WOULD NEED TO WRITE CODE FOR THE MICROCONTROLLER TO PROMPT THE CAMERA TO 'GRAB' IMAGES (WHICH ARE STORED ON THE CAMERAS OWN IMAGE BOARD), THEN THE MICROPROCESSOR WOULD USE THE DATA AVAILABLE FROM THE IMAGE(s) ON THE CAMERAS IMAGE BOARD TO PERFORM THE IMAGE PROCESSING. AND FINALLY THE CODE ON THE NXT WOULD READ THE RESULTS OF THE IMAGE PROCESSING FROM THE MICROCONTROLLER.

IS THIS CORRECT?

I will be doing alot more than object recognition but i think if it is capable of doing that then I should be fine; and by object recognition I dont just mean tracking blobs of colour; I need to be able to access and manipulate individual pixels from the images.

I've done alot of programming (C,C++,NXC,Matlab & assembly language) and covered theory on microprocontrollers and interfacing and stuff but I've never actually worked with hardware (apart from the nxt which is easy) so I'm farely new to it. If I could just get answers to the above questions then I can buy some hardware and start working on it. I'd like to go with the blackfin because it has the highest resolution but i will go with whichever one you all suggest!)

THAAANKS...

(SUPPLEMENTARY QUESTION BUT NOT IMPORTANT)
IS IT POSSIBLE TO Just CONNECT A WEBCAM TO A PC, AND USE IMAGE PROCESSING LIBRARIES FOR C++ or MATLAB TO GRAB IMAGES AND PERFORM THE IMAGE PROCESSING? (OR ROBO REALM, BUT I DONT KNOW MUCH ABOUT IT AND I WOULD RATHER NOT LEARN ANOTHER LANGUAGE!!!!!)

[linusa]

(SUPPLEMENTARY QUESTION BUT NOT IMPORTANT)
IS IT POSSIBLE TO Just CONNECT A WEBCAM TO A PC, AND USE IMAGE PROCESSING LIBRARIES FOR C++ or MATLAB TO GRAB IMAGES AND PERFORM THE IMAGE PROCESSING? (OR ROBO REALM, BUT I DONT KNOW MUCH ABOUT IT AND I WOULD RATHER NOT LEARN ANOTHER LANGUAGE!!!!!)

Yes, that's one of the simplest solutions you could try right away. I've tried MATLAB's "Image Acquisition Toolbox", and it's really easy to get the live webcam data stream into MATLAB. The Image Processing Toolbox is a great help of analyzing and processing everything. Together with the freely available toolbox for NXT control ( http://www.mindstorms.rwth-aachen.de/ ) you can write a pure native MATLAB program.

I've never used the various NXTCams. While they do some of the work for you (i.e. blob detection), they are less flexible than webcams (and usually have a lower resolution).

[photon-1]

Thanks alot linusa!! I'll start off with that and at least begin developing the algorithms. However i'd still like to find some answers to the other questions especially the second one (2) now since this sort of also takes care of the first question (1).. thanks again.

my hope was to use the same camera system for interfacing with the nxt and for connecting directly to the PC; but I need to start pretty soon so I'll end up buying a good webcam for connecting directly to the PC now and later on I'll buy one for interfacing with the nxt.

I think the pob-eye already has a microcontroller which can be interfaced directly with the nxt; but the resolution is just terrible (160 x 120) so I'm not sure if it will be able to handle the object recognition phase.

somehow my posts always end up being so long.... sorry about that ppl.

[markcrosbie]

What you need is RoboRealm (http://www.roborealm.com/). It allows you to do absolutely everything you outlined above, interfaces to multiple cameras *and* also will talk to the NXT. The documentation and tutorials on the RoboRealm site are very thorough, and almost constitute a machine-vision course in their own right. I've played with RoboRealm a year or two ago and it was really simple to use and very easy to quick test out algorithms and filters.

Unfortunately it is not free as in no-cost, and now costs $89!

I've worked with the Mindsensors NXTcam and they are modified CMUcams that do basic blob detection by colour. You train the camera as to what colour constitutes a blob and then it returns you an array of 8 blobs and their x,y location. Xander Soldaat has a library to do blob-merging, but given the resolution of the NXTcam there's not much else you can do. You can switch it into line-tracking mode and can do more advanced path-following. A NXTcam will not be much use for object recognition by shape, unless your objects are radically different in size and posture. The advantage of a NXTcam is that it connects to the NXT and all processing is done in the camera and in the NXT - no PC is required.

Good luck - image processing on mobile robots is not for the faint of heart!
Mark

[HaWe]

I think the pob-eye already has a microcontroller which can be interfaced directly with the nxt;

yes, sure it has (for shape detection), like the mindsensores cam (which has a blob color detection). Both are directly plugged to a NXT sensor port. In both cases all image processing is done by the cam board hardware.
The pob-eye cam is incredibly slow, and the blob color detection of the mindsensors cam (v3) - well, let's say: - there is definitely room for improvement...

[physics-matt]

The NXTCam largely suffers from the fact that the ATmega chip it uses only has 1KB of RAM, which means it can't store an entire image in memory. This precludes the use of many good image processing algorithms.

I've not used any of the other camera systems, but looking at the spec they do appear to be much more capable - difficulty appears mainly to be to connect them to the NXT. Personally, I prefer the option of using a webcam and PC to do the processing, particularly with a small laptop that can be carried by the robot, but then my hardware debugging skills are pretty poor.

If you manage to interface any of these other sensors with the NXT I would be very interested to see how you got on. For instance, is it possible to connect the CMUCam to the NXT using a RS232 to RS485 converter?

Matt