Advice on scanning colour bricks on a moving plate

[markcrosbie]

Hi everyone,

I'm working on a project to scan 2x4 LEGO blocks arranged on a 32x32 blue plate. The blocks are arranged in 4 parallel "tracks" and I have 4 static LEGO colour sensors mounted pointing perpendicularly down over each track (i.e. looking at the studded top of each brick underneath). The bricks move under the sensors as the plate is advanced by a drive mechanism. So far so good.

I'm having very mixed results getting reliable colour detection (!) and have reviewed a lot of the advice out there. Has anyone done colour detection using the LEGO colour sensor on moving blocks (vs. a block held statically in front of the sensor)? Obviously I have shielded around the sensor as best I can, but light still leaks in. The plate moves at a fixed rate and takes about 3 seconds for the 32x32 stud plate to pass under the sensor array. I set the colour sensors to use the WHITE floodlight, and take 3 samples 20ms apart to make a reading - using a majority of 3 decision as to the "true" colour under the sensor. I'm taking the colour as returned by the leJOS algorithm which is meant to return the LEGO colours (i.e. ColorSensor.getColor().

The colours I'm trying to detect are standard LEGO colours: a blue plate, red, white, yellow and green blocks, and the background is LEGO black. The light sensor array is mounted about 2 studs over the blocks passing underneath.

Any advice, tips or pointers are very appreciated!
Mark

[mattallen37]

A picture or two might help to visualize the situation a bit better.

What color sensors are they? You said "LEGO colour sensors", but then you said you are using "the WHITE floodlight". The HT Color sensor V2 is the only color sensor with a white LED. Or, is that just a term used to refer to the rapid cycling of the RGB LED on the Lego color sensor (vs. e.g. solid green)?

I've heard that placing the sensor on a bit of an angle works best (so that it's not 90 degrees perpendicular to the brick it's reading). Another thing that might improve detection, is to use your own color algorithm (based on the raw values).

[tabbycatrobots]

Here are a couple ideas that I use. I have used this same algorithm when running NXT race
cars over colored stickers, and sliding colored Lego tiles on a baseplate over a colored sensor.
I simply read the 6 basic colors (or try to read), black, blue, green, yellow, red, as you are
trying.
My sensor is about 1.5 studs above the tile or sticker. So, experiment with height.
Yellow is often mistaken. I always do a 3rd read when detecting yellow.
My race cars are moving fairly quickly, and it took me 10 - 20 hours of experimentation,
( it was 2 years ago, and I don't remember exactly, but a long time), to get the delays
between reads, versus car speed to be very close to 100% reliable.
I've had the best luck with mounting the sensor perpendicular to the tile or sticker
passing underneath.
Extraneous light is a killer. Use black duct tape, (or black sails from The Black Pearl, Pirates of the
Caribbean, if you want an all Lego solution) to close the gap between the sensor and tiles.
Attached is the task that I use for color reading. Probably only the first ~80 lines, up to the
switch statement, are of interest for your project. This is NXC code, but even if you are not
familiar with NXC, I think the read - delay sequence is understandable. But feel free to
post questions, and I'll try to explain more fully.
In summary, I guess my method was trial and error, trial and error, trial and error, repeat 100
times, then trial and success. Good luck, have fun.
Howard - Tabbycat Robots

[markcrosbie]

Hi Matt and Howard, thanks for your replies!

Matt - I've posted a few images to illustrate the current construction. I'm trying different patterns of blocks on the plate to see what works best for detection. I also have experimented with moving the plate a fixed number of tacho rotations and stopping it to scan, but that leads me into difficulties keeping the plate moving at a constant smooth speed. I'm using LEGO NXT kit v2 colour sensors, as you can see below they are shielded using simple liftarms and sit about 1-2 studs off the top of a LEGO brick. Colour detection is reliable when the bricks are stationary under the sensors, but unreliable once the plate is moving, regardless of the speed I choose. No doubt this is caused by spurious reflections of light as the bricks move. I'm using the leJOS call to set the floodlight to white, i.e. all 3 LEDs fire in rapid succession.

Howard - thanks for your sample program and thoughts. It's encouraging to see that someone else has struggled with this problem too, so I'm not alone. I've read your NXC code and it seems similar in approach to my leJOS code. I've done some processing on the samples by actually taking a frequency distribution of the colours detected in a given "slot" on the plate and then finding the most popular colour. I've commented this code out for now.

I think Matt's suggestion is what I'll experiment with tonight - build a calibration plate and get the raw RGB values from the sensors and model what they look like for each sensor and each colour and build my own calibration table which I can store in a file and load as needed. Howard - you comments on light shielding have me thinking that I need to build a LEGO "hood" over the sensor assembly to make it fully light-sealed.

Keep the ideas coming folks!
Mark

[tabbycatrobots]

Thanks for posting some photos. Here are some random thoughts that may help.
I wonder if the studs on the bricks are causing any problems. I'm always scanning
a flat surface, i.e a flat sticker, or 2x2 tiles. Have you tried mounting your bricks
on the side, so you're scanning a flat surface?
A second thought - my code is looking for a color change. I think this may be easier
than trying to measure the length of the same color, i.e. to determine how many
bricks of the same color are in a row - assuming I understand your code.
I wonder if by trying to control 4 sensors, if a reading is missed, or the timing is
not as expected. Have you tried running with only one sensor?
Did you notice my debug code that was commented out? When I was trying to debug
my code, I just wrote the color codes read, into an array. When the array got full,
the race car would stop, and the codes displayed on the screen. If a color reading is
done, as the sensor is half over a yellow and half over a red, or yellow - green or ??,
it may be a problem, depending on how your code handles this. That's why I have to
get 2 identical readings in a row, before using the value. Your code that sometimes
uses the first and third readings, throwing out the second, is a little different
algorithm.
Random thoughts - guesses - maybe one will help.
Howard - TabbyCat Robots