Connect 4 and Image Processing

[gloomyandy]

Folks I've just upload a video of my latest project, a Connect 4 playing robot that uses an NXTCam to identify the moves made by the human player:



The game playing code is based on that from the 4RowBot project, but the major difference is the use of a Mindsensors NXTCam to recognize moves made by the human player. The actual move making robot is also my own design.

There are some more details of how things work over on the leJOS forums but let me know if you are interested in any of the details.

Andy

[haydenstudios]

I like it, bery nice. But just one question, at the end of the video, it shows the board, and says, "a win for yellow!". I looked high and low, and could not see four yellow pieces stacked four-in-a-row, horizontally, vertically, or diagnally. Was this some accident you made when you edited it?

[gloomyandy]

No Accident, look again. There is a yellow diagonal running down and right from the top yellow in column 2 (columns start at 0 on the left), down to second row up on column 5.

[haydenstudios]

OH MY GOSH! I cannot believe I didn't see that!
Forgive me for my lack of ability to recognize that.

[loopymech]

Very cool project! I assume the custom f/w in the nxtcam is to allow tracking of more than 8 objects?

-Loopy

p.s. can you post a link to the forums where you have details?

[gloomyandy]

Hi,
No the custom firmware allows the NXT to read the video frames from the NXTCam (either in one go or individual scan lines). So no object tracking takes place on the NXTCam. All of the image processing takes place on the NXT. There are a few more details here:
http://lejos.sourceforge.net/forum/view ... f=5&t=2648
But if there is anything else you'd like to know just ask away...

Andy

[rbnnxt]

This looks really cool. I built aversion of the connect 4 game which used the colour sensor. see http://www.youtube.com/user/rbn1tube#p/u/2/UDi7ORELoSk
and http://www.youtube.com/user/rbn1tube#p/u/1/ZOShrwxwgtc
However it is tricky to set up and I much prefer your method which scans the whole board from a static position. I think I'll have to invest in a lego camera and give it a whirl.
Are your firmware changes available? How do you implement it? Is it just a change in the mindstorms brick, or is there separate firmware in the lego camera sensor? (I haven't got one (yet) so I haven't much of a clue about it.
Do you have a build file for the lego for lego digital designer?

Robin

[doc222]

well done. Can you set how hard it trys? I have not played the game ever. can it be won or in the end would it be a draw like tic tac toe?

[gloomyandy]

Hi,
I don't have ldraw or source packages available for this project and I long ago dismantled it (this was my Xmas 2010 project), sorry. My main interest was in seeing how far I could go with some form of image processing on the NXT. The actual game play made use of the Java code from another project 4RowBot (http://www.si-entertainment.com/index.php?pageid=55).

According to Wikipedia the first player can always win (assuming perfect play), but the code I used basically performs a limited depth mini-max search which certainly does not produce perfect play!

The firmware for the NXT-Cam used by this project is not available but an older version of my firmware is here:
http://nxtcam.svn.sourceforge.net/viewv ... loomyandy/
But be warned writing firmware for this device is a pretty tricky thing to do. The device NXT-cam is a very clever hardware design that uses a very minimal amount of hardware (basically an atmel microcontroller and a camera module, with no frame buffer), but this means that the software has some very tight memory and real time constraints. If you are interested in the design of such a device take a look at:
http://www.jrobot.net/
and in particular at:
http://www.jrobot.net/AVRcam_design_document.pdf
The above describe a design which I assume form the basis of the NXT-cam...

Andy

[dodgey]

Very clever design using the cam. It's also interesting how Robin has a different perspective on the 4RowBot.

I built the 4RowBot (took me a week to learn how to compile Lejos Java apps , in particular, packages - it's all about the paths!, I digress). I agree you have to make sure the board is aligned perfectly, and the background light has to be even so the colour sensor works ok, but I love the overly complicated mechanics like the chain drive

I assumed using a webcam instead of a colour sensor would simply be "better" but so far I've found they are are about equal (obviously Andy's project is about mastering a specific goal which he's done wonderfully).

The classic robot that uses the colour sensor to the max is Tilted Twister 2. Once built correctly, and operated in a gloomy lit room, it works flawlessly. And uneven or bright light and it fails a lot.

I thought the scanning process of TT2 was too slow and that using a webcam solver would speed things up, being able to scan a cube face all at once. In reality I've found it is on a par with the colour sensor as webcams are really succeptible to uneven or too bright light when being used to determine colours. While testing my current cube solver it's been driving me mad. I get the best results with natural daylight. At night time, overhead bright lights are a no no, and I frequently have to shade the machine with my arms and hands to get successful readings.

btw - here are videos of my builds of 4rowbot, and another mini one that uses NXC

Anyhow - on a tangent I guess, but well done Andy - very nice!