NXC interface to smartphone cam?

[HaWe]

hey,
how is it possible to have a NXC communication interface (presumably by BT) to access a smartphone cam for color and blob recognition?
e.g., in the following video the code is probably running on the android.


But for my approach, the cube solver algorithm should run on the NXT autonomously, I just want the cube faces to be detected via the smartphone cam and the blob localization should be transmitted to the NXT.

[mightor]

All the work is done on the phone in this case. Only motor commands are sent to the NXT. The description of the video explains it all:

The ARM Powered MultiCuber 3 robot, featuring the Huawei Ascend P6, solves the 4x4x4 cube faster than any other robot.

A custom Android app uses the smartphone's camera to capture images of each face of the cube to determine how the puzzle is scrambled. The app includes a sophisticated, look-up table driven, multi-threaded algorithm running natively on the ARM Cortex-A9 quad-core processor to calculate an efficient, short solution to the puzzle of about 50 moves and physically solve the cube in about 2.5 minutes. The Ascend P6's 2GB RAM is essential for this robot as the app and associated data will not fit in a smartphone with less memory than this. The app communicates via Bluetooth with the ARM processor in the LEGO MINDSTORMS NXT brick to control the motors to manipulate and solve the cube.

MultiCuber 3 was designed by David Gilday (Twitter @DavidGilday)

If you want the kind of NXTCam functionality on the phone, you'd have to learn to program either native C++ (very hard) or Java

= Xander

[HaWe]

you didn't read my question closely enough.
You should first read closely, and then answer.
I already assumed that the code of the video construction is running on the phone:

e.g., in the following video the code is probably running on the android.

So my question was not about the function of the construction shown in the video.

What I asked was:

how is it possible to have a NXC communication interface (presumably by BT) to access a smartphone cam for color and blob recognition?
...
... for my approach, the cube solver algorithm should run on the NXT autonomously, I just want the cube faces to be detected via the smartphone cam and the blob localization should be transmitted to the NXT.

that's why I asked how to get a NXC interface to a smartphone (Android) cam.

Finally, if you know a solution, just say it.

If you don't know or when you just do not want to contribute to a solution: just don't post anything and keep quiet.
But answers like "You have to learn C++ or Java" are simply silly.

[mightor]

If you don't know or when you just do not want to contribute to a solution: just don't post anything and keep quiet.
But answers like "You have to learn C++ or Java" are simply silly.

The answer is: it does not exist (using a quick Google Search). The solution is: learn to program it yourself. Nothing silly about it, it's just how it works. You have an itch, learn to scratch it. It's how almost every single piece of software is originally created. The alternative is to find someone who has a similar itch and will do the work for you.

Could you tell me what it is you're looking for other than NXTCam-like functionality? The software on the Cubestormer II is very sophisticated and it's so much more than just image recognition. Should it compensate for the shape of the objects? On the NXTCam bounding boxes are used, which may not work very well if you have a tilted cube, for example. Do you want RGB filtering or HSV? What kind of framerate are you looking for? How much information do you think you can handle on your NXT?

Be more specific about your requirements and you're probably going to get a lot more information from some of the more experienced people on this forum (I think David Gilday, the Cubestormer II programmer, actually has an account on this forum, not sure though).

= Xander

[HaWe]

my question simply was:

how is it possible to have a NXC communication interface (presumably by BT) to access a smartphone cam for color and blob recognition?

who has eyes to see, shall see!
I didn't ask anything about the cube solver program running on the smartphone - what are your constant teachings about it? Did I ask for it anywhere?
I also don't understand why you ask me anything about NXT cam, but if NXT cam can do this (what i actually doubt), then maybe yes, or otherwise maybe even no.
I think my question is clear by the way I asked it.
What has NXT cam to do with it at all? Do you understand my question better if I said "sort of f NXT cam functionality "? Would you now the answer in this case?
What if it's more like "Vision Command" functionality?
What if anyone has made something completely different from both?


Finally, I didn't ask who does NOT know about the subject, I asked who DOES know.
So if YOU don't know, just say "I don't know", or keep quiet. You're not supposed to know everything and anything that might exist, do you?
Or is your family name "The Omniscient" ?

Honestly, I actually think your only interest is constantly to annoy me.

[mightor]

You'd be surprised how much I learned about image processing when I rewrote large chunks of the NXTCam firmware. I did a fair of research into how to achieve similar functionality on an Android phone. I've also written a C# program that uses the Microsoft Kinect to control two real NXT robots, using the 3D positions of the hands and arms. This is why I asked these specific questions. They make all the difference in how you approach the problem and formulate a possible solution.

The only thing I am not familiar with is the Vision Command system but I didn't get an RCX until I had already had my NXT for several years. The NXTCam's functionality is all about colour matching to make blobs, which are then sent to the NXT. You asked "how is it possible to have a NXC communication interface (presumably by BT) to access a smartphone cam for color and blob recognition". A possible answer is: make an NXTCam-like colour matching system for Android. Does it exist it? No probably not.

I don't live to annoy you, that's simply perception on your part. I wouldn't waste my time with such trivialities.

Good luck finding a solution to your colour/blob matching problem, though, it sounds like an interesting challenge. You could take a look at OpenCV for Android, which is a very well developed system for many platforms and is capable of a LOT of things which you've mentioned in the past. Here's the link: [LINK].


= Xander

[HaWe]

now that sounds much friendlier.
just to make it clear:
I'll (probably) never will write an image processing program, I just need the functionality which I once had (10 years ago) by my Vision Command program for th RCX (and which I never got again for the NXT, certainly not by the NXT cam which was too poor in it's performance (both color detection quality and number of possible BLOBS) , too complicated in it's handling for BLOB recognition, and a too poor support by tutorials and example programs).
The Vision Command once was fine - easy and handy and perfectly documented.

I just said that before: I'm no hardware programmer and don't want it to be, not for NXT device drivers (e.g., i2c), and certainly not on Android hardware.
What I need are ready-made solutions that I can take.
I think I have proven that I first don't have to learn to program, as you adviced me to, because I was the first one who established and published the first real-world lawn-mower Robot with (simplified) environment recognition driven by a subsumption program running on an RCX, some architectures of autonomous neural nets on the NXT, the first NXT astar implementation, the first NXT map-and-explore navigation robot, the first autonomous NXT chess robot, the first NXC speech-recognition program, the first NXC implementation of a Mersenne-Twister, the first NXC implementation of a Fast Fourier Transformation, the first NXC matrix algebra library, and a NXC PID motor regulation with universal fine-tuning abilities, and some more things - and I shared all of that code. And just that's my programming level.

Therefore, I'm sensitive on wisecracks like "The solution is: learn to program it yourself. Nothing silly about it, it's just how it works. You have an itch, learn to scratch it. It's how almost every single piece of software is originally created. The alternative is to find someone who has a similar itch and will do the work for you."
Even if kind of this was not addressed to me but to anyone else: in my opinion it's stupid, arrogant twaddle.

[afanofosc]

Doc, please stop referring to stuff other people write in a derogatory fashion, i.e., "ignorant twaddle". I don't see that Xander was trying to annoy you or that he missed the point of your inquiry. If you keep doing this I will be forced to ban you.

In any case, the bottom line is that I don't think there is any chance of NXC or any programming language running on the NXT ever being able to directly interface with a smartphone camera. If there was a custom app running on the smartphone then it might be able to send camera data over a bluetooth connection to an NXT in a variety of ways and possibly someone could write such a custom app so that it could accept commands over bluetooth from a program running on an NXT.

I'm not writing that custom app, for sure.

John Hansen

[HaWe]

John,
I didn't expect you to write such a program or app ever, I just asked if anyone knows if such a thing already exists to have the functionality.

But anyway, if I'm asking if anyone knows a ready-made solution, then telling me "The solution is: learn to program " is for me arrogant twaddle.

[bmancan]

I really enjoy this forum for all of its interesting and relevant NXT posts. It is annoying to see the constant disrespectful posts and comments made by some users directly to highly respected members of the NXT community (and site admins).

He is specifically asked to stop this behaviour, and immediately responds with the same attitude.

Doc, please stop referring to stuff other people write in a derogatory fashion, i.e., "ignorant twaddle". I don't see that Xander was trying to annoy you or that he missed the point of your inquiry. If you keep doing this I will be forced to ban you.
John Hansen

But anyway, if I'm asking if anyone knows a ready-made solution, then telling me "The solution is: learn to program " is for me arrogant twaddle.

Showing some respect could help to get more favourable responses and support from other members.

BManCan