Converting images to 1 bit color depth

[spillerrec]

I'm currently working on making a dialog which enables users to convert their existing images into RIC sprites (for RICcreator). However converting images to use on the NXT is often a tricky process in my opinion. 1 bit color depth isn't an issue by itself but when it is combined with the small resolutions needed it becomes difficult to keep an acceptable amount of detail.

My normal approach when using GIMP is to start with scaling and cropping the image, then converting the color space into grayscale. The threshold tool is then used to limit it to pure black/white. I normally use the auto button to get a good starting point and then fine tuning it for better results. In the very end, I manually edit some pixels to add detail.

I have created a dialog with this work flow however the issue is that there are several methods to convert to grayscale. They produce similar results yet sometimes completely different. I don't know much about color spaces though
Method 1:

Code: Select all

sqrt( 0.241*R*R + 0.691*G*G + 0.068*B*B )

Seems to provide rather good results.
Method 2: Desaturate a HSV value. Doesn't look too realistic at times but can be useful? (But since I can't figure out how to calculate it, I have use some of QT color conversion functions which are too slow...)
Method 3:

Code: Select all

(R * 11 + G * 16 + B * 5)/32

Faster than method 1 but okay results.
(Just found out how GIMP calculates the "grayness" of a color: http://docs.gimp.org/2.6/C/gimp-tool-desaturate.html)

And then I found quite a few variations of method 1 and 3...
What other methods exits? Does anyone have some ideas of which actually are useful for this?
Perhaps I should just drop all those methods and use method 1, but where you can modify the constants which 3 scrollbars? (As far as I understand, this should modify the brightness of each color channel.)


Moving to a completely different approach, edge detection. I have used it in GIMP and with a bit of fiddling ended up with great results (depending on the source image).
Logically thinking it does seem to be the best approach when the available pixels and colors are sparse since it tries to preserve the shapes and not the colors.
However I know absolutely nothing about edge detection and I therefore don't know if there exists any useful algorithms for this. There are several in GIMP, but the issue is that they don't have any settings to modify the result. In particular I would like to control the complexity/amount of edges but again, I don't know if the algorithms can do this.
Of the algorithms in GIMP the best seemed to be laplace as it produces sharp lines. The amount of edges are a bit high, which is great at higher resolutions, but not always optimal with small. (In particular there was an implementation which made non-edges transparent instead of black which made the process much easier...)

What are your experiences with edge detection, do you think it could be useful in this context? If useful, would you like to see this in a program like RICcreator (as an extra method) or would you prefer to use GIMP anyway for this?
Does anyone have some good material about edge detection btw?

[linusa]

What other methods exits? Does anyone have some ideas of which actually are useful for this?
Perhaps I should just drop all those methods and use method 1, but where you can modify the constants which 3 scrollbars? (As far as I understand, this should modify the brightness of each color channel.)

The first method I learned in my image processing lecture was something like this from the link you posted:

Code: Select all

Luminosity = 0.21 × R + 0.72 × G + 0.07 × B 

The coefficients may vary, but in general, this is the method I'd always use to get the "brightness/luminosity/grayscale" of a picture. The reason for these coefficients (the naive approach would be to take the average of R, G, B, i.e. with coeffs of 0.3333 each) is that the human eye is more sensitive to green than to red, and blue has significantly less influence on overall sensed brightness. The commonly quoted graph is this:


So overall it's challenging to convert your 3-channel RGB pic to a 1-channel monochrome pic, but that should be the easiest part and should only contribute a small part to your result.

However I know absolutely nothing about edge detection and I therefore don't know if there exists any useful algorithms for this.

The most commonly used is the "Canny Edge Detector". You can look for the original paper or find lectures on it and/or tutorials.

What are your experiences with edge detection, do you think it could be useful in this context?

It could surely be useful, but -- as you've noted yourself -- it always depends on the input images. Very much so. It might be useful for comic pictures, but not for photographs. It always depends.

I know the NXT's resolution is not that big, but you could also look into dithering.

To help you with your actual problem of converting images to 1 bit "black/white" monochrome pictures:
I'd probably try to apply some sort of filter and then do thresholding. Very interesting for you should be the "adaptive thresholding" or "local dynamic thresholding", see here: http://homepages.inf.ed.ac.uk/rbf/HIPR2/adpthrsh.htm (and just google the keywords).

There are many image operations that might further help you with your binary image (or the grayscale image) -- like erosion and dilation or e.g. the top-hat filter, and others or a combination of them. Just some random links: http://www.mathworks.com/help/toolbox/i ... erode.html and http://www.mathworks.com/help/toolbox/i ... ophat.html (see pictures at the bottom).

Or you could do histogram equalization before any further processing steps. Standard technique, too.

You shoul look at the documentation of OpenCV, though it could be better. If you want to get started on such topics and start coding right away, I'd recomment the OpenCV book (pink, by O'Reilly). Maybe your library has it? It explains all such things very good.

I just found those two links, even though it's MATLAB code, the general concepts of image processing are very well explained: http://www.mathworks.com/help/toolbox/i ... 12508.html and http://www.mathworks.com/help/toolbox/i ... 14011.html -- they apply to other software such as OpenCV or AForge as well...


Bottom line: Adaptive thresholding should be the most important keyword of my post

[spillerrec]

Thank you very much for your insight, it is really helpful.

The main reason I looked into several method of converting to grayscale was that with global thresholding it sometimes made quite a difference. I imagine the difference will be much smaller with adaptive thresholding though.

Adaptive thresholding sounds promising and stop after that for now. Instead I will try to read some more about image processing in general while working on other areas in my project. By the time I have finished the basic functionally of the rest of the program I should hopefully have some better foundation to improve this part.

You shoul look at the documentation of OpenCV, though it could be better. If you want to get started on such topics and start coding right away, I'd recomment the OpenCV book (pink, by O'Reilly). Maybe your library has it? It explains all such things very good.

The local library doesn't have it, however some other libraries do and it should be possible to send it here.
However I would like to get some basic knowledge about image processing before trying to use OpenCV. Does this book only explain how to use OpenCV or does it also explain some of the fundamentals?
(I'm have signed up for a university course in computer science, if I end up being accepted I will move to a town in the near future which local library do have this book. If I'm going to read something else first anyway it probably wouldn't be worth the trouble getting it sent here.)

[linusa]

However I would like to get some basic knowledge about image processing before trying to use OpenCV. Does this book only explain how to use OpenCV or does it also explain some of the fundamentals?

The book does in fact explain the basics. I can't really decide how it works for novices, since I've had 1 year of image processing at university before reading it. But I think it's indeded to teach the basics "as you go". So it delivers a very "hands on" approach.

The book has some pictures that at least illustrate some of the concepts very well! However, be careful: It's about OpenCV 1.3, C style, while the current version is 2.2 (2.3), C++ style with some new datatypes.

IMO learning by doing works well for image processing / computer vision, and OpenCV is widely used (however it's a bit of a mess!). As the C/C++ handling can get a bit messy, too, maybe its best to use the Python interface for quick results?
Anyway, book is recommended!