c++ - OpenCV Edge/Border detection based on color

Question

I'm fairly new to OpenCV, and very excited to learn more. I've been toying with the idea of outlining edges, shapes.

I've come across this code (running on an iOS device), which uses Canny. I'd like to be able to render this in color, and circle each shape. Can someone point me in the right direction?

Thanks!

IplImage *grayImage = cvCreateImage(cvGetSize(iplImage), IPL_DEPTH_8U, 1);
cvCvtColor(iplImage, grayImage, CV_BGRA2GRAY);
cvReleaseImage(&iplImage);

IplImage* img_blur = cvCreateImage( cvGetSize( grayImage ), grayImage->depth, 1);
cvSmooth(grayImage, img_blur, CV_BLUR, 3, 0, 0, 0);
cvReleaseImage(&grayImage);

IplImage* img_canny = cvCreateImage( cvGetSize( img_blur ), img_blur->depth, 1);
cvCanny( img_blur, img_canny, 10, 100, 3 );
cvReleaseImage(&img_blur);

cvNot(img_canny, img_canny);

And example might be these burger patties. OpenCV would detect the patty, and outline it.

Original Image:

Answer 1

Color information is often handled by conversion to HSV color space which handles "color" directly instead of dividing color into R/G/B components which makes it easier to handle same colors with different brightness etc.

if you convert your image to HSV you'll get this:

cv::Mat hsv;
cv::cvtColor(input,hsv,CV_BGR2HSV);

std::vector<cv::Mat> channels;
cv::split(hsv, channels);

cv::Mat H = channels[0];
cv::Mat S = channels[1];
cv::Mat V = channels[2];

Hue channel:

Saturation channel:

Value channel:

typically, the hue channel is the first one to look at if you are interested in segmenting "color" (e.g. all red objects). One problem is, that hue is a circular/angular value which means that the highest values are very similar to the lowest values, which results in the bright artifacts at the border of the patties. To overcome this for a particular value, you can shift the whole hue space. If shifted by 50° you'll get something like this instead:

cv::Mat shiftedH = H.clone();
int shift = 25; // in openCV hue values go from 0 to 180 (so have to be doubled to get to 0 .. 360) because of byte range from 0 to 255
for(int j=0; j<shiftedH.rows; ++j)
    for(int i=0; i<shiftedH.cols; ++i)
    {
        shiftedH.at<unsigned char>(j,i) = (shiftedH.at<unsigned char>(j,i) + shift)%180;
    }

now you can use a simple canny edge detection to find edges in the hue channel:

cv::Mat cannyH;
cv::Canny(shiftedH, cannyH, 100, 50);

You can see that the regions are a little bigger than the real patties, that might be because of the tiny reflections on the ground around the patties, but I'm not sure about that. Maybe it's just because of jpeg compression artifacts ;)

If you instead use the saturation channel to extract edges, you'll end up with something like this:

cv::Mat cannyS;
cv::Canny(S, cannyS, 200, 100);

where the contours aren't completely closed. Maybe you can combine hue and saturation within preprocessing to extract edges in the hue channel but only where saturation is high enough.

At this stage you have edges. Regard that edges aren't contours yet. If you directly extract contours from edges they might not be closed/separated etc:

// extract contours of the canny image:
std::vector<std::vector<cv::Point> > contoursH;
std::vector<cv::Vec4i> hierarchyH;
cv::findContours(cannyH,contoursH, hierarchyH, CV_RETR_TREE , CV_CHAIN_APPROX_SIMPLE);

// draw the contours to a copy of the input image:
cv::Mat outputH = input.clone();
for( int i = 0; i< contoursH.size(); i++ )
 {
   cv::drawContours( outputH, contoursH, i, cv::Scalar(0,0,255), 2, 8, hierarchyH, 0);
 }

you can remove those small contours by checking cv::contourArea(contoursH[i]) > someThreshold before drawing. But you see the two patties on the left to be connected? Here comes the hardest part... use some heuristics to "improve" your result.

cv::dilate(cannyH, cannyH, cv::Mat());
cv::dilate(cannyH, cannyH, cv::Mat());
cv::dilate(cannyH, cannyH, cv::Mat());

Dilation before contour extraction will "close" the gaps between different objects but increase the object size too.

if you extract contours from that it will look like this:

If you instead choose only the "inner" contours it is exactly what you like:

cv::Mat outputH = input.clone();
for( int i = 0; i< contoursH.size(); i++ )
 {
    if(cv::contourArea(contoursH[i]) < 20) continue; // ignore contours that are too small to be a patty
    if(hierarchyH[i][3] < 0) continue;  // ignore "outer" contours

    cv::drawContours( outputH, contoursH, i, cv::Scalar(0,0,255), 2, 8, hierarchyH, 0);
 }

mind that the dilation and inner contour stuff is a little fuzzy, so it might not work for different images and if the initial edges are placed better around the object border it might 1. not be necessary to do the dilate and inner contour thing and 2. if it is still necessary, the dilate will make the object smaller in this scenario (which luckily is great for the given sample image.).

EDIT: Some important information about HSV: The hue channel will give every pixel a color of the spectrum, even if the saturation is very low ( = gray/white) or if the color is very low (value) so often it is desired to threshold the saturation and value channels to find some specific color! This might be much easier and much more stavle to handle than the dilation I've used in my code.