Rectangle 27 20

In the images, the background is mostly greenish and the area of the background is considerably larger than that of the foreground. So, if you take a color histogram of the image, the greenish bins will have higher values. Threshold this histogram so that bins having smaller values are set to zero. This way we'll most probably retain the greenish (higher value) bins and discard other colors. Then backproject this histogram. The backprojection will highlight these greenish regions in the image.

  • Then threshold this backprojection. This gives us the background.
  • Then find the contours of the foreground.

I think this gives a reasonable segmentation, and using this as mask you may be able to use a segmentation like GrabCut to refine the boundaries (I haven't tried this yet).

EDIT: I tried the GrabCut approach and it indeed refines the boundaries. I've added the code for GrabCut segmentation.

GrabCut segmentation using the foreground as mask:

I'm using the OpenCV C API for the histogram processing part.

// load the color image
IplImage* im = cvLoadImage("bFly6.jpg");

// get the color histogram
IplImage* im32f = cvCreateImage(cvGetSize(im), IPL_DEPTH_32F, 3);
cvConvertScale(im, im32f);

int channels[] = {0, 1, 2};
int histSize[] = {32, 32, 32};
float rgbRange[] = {0, 256};
float* ranges[] = {rgbRange, rgbRange, rgbRange};

CvHistogram* hist = cvCreateHist(3, histSize, CV_HIST_ARRAY, ranges);
IplImage* b = cvCreateImage(cvGetSize(im32f), IPL_DEPTH_32F, 1);
IplImage* g = cvCreateImage(cvGetSize(im32f), IPL_DEPTH_32F, 1);
IplImage* r = cvCreateImage(cvGetSize(im32f), IPL_DEPTH_32F, 1);
IplImage* backproject32f = cvCreateImage(cvGetSize(im), IPL_DEPTH_32F, 1);
IplImage* backproject8u = cvCreateImage(cvGetSize(im), IPL_DEPTH_8U, 1);
IplImage* bw = cvCreateImage(cvGetSize(im), IPL_DEPTH_8U, 1);
IplConvKernel* kernel = cvCreateStructuringElementEx(3, 3, 1, 1, MORPH_ELLIPSE);

cvSplit(im32f, b, g, r, NULL);
IplImage* planes[] = {b, g, r};
cvCalcHist(planes, hist);

// find min and max values of histogram bins
float minval, maxval;
cvGetMinMaxHistValue(hist, &minval, &maxval);

// threshold the histogram. this sets the bin values that are below the threshold to zero
cvThreshHist(hist, maxval/32);

// backproject the thresholded histogram. backprojection should contain higher values for the
// background and lower values for the foreground
cvCalcBackProject(planes, backproject32f, hist);

// convert to 8u type
double min, max;
cvMinMaxLoc(backproject32f, &min, &max);
cvConvertScale(backproject32f, backproject8u, 255.0 / max);

// threshold backprojected image. this gives us the background
cvThreshold(backproject8u, bw, 10, 255, CV_THRESH_BINARY);

// some morphology on background
cvDilate(bw, bw, kernel, 1);
cvMorphologyEx(bw, bw, NULL, kernel, MORPH_CLOSE, 2);

// get the foreground
cvSubRS(bw, cvScalar(255, 255, 255), bw);
cvMorphologyEx(bw, bw, NULL, kernel, MORPH_OPEN, 2);
cvErode(bw, bw, kernel, 1);

// find contours of the foreground
//CvMemStorage* storage = cvCreateMemStorage(0);
//CvSeq* contours = 0;
//cvFindContours(bw, storage, &contours);
//cvDrawContours(im, contours, CV_RGB(255, 0, 0), CV_RGB(0, 0, 255), 1, 2);

// grabcut
Mat color(im);
Mat fg(bw);
Mat mask(bw->height, bw->width, CV_8U);

mask.setTo(GC_PR_FGD, fg);

Mat bgdModel, fgdModel;
grabCut(color, mask, Rect(), bgdModel, fgdModel, GC_INIT_WITH_MASK);

Mat gcfg = mask == GC_PR_FGD;

vector<vector<cv::Point>> contours;
vector<Vec4i> hierarchy;
findContours(gcfg, contours, hierarchy, CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE, cv::Point(0, 0));
for(int idx = 0; idx < contours.size(); idx++)
    drawContours(color, contours, idx, Scalar(0, 0, 255), 2);

// cleanup ...

UPDATE: We can do the above using the C++ interface as shown below.

const int channels[] = {0, 1, 2};
const int histSize[] = {32, 32, 32};
const float rgbRange[] = {0, 256};
const float* ranges[] = {rgbRange, rgbRange, rgbRange};

Mat hist;
Mat im32fc3, backpr32f, backpr8u, backprBw, kernel;

Mat im = imread("bFly6.jpg");

im.convertTo(im32fc3, CV_32FC3);
calcHist(&im32fc3, 1, channels, Mat(), hist, 3, histSize, ranges, true, false);
calcBackProject(&im32fc3, 1, channels, hist, backpr32f, ranges);

double minval, maxval;
minMaxIdx(backpr32f, &minval, &maxval);
threshold(backpr32f, backpr32f, maxval/32, 255, THRESH_TOZERO);
backpr32f.convertTo(backpr8u, CV_8U, 255.0/maxval);
threshold(backpr8u, backprBw, 10, 255, THRESH_BINARY);

kernel = getStructuringElement(MORPH_ELLIPSE, Size(3, 3));

dilate(backprBw, backprBw, kernel);
morphologyEx(backprBw, backprBw, MORPH_CLOSE, kernel, Point(-1, -1), 2);

backprBw = 255 - backprBw;

morphologyEx(backprBw, backprBw, MORPH_OPEN, kernel, Point(-1, -1), 2);
erode(backprBw, backprBw, kernel);

Mat mask(backpr8u.rows, backpr8u.cols, CV_8U);

mask.setTo(GC_PR_FGD, backprBw);

Mat bgdModel, fgdModel;
grabCut(im, mask, Rect(), bgdModel, fgdModel, GC_INIT_WITH_MASK);

Mat fg = mask == GC_PR_FGD;

I would like to thank u for your contribution.. it's exaclty what I want and you deserve the bounty :) thanks again.

can u just post the code in c++? Thanks.

@Maystro See the update. I was thinking of adding this update for sometime, but missed it. Now is the best time :)

Thanks, is it possible to add the missing sections as well?

cluster analysis - Detecting object regions in image opencv - Stack Ov...

opencv cluster-analysis object-detection connected-components
Rectangle 27 0

Well...just saying, I'm not sure if this will work, but you can apply the subtraction based on a bigger area (like a kernel) and then place at that point the histogram variation. E.g. lets say our kernel patch is set to 30x30 pixels and we apply this on the pixels p(x,y) from the background image and q(x,y) from the test image (p and q are the same respective positions on each image). As as result we have a 30x30 subtracted patch image.

Now, try to work on this subtracted patch products like its histogram. A zero histogram means identical compared patches. An "almost flat" histogram might mean the same. Otherwise, consider it as foreground.

Would you happen to have some code for this approach? Unfortunately I am not that advanced in OpenCV / C++.

Search for "access cvMat elements", "opencv histogram comparison" and "opencv filter2d".

image processing - OpenCV Background subtraction with varying illumina...

opencv image-processing background-subtraction