Straighten edges of black rectangle in binary image

I have a binary image that represents a 'rectangle'. The rectangle is not perfect because a top view of the box was taken (then converted to a binary image). My objective is to find four corner points of the black rectangle. In order for the corner function to work the edges must be completely straight.

http://i38.tinypic.com/w0lwro.jpg

clc; clear;
image = imread('0148pm.jpg');
g = rgb2gray(image)
level = graythresh(g);
binary = im2bw(image,level);

imwrite(binary,'imageBinary.jpg');

% Iinv = ~binary; %Invert your binary image
% Iinv = bwareaopen(Iinv,20); %Get rid of small areas (below your size criterion)
% I = ~Inv; %Invert back

imshow(I);

% fill = bwmorph(binary,'hbreak');
% 
% f = bwmorph(fill,'majority');
% 
% k = bwmorph(f,'close');

%corner algorithm///////////////////////////////////////////////////
% C = corner(k,'MinimumEigenvalue', 4)
% imtool(k);
% hold on
% plot(C(:,1), C(:,2), 'r*');
%end corner algorithm///////////////////////////////////////////////

[Image Analyst]

What do you really want to do: find corners or straighten edges? Do you know it's possible to straighten the square without even finding the corners?

[Kal]

I need to find the corners of the black 'rectangle'. That corner algorithm isn't finding the corners of rectangle because edges are not straight and it has discontinuities.

[Kal]

These are the corners being detected currently.

Answer 1

[Image Analyst]

Another way that is pretty easy way, probably the most straightforward is to simply find the centroid of the square and divide it up into quadrants around the centroid. The examine all the points to see which is farthest from the centroid. Like this code:

clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
imtool close all;  % Close all imtool figures if you have the Image Processing Toolbox.
clear;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 20;

% Read in a standard MATLAB gray scale demo image.
folder = 'C:\Users\User\Documents\Temporary';
baseFileName = 'w0lwro.jpg';
% Get the full filename, with path prepended.
fullFileName = fullfile(folder, baseFileName);
% Check if file exists.
if ~exist(fullFileName, 'file')
  % File doesn't exist -- didn't find it there.  Check the search path for it.
  fullFileName = baseFileName; % No path this time.
  if ~exist(fullFileName, 'file')
    % Still didn't find it.  Alert user.
    errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName);
    uiwait(warndlg(errorMessage));
    return;
  end
end
grayImage = imread(fullFileName);
% Get the dimensions of the image.  
% numberOfColorBands should be = 1.
[rows, columns, numberOfColorBands] = size(grayImage);
% Display the original gray scale image.
subplot(2, 2, 1);
imshow(grayImage, []);
title('Original Grayscale Image', 'FontSize', fontSize);
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]);
% Give a name to the title bar.
set(gcf,'name','Demo by ImageAnalyst','numbertitle','off') 

% Let's compute and display the histogram.
[pixelCount, grayLevels] = imhist(grayImage);
subplot(2, 2, 2); 
bar(pixelCount);
grid on;
title('Histogram of original image', 'FontSize', fontSize);
xlim([0 grayLevels(end)]); % Scale x axis manually.

binaryImage = grayImage < 128;
% Get rid of small blobs, smaller than two rows of pixels.
binaryImage = bwareaopen(binaryImage, 2*rows);
% Display the original gray scale image.
subplot(2, 2, 3);
imshow(binaryImage, []);
title('Binary Image', 'FontSize', fontSize);

% Label the image.
[labeledImage, numberOfBlobs] = bwlabel(binaryImage);
% Find the centroid
measurements = regionprops(labeledImage, 'Centroid');
% Put a cross at the centroid.
xCentroid = measurements.Centroid(1);
yCentroid = measurements.Centroid(2);
fprintf('X Centroid = %.3f, Y Centroid = %.3f', xCentroid, yCentroid);
hold on;
plot(xCentroid, yCentroid, 'r+', 'MarkerSize', 30);

% Find out how far the centroid is from points in each quadrant
% First get all the points.
[rows columns] = find(binaryImage);
xCorners = [0 0 0 0]; % X coordinate of corners in each quadrant.
yCorners = [0 0 0 0]; % X coordinate of corners in each quadrant.
maxDistance = [0 0 0 0]; % Distance of furthers X coordinate from centroid in each quadrant.
for k = 1 : length(columns)
  rowk = rows(k);
  colk = columns(k);
  distanceSquared = (colk - xCentroid)^2 + (rowk - yCentroid)^2;
  if rowk < yCentroid
    % It's in the top half
    if colk < xCentroid
      % It's in the upper left quadrant
      if distanceSquared > maxDistance(1)
        % Record the new furthest point in quadrant #1.
        maxDistance(1) = distanceSquared;
        xCorners(1) = colk;
        yCorners(1) = rowk;
      end
    else
      % It's in the upper right quadrant
      if distanceSquared > maxDistance(2)
        % Record the new furthest point in quadrant #2.
        maxDistance(2) = distanceSquared;
        xCorners(2) = colk;
        yCorners(2) = rowk;
      end
    end
  else
    % It's in the bottom half.
    if colk < xCentroid
      % It's in the lower left quadrant
      if distanceSquared > maxDistance(3)
        % Record the new furthest point in quadrant #3.
        maxDistance(3) = distanceSquared;
        xCorners(3) = colk;
        yCorners(3) = rowk;
      end
    else
      % It's in the lower right quadrant
      if distanceSquared > maxDistance(4)
        % Record the new furthest point in quadrant #4.
        maxDistance(4) = distanceSquared;
        xCorners(4) = colk;
        yCorners(4) = rowk;
      end
    end
  end
end
% Display in command window.
xCorners
yCorners
figure;
% Display the original gray scale image.
hImage = imshow(binaryImage, []);
title('Binary Image', 'FontSize', fontSize);
hold on;
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]);
% Place markers at the corners
plot(xCorners, yCorners,  'rs', 'MarkerSize', 10, 'LineWidth', 3);
impixelinfo(hImage);
% Plot centroid again
plot(xCentroid, yCentroid, 'r+', 'MarkerSize', 30, 'LineWidth', 3);

Results:

[Kal]

First of all thank you for the code. I changed the path and the image name ofcourse and kept everything else the same. The imhist function as well as the pixelCount function are giving me stubborn error messages. Ofcourse I don't want to modify imhist function because it is built in...is pixelCount a function you made? Do you have an m file I could use? Thank you so much!

[Image Analyst]

Just get rid of the imhist stuff - it's not really needed. It's just boilerplate demo code. Just for fun to compute a hist - not needed. What are the errors? It's not a color image is it?

[Anandakumar Selvaraj]

*hello sir i am a newbie i tried a code working well i have a similar task in my hand i need to do something like the image below were i marked points in orange color...help

i used same code and achieved this marked in blue

i need to achieve something like this image were i marked in orange

sir please say what to do...or show me what to do... *

[Image Analyst]

I'm not sure that algorithm is good for irregular blob-shaped regions. Why don't you start a new question and we'll discuss it there. In the mean time, try to use regionprops() on your binary image and then get the orientation and use imrotate() to align it with the axes.

[Anandakumar Selvaraj]

i have posted these question regarding my task in here

http://www.mathworks.com/matlabcentral/answers/109002-is-there-any-way-to-draw-a-vertical-and-horizontal-line-that-pass-through-blobs-centroid

http://www.mathworks.com/matlabcentral/answers/108988-how-to-locate-blobs-upper-middle-and-lower-middle-points-and-plot-it

[Image Analyst]

And I answered them here http://www.mathworks.com/matlabcentral/answers/109002#answer_117775

Answer 2

[Image Analyst]

Why not use hough() and look for line intersections? There is an example for it in the help.

[Kal]

There are so many imperfections on the edges, so it hough transform obtains all those small lines that make up the edges on the line.

Answer 3

[Jeff E]

The following may be helpful in validating any solution you do end up finding. If the rectangles in your images are rotated to any significant degree, just change "diamond" into "disk" to get a more robust, but slightly more noisy result.

bw = im2bw(imread('w0lwro.jpg'));
bw = ~bwareaopen(~bw, 5000);
cmask = imclose(bw, strel('diamond', 20));
cornermask = cmask & ~bw;
cornermask = bwareaopen(cornermask, 50);

The result is a binary mask that should contain, or be very close to, your corners, either through corner point detection or hough transform as suggested by Image Analyst.