Can I use blob detection to detect my hand?

I want to detect my hand from a video, and I've used the blob detection code from ImageAnalyst's file exchange to try and do it, but it won't work. Can someone tell me where I'm going wrong? Or suggest some other method for hand detection? Here's the code I've used:

clc;
clear all;
close all;

closepreview;
vid = videoinput('winvideo', 1, 'YUY2_640x480'); %specify the video adaptor
src = getselectedsource(vid);
vid.ReturnedColorspace = 'grayscale'; %define the color format to GRAYSCALE
vid.FramesPerTrigger = 5;
preview(vid); %preview the video object

while(1)
preview(vid); %preview the video object
currentimg=getsnapshot(vid); %capture the image of interest

grayImage = currentimg;
% Get the dimensions of the image. numberOfColorBands should be = 1.
[rows columns numberOfColorBands] = size(grayImage);

% Binarize the image.
binaryImage = im2bw(grayImage,0.58);

binaryImage = imfill(binaryImage, 'holes');
binaryImage = bwareaopen(binaryImage, 8000);

[labeledImage numberOfBlobs] = bwlabel(binaryImage, 8);

% Get all the blob properties.
blobMeasurements = regionprops(labeledImage, 'BoundingBox','Area');
allBlobAreas = [blobMeasurements.Area];

% Display the original gray scale image.
imshow(grayImage, []);

% Loop through all blobs, putting up Bounding Box.
hold on; % Prevent boxes from blowing away the image and prior boxes.
for k = 1 : numberOfBlobs
boundingBox = blobMeasurements(k).BoundingBox;   % Get box.
x1 = boundingBox(1);
y1 = boundingBox(2);
x2 = x1 + boundingBox(3) - 1;
y2 = y1 + boundingBox(4) - 1;
verticesX = [x1 x2 x2 x1 x1];
verticesY = [y1 y1 y2 y2 y1];
plot(verticesX, verticesY);

end
end

Thank you!

Answer 1

[Image Analyst]

"it won't work" is not very descriptive. Can you attach an image/snapshot that it did not work for. You may need to adjust your threshold.

[Anjana Rao]

It only detects parts of my face, or pieces of the background, but never my hand. And every time there's a bounding box that encompasses the entire video frame, like in the picture below, so I can't find the biggest blob either. The image on the right is the video from the webcam, and the one on the left shows the bounding box.

[Image Analyst]

Can you attach the original image? Maybe you'd prefer to adapt my activecontour demo (attached below).

[Anjana Rao]

It's video input from a webcam. Is there something wrong with the threshold value? I tried a lot of values, but it still didn't work.

And Thank you for the code! I'll try it out!

[Image Analyst]

Probably. Can't you just use imwrite to write the snapshot out to disk. Otherwise I'll have to get your screen cpature and create an image out of it with Photoshop. Make it easy for us to help you,not hard!

[Anjana Rao]

I'm sorry, I didn't think of that. Is this okay?

[Image Analyst]

Try this:

function test1()
clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clear;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 20;

% Check that user has the Image Processing Toolbox installed.
hasIPT = license('test', 'image_toolbox');
if ~hasIPT
  % User does not have the toolbox installed.
  message = sprintf('Sorry, but you do not seem to have the Image Processing Toolbox.\nDo you want to try to continue anyway?');
  reply = questdlg(message, 'Toolbox missing', 'Yes', 'No', 'Yes');
  if strcmpi(reply, 'No')
    % User said No, so exit.
    return;
  end
end

% Read in a standard MATLAB gray scale demo image.
folder = 'C:\Users\Ajnana\Documents\Temporary';
baseFileName = 'snap.jpg';
% Get the full filename, with path prepended.
fullFileName = fullfile(folder, baseFileName);
% Check if file exists.
if ~exist(fullFileName, 'file')
  % File doesn't exist -- didn't find it there.  Check the search path for it.
  fullFileNameOnSearchPath = baseFileName; % No path this time.
  if ~exist(fullFileNameOnSearchPath, 'file')
    % Still didn't find it.  Alert user.
    errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName);
    uiwait(warndlg(errorMessage));
    return;
  end
end
grayImage = imread(fullFileName);
% Get the dimensions of the image.  
% numberOfColorBands should be = 1.
[rows, columns, numberOfColorBands] = size(grayImage);
if numberOfColorBands > 1
  % It's not really gray scale like we expected - it's color.
  % Convert it to gray scale by taking only the green channel.
  grayImage = grayImage(:, :, 2); % Take green channel.
end
% Display the original gray scale image.
subplot(2, 3, 1);
imshow(grayImage, []);
title('Original Grayscale Image', 'FontSize', fontSize);
% Enlarge figure to full screen.
set(gcf, 'Units', 'Normalized', 'OuterPosition', [0 0 1 1]);
% Give a name to the title bar.
set(gcf, 'Name', 'Demo by ImageAnalyst', 'NumberTitle', 'Off') 

% Let's compute and display the histogram.
[pixelCount, grayLevels] = imhist(grayImage);
subplot(2, 3, 2); 
bar(grayLevels, pixelCount);
grid on;
title('Histogram of original image', 'FontSize', fontSize);
xlim([0 grayLevels(end)]); % Scale x axis manually.

% Get the gradient image to find the edges of the hand.
gradientImage = imgradient(grayImage);
% Put a white line along the bottom, right, 
% and left side to "close off" the hand and arm.
gradientImage(:,1) = 255; % Make column 1 white.
gradientImage(end,:) = 255; % Make last row white.
gradientImage(:,end) = 255; % Make last column white.
% Display the image.
subplot(2, 3, 3);
imshow(gradientImage, []);
axis on;
title('gradient Image', 'FontSize', fontSize);

% Let's compute and display the histogram.
[pixelCount, grayLevels] = hist(gradientImage(:), 100);
subplot(2, 3, 4); 
bar(grayLevels, pixelCount);
grid on;
title('Histogram of gradient image', 'FontSize', fontSize);
xlim([0 grayLevels(end)]); % Scale x axis manually.

binaryImage = gradientImage > 60;
% Fill Holes
binaryImage = imfill(binaryImage, 'holes');
% Remove tiny regions.
binaryImage = bwareaopen(binaryImage, 1000);
% Display the image.
subplot(2, 3, 5);
imshow(binaryImage, []);
title('Binary Image', 'FontSize', fontSize);

%---------------------------------------------------------------------------
% Extract the largest area using our custom function ExtractNLargestBlobs().
% This is the meat of the demo!
biggestBlob = ExtractNLargestBlobs(binaryImage, 1);
% Display the image.
subplot(2, 3, 6);
imshow(biggestBlob, []);
title('Final Image', 'FontSize', fontSize);
%---------------------------------------------------------------------------

%==============================================================================================
% Function to return the specified number of largest or smallest blobs in a binary image.
% If numberToExtract > 0 it returns the numberToExtract largest blobs.
% If numberToExtract < 0 it returns the numberToExtract smallest blobs.
% Example: return a binary image with only the largest blob:
%   binaryImage = ExtractNLargestBlobs(binaryImage, 1)
% Example: return a binary image with the 3 smallest blobs:
%   binaryImage = ExtractNLargestBlobs(binaryImage, -3)
function binaryImage = ExtractNLargestBlobs(binaryImage, numberToExtract)
try
  % Get all the blob properties.  Can only pass in originalImage in version R2008a and later.
  [labeledImage, numberOfBlobs] = bwlabel(binaryImage);
  blobMeasurements = regionprops(labeledImage, 'area');
  % Get all the areas
  allAreas = [blobMeasurements.Area];
  if numberToExtract > 0
    % For positive numbers, sort in order of largest to smallest.
    % Sort them.
    [sortedAreas, sortIndexes] = sort(allAreas, 'descend');
  elseif numberToExtract < 0
    % For negative numbers, sort in order of smallest to largest.
    % Sort them.
    [sortedAreas, sortIndexes] = sort(allAreas, 'ascend');
    % Need to negate numberToExtract so we can use it in sortIndexes later.
    numberToExtract = -numberToExtract;
  else
    % numberToExtract = 0.  Shouldn't happen.  Return no blobs.
    binaryImage = false(size(binaryImage));
    return;
  end
  % Extract the "numberToExtract" largest blob(a)s using ismember().
  biggestBlob = ismember(labeledImage, sortIndexes(1:numberToExtract));
  % Convert from integer labeled image into binary (logical) image.
  binaryImage = biggestBlob > 0;
catch ME
  errorMessage = sprintf('Error in function ExtractNLargestBlobs().\n\nError Message:\n%s', ME.message);
  fprintf(1, '%s\n', errorMessage);
  uiwait(warndlg(errorMessage));
end

%

[Anjana Rao]

Thank you so much! It worked! Really, thank you for your time and patience.

One last question: Why do I get the error message "Index exceeds matrix dimensions" a little after I run the program for video? It works fine at the beginning.

[Image Analyst]

No one can say without looking at code. Put a stop in your catch statement and then investigate variables.

[Anjana Rao]

Okay! Thank you again!

Answer 2

[zhang]

mark