Adaptive background Subtraction Algorithm

Hi every one

I have a problem while applying threshold into the Adaptive background Algorithm My code is

threshold=25
for f=1:frames              
   I=read(obj,f);   
   figure(1);imshow(I);title('Input Video Sequence');     
   if f==1    
     Background=I;    
     Fg=abs(I-Background);
   else
     Fg=abs(I-Background); 
   end
end
for i=1:width
  for j=1:height
    if Fg(i,j)>threshold
      Fg(i,j)=255;
    else
      Fg(i,j)=0;
    end  
  end
end
figure(2);imshow(Fg);

This is a video file in which I am applying this algorithm but when I apply the threshold conditions the results does not effect and I cannot get the foreground mask.

Thanks in advance

Answer 1

[Image Analyst]

After you get through the first loop over frames, FG will have the value of the last iteration because you keep overwriting it on each iteration and never do anything with it. So after that loop ends, FG is just the last frame minus the first frame. Clearly not what you want to do. Then you're thresholding FG, which is just the last difference like I said, and displaying it.

Put the background modification and the thresholding inside your loop over frames, and display inside that loop, or else assign a frame FG{f} to store all your difference frames. I added just a few lines to my demo (which I've posted before) to handle your adaptive background stuff. Copy, paste, and run this code:

% Demo macro to extract frames and get frame means from an avi movie
% and save individual frames to separate image files.
% Also computes the mean gray value of the color channels.
clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
imtool close all;  % Close all imtool figures.
clear;  % Erase all existing variables.
workspace;  % Make sure the workspace panel is showing.
fontSize = 14;

% Change the current folder to the folder of this m-file.
% (The line of code below is from Brett Shoelson of The Mathworks.)
if(~isdeployed)
  cd(fileparts(which(mfilename)));
end

% Open the rhino.avi demo movie that ships with MATLAB.
folder = fullfile(matlabroot, '\toolbox\images\imdemos');
movieFullFileName = fullfile(folder, 'rhinos.avi');
% Check to see that it exists.
if ~exist(movieFullFileName, 'file')
  strErrorMessage = sprintf('File not found:\n%s\nYou can choose a new one, or cancel', movieFullFileName);
  response = questdlg(strErrorMessage, 'File not found', 'OK - choose a new movie.', 'Cancel', 'OK - choose a new movie.');
  if strcmpi(response, 'OK - choose a new movie.')
    [baseFileName, folderName, FilterIndex] = uigetfile('*.avi');
    if ~isequal(baseFileName, 0)
      movieFullFileName = fullfile(folderName, baseFileName);
    else
      return;
    end
  else
    return;
  end
end

try
  videoObject = VideoReader(movieFullFileName)
  % Determine how many frames there are.
  numberOfFrames = videoObject.NumberOfFrames;
  vidHeight = videoObject.Height;
  vidWidth = videoObject.Width;

  numberOfFramesWritten = 0;
  % Prepare a figure to show the images in the upper half of the screen.
  figure;
%   screenSize = get(0, 'ScreenSize');
  % Enlarge figure to full screen.
  set(gcf, 'units','normalized','outerposition',[0 0 1 1]);
  
    % Ask user if they want to write the individual frames out to disk.
    promptMessage = sprintf('Do you want to save the individual frames out to individual disk files?');
    button = questdlg(promptMessage, 'Save individual frames?', 'Yes', 'No', 'Yes');
    if strcmp(button, 'Yes')
        writeToDisk = true;

          % Extract out the various parts of the filename.
          [folder, baseFileName, extentions] = fileparts(movieFullFileName);
          % Make up a special new output subfolder for all the separate
          % movie frames that we're going to extract and save to disk.
          % (Don't worry - windows can handle forward slashes in the folder name.)
          folder = pwd;   % Make it a subfolder of the folder where this m-file lives.
          outputFolder = sprintf('%s/Movie Frames from %s', folder, baseFileName);
          % Create the folder if it doesn't exist already.
          if ~exist(outputFolder, 'dir')
              mkdir(outputFolder);
          end
      else
          writeToDisk = false;        
      end

      % Loop through the movie, writing all frames out.
    % Each frame will be in a separate file with unique name.
    meanGrayLevels = zeros(numberOfFrames, 1);
    meanRedLevels = zeros(numberOfFrames, 1);
    meanGreenLevels = zeros(numberOfFrames, 1);
    meanBlueLevels = zeros(numberOfFrames, 1);
      for frame = 1 : numberOfFrames
      % Extract the frame from the movie structure.
      thisFrame = read(videoObject, frame);
      
          % Display it
          hImage = subplot(2, 2, 1);
          image(thisFrame);
          caption = sprintf('Frame %4d of %d.', frame, numberOfFrames);
          title(caption, 'FontSize', fontSize);
          drawnow; % Force it to refresh the window.    

    % Write the image array to the output file, if requested.
        if writeToDisk
             % Construct an output image file name.
            outputBaseFileName = sprintf('Frame %4.4d.png', frame);
            outputFullFileName = fullfile(outputFolder, outputBaseFileName);

              % Stamp the name and frame number onto the image.
              % At this point it's just going into the overlay, 
              % not actually getting written into the pixel values.
              text(5, 15, outputBaseFileName, 'FontSize', 20);

              % Extract the image with the text "burned into" it.
              frameWithText = getframe(gca);
              % frameWithText.cdata is the image with the text
              % actually written into the pixel values.
              % Write it out to disk.
              imwrite(frameWithText.cdata, outputFullFileName, 'png');
          end

    % Calculate the mean gray level.
    grayImage = rgb2gray(thisFrame);
    meanGrayLevels(frame) = mean(grayImage(:));

    % Calculate the mean R, G, and B levels.
    meanRedLevels(frame) = mean(mean(thisFrame(:, :, 1)));
    meanGreenLevels(frame) = mean(mean(thisFrame(:, :, 2)));
    meanBlueLevels(frame) = mean(mean(thisFrame(:, :, 3)));

    % Plot the mean gray levels.
    hPlot = subplot(2, 2, 2);
        hold off;
    plot(meanGrayLevels, 'k-', 'LineWidth', 2);
        hold on;
        plot(meanRedLevels, 'r-');
        plot(meanGreenLevels, 'g-');
        plot(meanBlueLevels, 'b-');
        grid on;
    
        % Put title back because plot() erases the existing title.
    title('Mean Gray Levels', 'FontSize', fontSize);
        if frame == 1
      xlabel('Frame Number');
      ylabel('Gray Level');
            % Get size data later for preallocation if we read
            % the movie back in from disk.
            [rows columns numberOfColorChannels] = size(thisFrame);
        end

          % Update user with the progress.  Display in the command window.
          if writeToDisk
          progressIndication = sprintf('Wrote frame %4d of %d.', frame, numberOfFrames);
          else
          progressIndication = sprintf('Processed frame %4d of %d.', frame, numberOfFrames);
          end
      disp(progressIndication);
      % Increment frame count (should eventually = numberOfFrames
      % unless an error happens).
      numberOfFramesWritten = numberOfFramesWritten + 1;
      
      % Now let's do the differencing
      alpha = 0.5;
      if frame == 1
        Background = thisFrame;
      else
        % Change background slightly at each frame
  %       Background(t+1)=(1-alpha)*I+alpha*Background
        Background = (1-alpha)* thisFrame + alpha * Background;
      end
      % Display the changing/adapting background.
      subplot(2, 2, 3);
      imshow(Background);
      title('Adaptive Background', 'FontSize', fontSize);
      % Calculate a difference between this frame and the background.
      differenceImage = thisFrame - uint8(Background);
      % Threshold with Otsu method.
      grayImage = rgb2gray(differenceImage); % Convert to gray level
      thresholdLevel = graythresh(grayImage); % Get threshold.
      binaryImage = im2bw( grayImage, thresholdLevel); % Do the binarization
      % Plot the binary image.
      subplot(2, 2, 4);
      imshow(binaryImage);
      title('Binarized Difference Image', 'FontSize', fontSize);
      end

      % Alert user that we're done.
      if writeToDisk
          finishedMessage = sprintf('Done!  It wrote %d frames to folder\n"%s"', numberOfFramesWritten, outputFolder);
      else
          finishedMessage = sprintf('Done!  It processed %d frames of\n"%s"', numberOfFramesWritten, movieFullFileName);
      end
      disp(finishedMessage); % Write to command window.
      uiwait(msgbox(finishedMessage)); % Also pop up a message box.

      % Exit if they didn't write any individual frames out to disk.
      if ~writeToDisk
          return;
      end

      % Ask user if they want to read the individual frames from the disk,
      % that they just wrote out, back into a movie and display it.
      promptMessage = sprintf('Do you want to recall the individual frames\nback from disk into a movie?\n(This will take several seconds.)');
      button = questdlg(promptMessage, 'Recall Movie?', 'Yes', 'No', 'Yes');
      if strcmp(button, 'No')
          return;
      end

      % Read the frames back in, and convert them to a movie.
      % I don't know of any way to preallocate recalledMovie.
      for frame = 1 : numberOfFrames
          % Construct an output image file name.
          outputBaseFileName = sprintf('Frame %4.4d.png', frame);
          outputFullFileName = fullfile(outputFolder, outputBaseFileName);
          % Read the image in from disk.
          thisFrame = imread(outputFullFileName);
          % Convert the image into a "movie frame" structure.
          recalledMovie(frame) = im2frame(thisFrame);
      end
      % Get rid of old image and plot.
      delete(hImage);
      delete(hPlot);
      % Create new axes for our movie.
      subPlot(1, 3, 2);
      axis off;  % Turn off axes numbers.
      title('Movie recalled from disk', 'FontSize', fontSize);
      % Play the movie in the axes.
      movie(recalledMovie);
      % Note: if you want to display graphics or text in the overlay
      % as the movie plays back then you need to do it like I did at first
      % (at the top of this file where you extract and imshow a frame at a time.)
      msgbox('Done with this demo!');

catch ME
  % Some error happened if you get here.
  stError = lasterror;
  strErrorMessage = sprintf('Error extracting movie frames from:\n\n%s\n\nError: %s\n\n)', movieFullFileName, stError.message);
  uiwait(msgbox(strErrorMessage));
end

[Algorithms Analyst]

Hello Image Analyst

I am very thankful to you for your whole effort.But,I did not find the assumption that differenceImage>threshold(which is called the foreground map) instead your are computing threshold from Otsu algorithm.Secondly in your code you are finding the normal difference instead of DifferenceImage=abs(thisframe-background). I dont no why could you please explain it? Thanks in Advance.

[Image Analyst]

Instead of this:

% Threshold with Otsu method.
  grayImage = rgb2gray(differenceImage); % Convert to gray level
  threshdLevel = graythresh(grayImage); % Get threshold.
  binaryImage = im2bw( grayImage, thresholdLevel); % Do the binarization

You can do this if you want:

binaryImage = grayImage > threshold; % or less than if you want things that didn't change.

[Algorithms Analyst]

Thanks alot.

[viki grover]

thanks a lot

[Ash]

Hi, @imageanalyst if we perform the background subtraction using the different component of HSV colour space, will it be same as you have done using RGB? Can we break background into the different components of HSV and subtract it with the corresponding component of current components. Hue component wrap around so subtracting it with background frame Hue component is a proper way?

[Image Analyst]

Subtracting will show where the hue is different. The resulting subtraction image will not make any sense in that the difference hue image is rather meaningless. The sole usefulness of doing the subtraction would be to determine where the image is different from a reference image.

[Ash]

Hi, @imageanalyst thank you so much for replying. Can you just check the code? I have split the background and current frame into the components of HSV. But I have notice that like you have that H and S are not giving any role when subtracting from the background whereas the result of the Value component is good. Can we make the hue and sat component result better also?

code:

HSV=rgb2hsv(Background);

H_BG = HSV(:,:,1);

S_BG = HSV(:,:,2);

V_BG = HSV(:,:,3);

HSV2=rgb2hsv(CurrentFrame);

H_CURR = HSV2(:,:,1);

S_CURR = HSV2(:,:,2);

V_CURR = HSV2(:,:,3);

alpha = 0.2;

H_BG = (1-alpha)* H_CURR + alpha * H_BG;

S_BG = (1-alpha)* S_CURR + alpha * S_BG;

V_BG = (1-alpha)* V_CURR + alpha * V_BG;

H_SUB = (H_BG - H_CURR)>0.4;

S_SUB = (S_BG - S_CURR)>0.2;

V_SUB = (V_BG - V_CURR)>0.06;

RES = (H_SUB & S_SUB & V_SUB);

[Image Analyst]

Off the top of my head it doesn't look like it should cause a problem. Though you might want to calculate Delta E using rgb2lab() instead. I'm traveling for the next 5 days starting very early tomorrow morning, so if you have any further questions, attach your images and code to a brand new question so we don't keep notifying this original poster of activity on a 4+ year old question.

[Ash]

hi thank you for replying, how can i refer you if i write a new post.

[Image Analyst]

Just explain everything from the beginning as if no one knew anything yet about your question.

[Shahriar Nasim]

hi Image Analyst bro... i m getting this error while extracting video at the end of the program. can u tell me how to fix this? i m searching on google but cannot find it.

[Image Analyst]

Was that from a line of code in the program I gave? If so, which line threw that error?

Answer 2

[Laura Proctor]

There are a couple things that I notice in this code. First, I'm not sure, but I think that you might want to apply the threshold to each frame, in which case, you should put the threshholding inside the for loop when reading each frame. Second, you can achieve this threshholding without using a for loop.

Try this:

threshold=25;
for f=1:frames              
   I=read(obj,f);   
   figure(1);imshow(I);title('Input Video Sequence');     
   if f==1    
     Background=I;    
     Fg=abs(I-Background);
   else
     Fg=abs(I-Background); 
   end
   Fg(Fg>threshold) = 255;
   Fg(Fg<=threshold) = 0;
   figure(2);imshow(Fg);
   drawnow;
   pause(0.2)
end

[Algorithms Analyst]

Yes you are right I wanted to apply thresholding on each frame.It works fine but when I apply the mathematical form of Adaptive Background subtraction equation below

Background(t+1)=(1-alpha)*I+alpha*Background

i can not find any changes even the foreground mask disappear I dont no why. But the concept is the same as I did it in my code. Background at a time t is the input video sequence of first frame so if f==1 then Backgroung=I else Background(t+1)=(1-alpha)*I+alpha*Background

so after applying this cannot fine the forgrond mask there is no mask in output.

[Algorithms Analyst]

Even though I am finding the foreground mask when f==1 from Fg=abs(I-background); But cannot find when using equation above.

[Nusrat jahan]

why i found error in :

for f=1:frames

undifined function or variable frames ?

[Image Analyst]

You have to determine the number of frames in your video before you call that code.

[matin]

could you please tell me what obj and f are in this code?

[Image Analyst]

f is the frame number that you want to read. obj is what you get returned when you call VideoReader. What you have is just a snippet that left out a fully working demo. See my answer below for several video processing demos.

Answer 3

[Image Analyst]

See several attached video processing demos.

Also see my File Exchange.

[Fego Etese]

Please Image Analyst, I need your help on this question https://uk.mathworks.com/matlabcentral/answers/512047-how-can-i-find-out-if-two-minutiae-points-in-a-fingerprint-image-are-on-the-same-ridge

Sorry for the bother, but i really need your help as I am lost.

Thanks.