Segmenting an object inside image

I asked a similar question a while ago and im here again to ask about something similar.

I want to segment an object inside the image. I am using multithresholding to seperate the image into 3 parts but i am only interested in the middle part. Usually this middle part is the result from the second threshold value.

Here is my code:

fullFileName = '...'; % insert full image name
img = rgb2gray(imread(fullFileName));

Unrecognized function or variable 'fullFileName'.

originalimg=imread(fullFileName);
% smoothing the image because it usually helps a lot
img=imnlmfilt(img,'DegreeOfSmoothing',15,'SearchWindowSize',21,'ComparisonWindowSize',11);
img = imgaussfilt(img,3,'FilterSize',9);

thresh=multithresh(img,2);
thresh=[0,thresh,inf];

for k= 1:length(thresh)-1
mask=img>thresh(k) & img<thresh(k+1)-1;
caption=sprintf('Mask between %d and %d',thresh(k),thresh(k+1)-1);
mask =bwareaopen(mask,5000);
mask=imfill(mask,'holes');
mask=bwareafilt(mask,1); % get the biggest connected blob

if (k==2) % the region of interest

figure
BWshow=originalimg.*uint8(mask);
imshow(BWshow,[]);
axis('on','image');
title(caption,'Fontsize', fontSize,'Interpreter','None');
impixelinfo();

end

end

I know how to fill the black space inside the image: % Read in comments for actual correct code

%fill black space inside object
[rows, columns] = size(mask);
firstRows = zeros(1, columns);
lastRows = zeros(1, columns);
for col = 1 : columns
    fr = find(mask(:, col), 1, 'first')
    if ~isempty(fr)
        firstRows(col) = fr;
        lastRows(col) = find(mask(:, col), 1, 'last')
    end
end

My problem is with the extra pixels detected on top of the shadow. This is means it is a wrong detection. I highlighted the error in the attached image. What i want to do is similar to the "fill black space code" above but instead of "first" row i want to fill starting the second or third row for example.

Most of this code is from my previous question. My actual code is a bit different but for this question i am just using this.

[Image Analyst]

That code finds the top surface and bottom surface but doesn't actually fill any black space between. Anyway when you say "instead of "first" row i want to fill starting the second or third row" do you mean to just add 2 or 3 to firstRows?

firstRows = firstRows + 2;

and then to filling somehow?

[Swag Mane]

Not add x rows to firstRows. My question is unclear.

In 'result3.jpg' you can see some black areas inside the object. I could easily fill those since these areas are bound inside the object (find first row and bottom row then fill. Basically run the above program) but even then, in the middle part you can actually see my problem. Thresholding adds an x amount of pixels above my actual object. That is what i mean by 'shadow'. I labeled the whole area in orange. This is also what i meant by error in detection. It is adding the shadow and a couple of pixels above it. I want to remove everything above the shadow and the shadow itself if possible: the area in orange in result3.jpg

Dont look at the red boxes. Those black areas are usually not a big issue.

Oh yeah i forgot. I copy pasted your old comment instead of my actual code. Just add this after lastRows = ...

mask(firstRows : lastRows, col) = true;

The second code is not a general solution for every image but at least when i get a continuous black area that is 'bound' inside the object then i can fill it without having some errors like a horizontal black line along some columns of the image due to discontinuity.

Edit: Here is the actual code.

%fill black space inside object
[rows, columns] = size(mask);
firstRows = zeros(1, columns);
lastRows = zeros(1, columns);
for col = 1 : columns
	% Find the top most pixel.
	topRow = find(mask(:, col), 1, 'first');
	if ~isempty(topRow)
		% If there is a pixel in this column, then find the lowest/bottom one.
		bottomRow = find(mask(:, col), 1, 'last');    
        % Fill from top to bottom.
        mask(topRow : bottomRow, col) = true;
	end
end

[Image Analyst]

Still not sure what you mean by shadow.

clc;    % Clear the command window.
fprintf('Beginning to run %s.m ...\n', mfilename);
close all;  % Close all figures (except those of imtool.)
clearvars;
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 18;

fullFileName = 'image3.jpg'; % insert full image name
img = rgb2gray(imread(fullFileName));
originalimg=imread(fullFileName);
% smoothing the image because it usually helps a lot
img=imnlmfilt(img,'DegreeOfSmoothing',15,'SearchWindowSize',21,'ComparisonWindowSize',11);
img = imgaussfilt(img,3,'FilterSize',9);
subplot(2, 1, 1);
imshow(img, []);
title(fullFileName, 'FontSize', fontSize);
thresh=multithresh(img,2);
thresh=[0,thresh,inf];

% for k= 1:length(thresh)-1
	k = 2;
	mask=img>thresh(k) & img<thresh(k+1)-1;
	caption=sprintf('Mask between %d and %d',thresh(k),thresh(k+1)-1);
	mask =bwareaopen(mask,5000);
	mask=imfill(mask,'holes');
	mask=bwareafilt(mask,1); % get the biggest connected blob
	
% 	if (k==2) % the region of interest		
		subplot(2, 1, 2);
		imshow(img, []);
		title('Mask');
		BWshow=originalimg.*uint8(mask);
		imshow(BWshow,[]);
		axis('on','image');
		title(caption,'Fontsize', fontSize,'Interpreter','None');
		impixelinfo();
% 		break;
% 	end
	
% end

[Swag Mane]

I attached a perfect segmentation example done manually with the image semgmenter app in matlab. Notice the difference.

Back to the question:

Like this

What you showed me is exactly what i get in most images and is exactly my problem. The script is segmenting way too much above the actual outline of the object. I want the segmentation to follow the actual outline. If you look at the same image you showed me you will see that the bottom is segmented perfectly while the top is adding lots of pixels that dont belong to that object.

[Image Analyst]

This is a completely different image than the other that had three layers, each with a different gray level. For this image, you can simply threshold, like around 170 and keep the blob you want:

% Demo by Image Analyst.
clc;	% Clear the command window.
fprintf('Beginning to run %s.m ...\n', mfilename);
close all;	% Close all figures (except those of imtool.)
clearvars;
workspace;	% Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 20;

fileName = 'image3.jpg';
grayImage = imread(fileName);

% Get the dimensions of the image.
% numberOfColorChannels should be = 1 for a gray scale image, and 3 for an RGB color image.
[rows, columns, numberOfColorChannels] = size(grayImage)
if numberOfColorChannels > 1
	% It's not really gray scale like we expected - it's color.
	% Extract the red channel (so the magenta lines will be white).
	grayImage = grayImage(:, :, 1);
end

%--------------------------------------------------------------------------------------------------------
% Display the image.
subplot(2, 2, 1);
imshow(grayImage, []);
axis('on', 'image');
title('Original Gray Scale Image', 'FontSize', fontSize, 'Interpreter', 'None');
impixelinfo;
hFig = gcf;
hFig.WindowState = 'maximized'; % May not work in earlier versions of MATLAB.
drawnow;

% https://www.mathworks.com/matlabcentral/fileexchange/29372-thresholding-an-image?s_tid=srchtitle
lowThreshold = 170;
highThreshold = 255;
% https://www.mathworks.com/matlabcentral/fileexchange/29372-thresholding-an-image?s_tid=srchtitle
% [lowThreshold, highThreshold] = threshold(lowThreshold, highThreshold, grayImage);
mask = imfill(grayImage > lowThreshold, 'holes');
mask = bwareafilt(mask, [1000, 200000]); % Get rid of big upper blob and small noise blobs.
mask = bwareafilt(mask, 1); % Take largest of what's left.
% props = regionprops(mask, 'Area');
% allAreas = [props.Area]
% Display the image.
subplot(2, 2, 2);
imshow(mask, []);
axis('on', 'image');
title('Mask Image', 'FontSize', fontSize, 'Interpreter', 'None');
impixelinfo;

% Show masked image
maskedImage = grayImage;
maskedImage(~mask) = 0;
% Display the image.
subplot(2, 2, 3);
imshow(maskedImage, []);
axis('on', 'image');
title('Masked Image', 'FontSize', fontSize, 'Interpreter', 'None');
impixelinfo;

% Display the image.
subplot(2, 2, 3);
imshow(maskedImage, []);
axis('on', 'image');
title('Masked Image', 'FontSize', fontSize, 'Interpreter', 'None');
impixelinfo;

boundaries = bwboundaries(mask, 'noholes');
% Display the image.
subplot(2, 2, 4);
imshow(grayImage, []);
axis('on', 'image');
title('Original Image with Outline', 'FontSize', fontSize, 'Interpreter', 'None');
impixelinfo;
hold on;
b = boundaries{1};
xb = b(:, 2);
yb = b(:, 1);
plot(xb, yb, 'r-', 'LineWidth', 2);

[Swag Mane]

This doesnt work for the rest of the images. Specifying a fixed thresholding value is not what i asked for.

And by the way, the masked image or the mask itself is all i need. I just want to crop the object in the middle the image. The rest does not matter. Thanks for the effort anyway.

Answer 1

[yanqi liu]

sir，please check the follow code to get some information

for more information,you can read the book 《计算机视觉与深度学习实战》

clc; clear all; close all; 

fileName = 'https://ww2.mathworks.cn/matlabcentral/answers/uploaded_files/722444/image3.jpg';
im = imread(fileName);
if ndims(im) == 3
    im = rgb2gray(im);
end
bw = imbinarize(im,'adaptive','ForegroundPolarity','dark','Sensitivity',0.3);
bw = ~bw;
bt = bw;
% make target
bw = imclose(bw, strel('line', round(size(bw,2)*0.5), 0));
[L,num] = bwlabel(bw);
stats = regionprops(L);
for i = 1:num
    recti = stats(i).BoundingBox;
    if recti(3)>size(bw,2)*0.8 && recti(2)+recti(4)<size(bw,1)*0.8
    else
        bw(L==i)=0;
    end
end
bt = logical(bt.*bw);
% thin
bt = imclose(bt, strel('disk', 5));
bt = imopen(bt, strel('disk', 2));
bt = bwmorph(bt, 'thin', inf);
bt = imdilate(bt, strel('disk', 2));
% label image
im1 = im; im2 = im; im3 = im;
im1(bt) = 255; im2(bt) = 0; im3(bt) = 0;
imt = cat(3, im1, im2, im3);
figure; imshow(imt, []);