Finding location with image processing method

clc; clear all
%% 1
I=imread('erkan_1.jpeg'); %Reading Image
gray=im2gray(I); %Converting RGB Image to Grayscale
J = imnoise(gray,'salt & pepper',0.01);
K = medfilt2(J);
%S=imsharpen(K);
figure,imshow(I)
title("Original Image")
% gray2=imadjust(gray);
figure; imhist(K) %Checking the histogram of the gray Image
title("Grayscale Histogram")
figure,imshow(gray)
title("Grayscale")
Binary = im2bw(gray,0.8); %Creating Binary Image with Threshold Value
% figure, imshowpair(Binary_2,Binary,"montage"),title("Grayscale vs Binary") %Showing Grayscale and Binary Photos
Binary=uint8(Binary); %Converting the Binary Photo to uint8 style to change matrix values
Binary(Binary>0) = 255; %Changing the black and white colours
figure,imshow(Binary)
Result=I+Binary; %We know that sum of the matrix value and 255 give us 255. Therefore we can use it.
Result(Result==255)=0; %Then changing black and white places again
% figure,imshow(Result) %Showing the result
 
se = strel('disk',1); %Creating a pattern to dilation
BW22 = imdilate(Binary,se); %Dilation to selected panels to see lines more smooth
Result_2=I+BW22;
Result_2(Result_2==255)=0;
text1='As seen, the image looks more clear with dilation.';
position1 =  [5 60];
RGB_1 = insertText(Result_2,position1,text1,'FontSize',15);%Implementing the text to image
% figure,imshow(Result) %Showing the result
figure, imshowpair(Result,RGB_1,"montage"),title("Before Dilation vs After Dilation")
%As seen, the image looks more clear with dilation
 
% COUNTING
afteropening=imopen(BW22,se); %It fills the holes
afterclosing=imclose(afteropening,se); %Deletes the small noises
[L,num]=bwlabel(afterclosing,4); %Taking number of the panels
position2 =  [1 50];
text2 = ['There are ',num2str(num),' solar panels which not work.']; 
RGB_2 = insertText(Result_2,position2,text2,'FontSize',18);%Implementing the text to image
figure, imshow(RGB_2)
title("RESULT")

clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 25;
%===============================================================================
% Get the name of the first image the user wants to use.
baseFileName = 'erkan_1.jpeg';
folder = fileparts(which(baseFileName)); % Determine where demo folder is (works with all versions).
fullFileName = fullfile(folder, baseFileName);
% Check if file exists.
if ~exist(fullFileName, 'file')
  % The file doesn't exist -- didn't find it there in that folder.
  % Check the entire search path (other folders) for the file by stripping off the folder.
  fullFileNameOnSearchPath = baseFileName; % No path this time.
  if ~exist(fullFileNameOnSearchPath, 'file')
    % Still didn't find it.  Alert user.
    errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName);
    uiwait(warndlg(errorMessage));
    return;
  end
end
%=======================================================================================
rgbImage = imread(fullFileName);
% Get the dimensions of the image.
[rows, columns, numberOfColorChannels] = size(rgbImage);
% Display the original image.
subplot(2, 3, 1);
imshow(rgbImage, []);
axis on;
caption = sprintf('Original Color Image, %s', baseFileName);
title(caption, 'FontSize', fontSize, 'Interpreter', 'None');
% Set up figure properties:
% Enlarge figure to full screen.
set(gcf, 'Units', 'Normalized', 'OuterPosition', [0 0.05 1 0.95]);
% Get rid of tool bar and pulldown menus that are along top of figure.
% set(gcf, 'Toolbar', 'none', 'Menu', 'none');
% Give a name to the title bar.
set(gcf, 'Name', 'Coordination', 'NumberTitle', 'Off')
drawnow;
hp = impixelinfo(); % Set up status line to see values when you mouse over the image.
[rows, columns, numberOfColorChannels] = size(rgbImage)
if numberOfColorChannels > 1
  % It's not really gray scale like we expected - it's color.
  % Use weighted sum of ALL channels to create a gray scale image.
%   grayImage = rgb2gray(rgbImage);
  % ALTERNATE METHOD: Convert it to gray scale by taking only the green channel,
  % which in a typical snapshot will be the least noisy channel.
  grayImage = rgbImage(:, :, 1); % Take red channel.
end
% Display the image.
subplot(2, 3, 2);
imshow(grayImage, []);
axis on;
title('Gray Image', 'FontSize', fontSize, 'Interpreter', 'None');
hp = impixelinfo();
drawnow;
% Display the histogram of the image.
subplot(2, 3, 3);
histogram(grayImage, 256);
title('Histogram of Gray Image', 'FontSize', fontSize, 'Interpreter', 'None');
grid on;
drawnow;
%=======================================================================================
binaryImage = grayImage < 240;
% Keep only the largest blob.
binaryImage = bwareafilt(binaryImage, 1);
% Display the masked image.
subplot(2, 3, 3);
imshow(binaryImage, []);
axis on;
title('Binary Image', 'FontSize', fontSize, 'Interpreter', 'None');
hp = impixelinfo();
drawnow;
% Compute the Euclidean Distance Transform
edtImage = bwdist(~binaryImage);
I = imread('Adsız.png');  % black/white image you provided which has RGB format

BW = logical(sum(I,3));     % nonzero pixels are set to 1, otherwise 0.
p = regionprops(BW,'Centroid');

imshow(BW)
hold on

mColors = {'r','g','cyan'};
arrayfun(@(x) ...
    plot(p(x).Centroid(1),p(x).Centroid(2),...
    'o','MarkerFaceColor',mColors{x},'MarkerSize',10)...
    ,1:numel(p))

[Erkan Karaoglu]

Dear @Image Analyst @Turlough Hughes . Can you help me please ?

[Erkan Karaoglu]

I'm short of time to submit my project. I tried hard but I couldn't.

[Image Analyst]

How do you want to show them? It looks like you already are, by having a map with a big white spot where the object of interest is. Is that now what you want? Do you want a big red crosshairs centered on the blob centroid? Please be specific.

[Erkan Karaoglu]

No, I want to know the position of the panel. For example; If there are 30 panels in the image, I want to know where this defective panel is. I developed the code as follows. I determined the position of the panels according to the value range. I click on the image and type its location manually. I want to get automatic results or learn if there is a different solution.

clc; clear all
I=imread('Tuzak.jpg'); %Reading Image
gray=im2gray(I); %Converting RGB Image to Grayscale
J = imnoise(gray,'salt & pepper',0.01);
K = medfilt2(J);
%S=imsharpen(K);
figure,imshow(I)
title("Original Image")
% gray2=imadjust(gray);
figure; imhist(K) %Checking the histogram of the gray Image
title("Grayscale Histogram")
figure,imshow(gray)
title("Grayscale")
Binary = im2bw(gray,0.8); %Creating Binary Image with Threshold Value
% figure, imshowpair(Binary_2,Binary,"montage"),title("Grayscale vs Binary") %Showing Grayscale and Binary Photos
Binary=uint8(Binary); %Converting the Binary Photo to uint8 style to change matrix values
Binary(Binary>0) = 255; %Changing the black and white colours
figure,imshow(Binary)
Result=I+Binary; %We know that sum of the matrix value and 255 give us 255. Therefore we can use it.
Result(Result==255)=0; %Then changing black and white places again
% figure,imshow(Result) %Showing the result
 
se = strel('disk',1); %Creating a pattern to dilation
BW22 = imdilate(Binary,se); %Dilation to selected panels to see lines more smooth
Result_2=I+BW22;
Result_2(Result_2==255)=0;
text1='As seen, the image looks more clear with dilation.';
position1 =  [5 60];
RGB_1 = insertText(Result_2,position1,text1,'FontSize',15);%Implementing the text to image
% figure,imshow(Result) %Showing the result
figure, imshowpair(Result,RGB_1,"montage"),title("Before Dilation vs After Dilation")
%As seen, the image looks more clear with dilation
 
% COUNTING
afteropening=imopen(BW22,se); %It fills the holes
afterclosing=imclose(afteropening,se); %Deletes the small noises
[L,num]=bwlabel(afterclosing,4); %Taking number of the panels
position2 =  [1 50];
text2 = ['Toplam Olarak ',num2str(num),' FV Hücre Çalışmıyor']; 
RGB_2 = insertText(Result_2,position2,text2,'FontSize',18);%Implementing the text to image
figure, imshow(RGB_2)
title("RESULT")

labeledImage = bwlabel(Binary, 8);
subplot(3, 3, 4);
imshow(labeledImage, []);  % Show the gray scale image.

% Let's assign each blob a different color to visually show the user the distinct blobs.
coloredLabels = label2rgb (labeledImage, 'hsv', 'k', 'shuffle'); % pseudo random color labels
% coloredLabels is an RGB image.  We could have applied a colormap instead (but only with R2014b and later)
subplot(3, 3, 5);
imshow(coloredLabels);
axis image; % Make sure image is not artificially stretched because of screen's aspect ratio.

xy = input('X ve Y koordinatlarını giriniz = : ')
x = xy(1)
y = xy(2)
if 0 < x && x < 58 && y < 120
	uiwait(msgbox('1. Panel Arızalı'))
elseif 0 < x && x < 58 && y > 120
	uiwait(msgbox('16. Panel Arızalı'))
end	
if 58 < x && x < 118 && y < 118
	uiwait(msgbox('2. Panel Arızalı'))
elseif 58 < x && x < 118 && y > 118
	uiwait(msgbox('17. Panel Arızalı'))
end	
if 118 < x && x < 179 && y < 118
	uiwait(msgbox('3. Panel Arızalı'))
elseif 118 < x && x < 179 && y > 118
	uiwait(msgbox('18. Panel Arızalı'))
end	
if 179 < x && x < 238 && y < 113
	uiwait(msgbox('4. Panel Arızalı'))
elseif 179 < x && x < 238 && y > 113
	uiwait(msgbox('19. Panel Arızalı'))
end	
if 247 < x && x < 306 && y < 111
	uiwait(msgbox('5. Panel Arızalı'))
elseif 247 < x && x < 306 && y > 111
	uiwait(msgbox('20. Panel Arızalı'))
end	
if 306 < x && x < 371 && y < 106
	uiwait(msgbox('6. Panel Arızalı'))
elseif 306 < x && x < 371 && y > 106
	uiwait(msgbox('21. Panel Arızalı'))
end	
if 371 < x && x < 434 && y < 106
	uiwait(msgbox('7. Panel Arızalı'))
elseif 371 < x && x < 434 && y > 106
	uiwait(msgbox('22. Panel Arızalı'))
end	
if 434 < x && x < 498 && y < 101
	uiwait(msgbox('8. Panel Arızalı'))
elseif 434 < x && x < 498 && y > 101
	uiwait(msgbox('23. Panel Arızalı'))
end	
if 498 < x && x < 566 && y < 101
	uiwait(msgbox('9. Panel Arızalı'))
elseif 498 < x && x < 566 && y > 101
	uiwait(msgbox('24. Panel Arızalı'))
end	
if 566 < x && x < 629 && y < 94
	uiwait(msgbox('10. Panel Arızalı'))
elseif 566 < x && x < 629 && y > 94
	uiwait(msgbox('25. Panel Arızalı'))
end	
if 629 < x && x < 697 && y < 94
	uiwait(msgbox('11. Panel Arızalı'))
elseif 629 < x && x < 697 && y > 94
	uiwait(msgbox('26. Panel Arızalı'))
end	
if 697 < x && x < 765 && y < 91
	uiwait(msgbox('12. Panel Arızalı'))
elseif 697 < x && x < 765 && y > 91
	uiwait(msgbox('27. Panel Arızalı'))
end	
if 765 < x && x < 831 && y < 89
	uiwait(msgbox('13. Panel Arızalı'))
elseif 765 < x && x < 831 && y > 89
	uiwait(msgbox('28. Panel Arızalı'))
end	
if 848 < x && x < 911 && y < 86
	uiwait(msgbox('14. Panel Arızalı'))
elseif 848 < x && x < 911 && y > 86
	uiwait(msgbox('29. Panel Arızalı'))
end	
if 911 < x && x < 979 && y < 91
	uiwait(msgbox('15. Panel Arızalı'))
elseif 911 < x && x < 979 && y > 91
	uiwait(msgbox('30. Panel Arızalı'))
end	

[Image Analyst]

I still don't understand. regionprops() can give you both the centroid and bounding box of all blobs in the image "automatically". That is what you said you want, so why is there still a problem? Also you forgot to attach 'Tuzak.jpg'.

And why are you adding noise to your original image? That can't help.

[Erkan Karaoglu]

I'm new to MATLAB and trying to learn. How can I optimize this code? The white ones are faulty and all I want is a code to tell which panel the whites are on. For example; There is a white dot on the 15th panel. Result: Your 15th panel is broken.

[DGM]

What can be used as a spatial reference? Is a small amount of user-interaction acceptable? Is the camera position and framing always repeatable? Are we only concerned with the central rack of 18 panels, or do we need to map all panels in view? Is it of any concern to identify which parts of the panel were bad, or just the entire panel?

Answer 1

[Image Analyst]

Tons of stuff wrong with your code. Too much to explain. Just try this and study what it does. It's fairly well commented.

clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clear;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 15;

rgbImage=imread('Tuzak.jpg'); %Reading Image
hFig1 = figure;
subplot(3, 2, 1);
imshow(rgbImage)
axis('on', 'image');
grayImage = im2gray(rgbImage); %Converting RGB Image to Grayscale
% grayImage = rgb2ind(rgbImage, hot(256));
impixelinfo;
title("Original Image", 'FontSize', fontSize)
% gray2=imadjust(gray);
subplot(3, 2, 2);
imhist(grayImage) %Checking the histogram of the gray Image
grid on;
title("Grayscale Histogram", 'FontSize', fontSize)
g = gcf;
g.WindowState = 'maximized';

subplot(3, 2, 3);
imshow(grayImage)
axis('on', 'image');
impixelinfo;
title("Grayscale Image", 'FontSize', fontSize)
drawnow;

% Binarize the image.
binaryImage = imbinarize(grayImage, 0.8);
subplot(3, 2, 4);
imshow(binaryImage)
axis('on', 'image');
title("Binary Image", 'FontSize', fontSize)
drawnow;

% Make measurements of Area and bounding box
props = regionprops(binaryImage, 'Area', 'BoundingBox', 'Centroid');
allAreas = [props.Area]
allBB = vertcat(props.BoundingBox);
xy = vertcat(props.Centroid);
% Show them in the overlay
numBlobs = length(props)
hold on;
for k = 1 : numBlobs
	plot(xy(k, 1), xy(k, 2), 'r+', 'MarkerSize', 30, 'LineWidth', 2);
	rectangle('Position', allBB(k, :), 'EdgeColor', 'y');
end

labeledImage = bwlabel(binaryImage, 8);
subplot(3, 2, 5);
imshow(labeledImage, []);  % Show the gray scale image.
axis('on', 'image');
impixelinfo;
caption = sprintf('Labeled Image with %d Hot Spots', numBlobs);
title(caption, 'FontSize', fontSize)
drawnow;

% Let's assign each blob a different color to visually show the user the distinct blobs.
coloredLabels = label2rgb (labeledImage, 'hsv', 'k', 'shuffle'); % pseudo random color labels
% coloredLabels is an RGB image.  We could have applied a colormap instead (but only with R2014b and later)
subplot(3, 2, 6);
imshow(coloredLabels);
axis('on', 'image');
impixelinfo;
axis('on', 'image'); % Make sure image is not artificially stretched because of screen's aspect ratio.
caption = sprintf('%d Colorized Labels', numBlobs);
title(caption, 'FontSize', fontSize)

% xy = input('X ve Y koordinatlarını giriniz = : ')
hold on;
for k = 1 : numBlobs
	x = xy(k, 1);
	y = xy(k, 2);
	% Put a big circle around it.
	plot(xy(k, 1), xy(k, 2), 'ro', 'MarkerSize', 20, 'LineWidth', 2);
	fprintf('The panel centered at (x,y) = (%.1f, %.1f) is hot.\n', x, y);
	if 0 < x && x < 58 && y < 120
		uiwait(msgbox('1. Panel Arızalı'))
	elseif 0 < x && x < 58 && y > 120
		uiwait(msgbox('16. Panel Arızalı'))
	end
	if 58 < x && x < 118 && y < 118
		uiwait(msgbox('2. Panel Arızalı'))
	elseif 58 < x && x < 118 && y > 118
		uiwait(msgbox('17. Panel Arızalı'))
	end
	if 118 < x && x < 179 && y < 118
		uiwait(msgbox('3. Panel Arızalı'))
	elseif 118 < x && x < 179 && y > 118
		uiwait(msgbox('18. Panel Arızalı'))
	end
	if 179 < x && x < 238 && y < 113
		uiwait(msgbox('4. Panel Arızalı'))
	elseif 179 < x && x < 238 && y > 113
		uiwait(msgbox('19. Panel Arızalı'))
	end
	if 247 < x && x < 306 && y < 111
		uiwait(msgbox('5. Panel Arızalı'))
	elseif 247 < x && x < 306 && y > 111
		uiwait(msgbox('20. Panel Arızalı'))
	end
	if 306 < x && x < 371 && y < 106
		uiwait(msgbox('6. Panel Arızalı'))
	elseif 306 < x && x < 371 && y > 106
		uiwait(msgbox('21. Panel Arızalı'))
	end
	if 371 < x && x < 434 && y < 106
		uiwait(msgbox('7. Panel Arızalı'))
	elseif 371 < x && x < 434 && y > 106
		uiwait(msgbox('22. Panel Arızalı'))
	end
	if 434 < x && x < 498 && y < 101
		uiwait(msgbox('8. Panel Arızalı'))
	elseif 434 < x && x < 498 && y > 101
		uiwait(msgbox('23. Panel Arızalı'))
	end
	if 498 < x && x < 566 && y < 101
		uiwait(msgbox('9. Panel Arızalı'))
	elseif 498 < x && x < 566 && y > 101
		uiwait(msgbox('24. Panel Arızalı'))
	end
	if 566 < x && x < 629 && y < 94
		uiwait(msgbox('10. Panel Arızalı'))
	elseif 566 < x && x < 629 && y > 94
		uiwait(msgbox('25. Panel Arızalı'))
	end
	if 629 < x && x < 697 && y < 94
		uiwait(msgbox('11. Panel Arızalı'))
	elseif 629 < x && x < 697 && y > 94
		uiwait(msgbox('26. Panel Arızalı'))
	end
	if 697 < x && x < 765 && y < 91
		uiwait(msgbox('12. Panel Arızalı'))
	elseif 697 < x && x < 765 && y > 91
		uiwait(msgbox('27. Panel Arızalı'))
	end
	if 765 < x && x < 831 && y < 89
		uiwait(msgbox('13. Panel Arızalı'))
	elseif 765 < x && x < 831 && y > 89
		uiwait(msgbox('28. Panel Arızalı'))
	end
	if 848 < x && x < 911 && y < 86
		uiwait(msgbox('14. Panel Arızalı'))
	elseif 848 < x && x < 911 && y > 86
		uiwait(msgbox('29. Panel Arızalı'))
	end
	if 911 < x && x < 979 && y < 91
		uiwait(msgbox('15. Panel Arızalı'))
	elseif 911 < x && x < 979 && y > 91
		uiwait(msgbox('30. Panel Arızalı'))
	end
end
uiwait(msgbox('Done!'))
fprintf('Done!  Thanks Image Analyst!\n');

[Erkan Karaoglu]

First of all, thank you very much. I have some questions in my mind. I searched but couldn't find the source.

1. How did numBlobs result in 5 ?

2. [for k = 1 : numBlobs

plot(xy(k, 1), xy(k, 2), 'r+', 'MarkerSize', 30, 'LineWidth', 2);

rectangle('Position', allBB(k, :), 'EdgeColor', 'y');

end] what is the working logic of this nolum ?

3.[for k = 1 : numBlobs

x = xy(k, 1);

y = xy(k, 2);

% Put a big circle around it.

plot(xy(k, 1), xy(k, 2), 'ro', 'MarkerSize', 20, 'LineWidth', 2);

fprintf('Cell at position (x,y) = (%.1f, %.1f) of panel has hotspot.\n', x, y);

if 0 < x && x < 58 && y < 120

uiwait(msgbox('Panel 1 Failed'))

elseif 0 < x && x < 58 && y > 120

uiwait(msgbox('16. Panel Failed'))

end] what is the working logic of this section? Can you explain please?

[Image Analyst]

numBlobs is the length of the structure array returned by regionprops. one structure for every blob. To me it looks like there should be 5. How many do you think are there?

plot('r+') puts a red cross hairs at the centroid of the object.

rectangle() puts up a box around it.

Not sure what you don't understand in 3. fprintf() types stuff to the command window. uiwait(msgbox) pops up a dialog box for the user to click on. plot() puts a red circle at the centroid of the blob. Your code checking x and y location is to determine what panel number is bad/hot.

[Erkan Karaoglu]

Sir, I know what you are saying. I was asking to understand what kind of logic you used when creating the algorithm.

[Image Analyst]

I'm not sure what you're asking then. In general the logic/process is

1. Process your image somehow until you can get an image that can be thresholded.
2. Threshold that image into foreground and background to get a binary image.
3. Call regionprops() to make measurements on the foreground blobs.
4. Add any graphics or overlays to indicate what you found.
5. Export the measurements to an Excel workbook or a text file.

Of course Step 1 is highly dependent on what kind of image you're starting with and what you want to measure.

See my well commented Image Segmentation Tutorial in my File Exchange:

Image Analyst's File Exchange

Answer 2

[DGM]

Given that this was probably done by drone, I'm going to assume that framing and camera position are not practically repeatable. I'm going to assume that we're only interested in one particular rack per image.

Consider the example:

panelsperrack = [2 9]; % expected rack layout

% read image
inpict = rgb2gray(imread('erkan.jpg'));
s = [size(inpict,1) size(inpict,2)];

% manually get location of this rack
% just specify the rack extents with 4 points
imshow(inpict)
P = drawpolygon();
rpos = P.Position;

% need to enforce vertex ordering to maintain orientation
d = rpos-mean(rpos);
a = atan2(-d(:,2),d(:,1));
[~,idx] = sort(a,'ascend');
rpos = rpos(idx,:);

% get transform
fpos = [0 500; 1000 500; 1000 0; 0 0];
TF = fitgeotrans(rpos,fpos,'projective');

% transform and crop the image to the control point extents
[rpict outref] = imwarp(inpict,TF);
[x y] = transformPointsForward(TF,rpos(:,1),rpos(:,2));
[x y] = worldToIntrinsic(outref,x,y);
cpos = [min(x) min(y) max(x)-min(x) max(y)-min(y)];
rpict = imcrop(rpict,cpos); % rectified ROI image

% do whatever you need to do to isolate the hot spots
% i'm just going to do something simple for sake of example
hotpict = rpict>200;

% find row-major linear index of bad panel for each hot object
S = regionprops(hotpict,'centroid');
C = vertcat(S.Centroid);
badrow = ceil(panelsperrack(1)*C(:,2)/size(rpict,1));
badcol = ceil(panelsperrack(2)*C(:,1)/size(rpict,2));
badidx = sub2ind(fliplr(panelsperrack),badcol,badrow)

The output is:

badidx =
    15
    15
    15

Which could be handled however you choose for reporting purposes.