roi mask and working on the masked image

Hi, I am working on a project, where i have to work with only the foreground, not the background, so iused color threshold, image segment app nothing worked, so finally i made a transparent background in alpha layer and stored the image usng imwrite. Now for my foreground analysis, i used the saved image but it calculates including transparent backgroun. I tried to use the imfreehand roi , i am able to make the roi, but cannot use the masked ROI for my further analysis. Even i freehand roi is not exactly masking the foreground, there are some features that missing. so i used this below mentioned code

img = double(imread('alphaimg1.png'))/255; 
imshow(img);
ROI = imfreehand;
img_mask = ROI(ROI ~=1)
Maximg = max(img_mask); % trying to find the max value of the roi image 
Minimg = min(img_mask); % trying to find the max value of the roi image
 hist(img_mask,0:0.01:1)
medMask = median(img_mask);
imshow(medMask)

It shows the following error

img_mask =

imfreehand with properties:

Deletable: 1

Error using max

Invalid data type. First argument must be numeric or logical.

Error in alphaimgtrial1 (line 54)

Maximg = max(img_mask); % trying to find the max value of the roi image

using this ROI, i have find median pixel value of the image and try to split the histogram like dark pixels less than median value set to median value and light pixels higher than median value set to median value.

How do I proceed? I am attaching an alpha layer transparent image.

Answer 1

[Walter Roberson]

img = im2double( imread('alphaimg1.png'));
imshow(img);
ROI = imfreehand;
img_mask = ROI(ROI ~=1);

roi_img = img(img_mask);
Maximg = max(roi_img);
Minimg = min(roi_img);
hist(roi_img, 0:0.01:1);
MedMask = median(roi_img);

imshow(MedMask) does not make sense to do, as the median will be a scalar. Perhaps you are thinking of median filtering on the image, such as medfilt2(). In that case, you have a problem with the masked pixels. One approach is:

dimg = img;             %must be double or single for this to work
dimg(~img_mask) = nan;
medimg = cast( nlfilt(dimg, [3 3], @(B) median(B(:), 'omitnan')), class(img) );
h = imshow(medimg);
h.AlphData = double(img_mask);    %0.0 for transparent, 1.0 for opaque

[Malini Bakthavatchalam]

Hi @ WalterRoberson : I am getting this error

Unable to use a value of type 'imfreehand' as an index.

Error in alphaimgtrial1 (line 69)

roi_img = img(img_mask);

[Walter Roberson]

img = im2double( imread('alphaimg1.png'));
himage = imshow(img);
ROI = imfreehand(himage);
img_mask = createMask(ROI, himage);

roi_img = img(img_mask);
Maximg = max(roi_img);
Minimg = min(roi_img);
hist(roi_img, 0:0.01:1);
MedMask = median(roi_img);

dimg = img;             %must be double or single for this to work
dimg(~img_mask) = nan;
medimg = cast( nlfilt(dimg, [3 3], @(B) median(B(:), 'omitnan')), class(img) );
himage = imshow(medimg);
himage.AlphData = double(img_mask);    %0.0 for transparent, 1.0 for opaque

[Malini Bakthavatchalam]

IS it good to use draw freehand? I feel i making mistke by using imfreehand(not exactly getting the borders of foreground). Professor also one more question the png attached here is file with alpha layer as background. so will the same code work with image without alpha background ?

[Malini Bakthavatchalam]

with the above mentioned code i am getting this error , I am not able to even draw ROI

Error using imfreehand>imfreehandAPI (line 154)

HPARENT must be able to have an hggroup object as a child.

Error in imfreehand (line 88)

[h_group,draw_api] = imfreehandAPI(varargin{:});

Error in alphaimgtrial1 (line 67)

ROI = imfreehand(himage);

[Malini Bakthavatchalam]

Now the drawfreehand function is working but it is showing error like

Undefined function or variable 'nlfilt'.

Error in alphaimgtrial1 (line 76)

medimg = cast( nlfilt(dimg, [3 3], @(B) median(B(:), 'omitnan')), class(img) );

[Walter Roberson]

https://www.mathworks.com/help/images/ref/nlfilter.html . I should have said nlfilter()

[Malini Bakthavatchalam]

Hi sir, I read about the nlfilter, and changed the code accordingly but still says error

img = im2double(imread('alphaimg1.png'));
himage = imshow(img);
ROI = drawfreehand
img_mask = createMask(ROI, himage);
roi_img = img(img_mask);
% Maximg = max(roi_img);
% Minimg = min(roi_img);
hist(roi_img, 0:0.01:1);
MedMask = median(roi_img);
dimg = img;             %must be double or single for this to work
dimg(~img_mask) = nan;
fun = @(B) median('(B(:)');
medimg = cast(nlfilter(dimg, [3 3], fun) ,('omitnan'),class(img));
himage = imshow(medimg);
himage.AlphData = double(img_mask); 
imshow(himage.AlphaData)  

It says following error ... i tried changing to colfilter, nothing works

Error using nlfilter

Expected input number 1, A, to be two-dimensional.

Error in nlfilter>parse_inputs (line 135)

validateattributes(a,{'logical','numeric'},{'2d'},mfilename,'A',1);

Error in nlfilter (line 52)

[a, nhood, fun, params, padval] = parse_inputs(args{:});

Error in alphaimgtrial1 (line 77)

medimg = cast(nlfilter(dimg, [3 3], fun) ,('omitnan'),class(img));

[Walter Roberson]

Ah... what do you expect median filter to mean for an rgb image?

[Malini Bakthavatchalam]

I wasnot sure why do we need nlfilter here. I wanted the masked foreground image median luminance, i wanted bascially to work with foreground image histogram, so wasnot sure why I need a 2D as the error suggests. My code of medMask = median(img_mask); imshow(medMask) is wrong , its just the number i need. I saw the example in nlfilter , montage, but i am not sure is that filtering is eccential for me.

[Walter Roberson]

rgb2gray and median with dimension 'all'

[Malini Bakthavatchalam]

Sir, The masked image is in complete white pixels and the background is in black right. can i get the masked foreground to color image (original face image) like if i use the imshow(maskedimage)- it shows only the foreground (face of the person)?

[Walter Roberson]

when you set the AlphaData of the imshow handle then the background will be transparent. https://www.mathworks.com/matlabcentral/answers/555028-roi-mask-and-working-on-the-masked-image#comment_914536 at the bottom does that

[Malini Bakthavatchalam]

@WalterRoberson: I kind of added one more line to the code but it still shows error in the code.

This is my code

img = im2double(imread('dkloriginalimgwithbcg.jpg')); 
himage = imshow(img);
ROI = drawfreehand
img_mask = createMask(ROI,img);
roi_img = img(img_mask);
%MedMask = median(roi_img);
dimg = img;             %must be double or single for this to work
dimg(~img_mask) = nan
%fun = @(B) median('(B(:)'); % median filter for mask
medimg = cast(dimg,'like',roi_img);% to get the mask to color image of face
himage = imshow(medimg); 
medimg.AlphaData = medimg; % alpha layer usage to make the background transparent
OMed = median(medimg);
imhist(medimg);
histogram(roi_img,0:0.01:1);
total_median = median(medimage);

error message

Unable to perform assignment because dot indexing is not supported for variables of

this type.

Error in alphaimgtrial1 (line 76)

medimg.AlphaData = medimg; % alpha layer usage to make the background transparent

After converting to trnasparent background I want to use the image foreground further for analysis, is that possible at this stage?

[Walter Roberson]

medimg is not the result of imshow() -- the result of imshow is himage .

medimg = cast(dimg,'like',roi_img);% to get the mask to color image of face

That is not a median image !

roi_img is uint8, which you do not want to cast() dimg into: you want to be casting it to the same class as img like I showed in

medimg = cast( nlfilt(dimg, [3 3], @(B) median(B(:), 'omitnan')), class(img) );

I wanted the masked foreground image median luminance

That value would not be a median image! That would be a scalar value!

I also notice that I was not extracting all three color panes for the ROI.

imhist(medimg);

Are you sure you want to be doing a histogram of a color image?

histogram(roi_img,0:0.01:1);

Are you sure you want to be doing a histogram of a color image?

I suspect that you want a histogram of the luminances.

img = im2double(imread('dkloriginalimgwithbcg.jpg')); 

clf

himage = imshow(img);
ROI = drawfreehand;
roi_mask = createMask(ROI,img);
roi_mask3 = repmat(roi_mask, [1 1 3]);

roi_img = nan(size(img));
roi_img(roi_mask3) = img(roi_mask3);

lum_img = rgb2gray(roi_img);
mean_luminance = mean(lum_img(roi_mask), 'all', 'omitnan');

subplot(1,3,1)
himg = imshow(roi_img);
himg.AlphaData = double(roi_mask);
title('masked image')

subplot(1,3,2)
imhist(roi_img(roi_mask3));
title({'histogram of color image within ROI', 'whatever that means'});

subplot(1,3,3)
imhist(lum_img(roi_mask));
title('histogram of luminance within ROI');

[Malini Bakthavatchalam]

thank you Sir Walter Roberson: this is the perfect masked image I want, My intention is to work with color image, the lightness distribution is already been done by my senior in gray level image. So my prof wants to look whether it works with color image. this special image is converted to DKL space, wheere the x axis is redgreen, and y axis is blue yellow and z axis is light dark. this is how DKL space cordinates were created. Now i have to work on histogram of the x & y axis, because in the histogram i am trying to work on rectify the image from the median value of the color image(because median is the middle value). All I am trying to do find the median value and pixels at lighter greys than the median are set to the median grey, while the other half are all lower in grey-level than the median.

[Walter Roberson]

Well, instead of

lum_img = rgb2gray(roi_img);

you would have something that looked sort of like

dkl_img = rgb2dkl(roi_img);
lum_img = dkl_img(:,:,3);

I have no idea whether anyone has implemented anything similar to rgb2dkl()

[Malini Bakthavatchalam]

@walterRoberson: No sir we cannot do rgb2DKL conversion. For this we need a matrix that is specifically created based on some calculations. I already had the conversion matrix so i did conversion from RGB to DKl, since DKLare not presentable in the monitor so I inversed the DKL matrix to get rgb image. MAy be your corrct I have to work on the DKL image and finally when all calculations is done i have to present in RGB form. This present picture is in RGB form already. I am interested in lightness distribution of color pictures.

ITs good that you asked me this question. So i have to work with the DKL space before converting to RGB. and finaly when all transition is done then i will inverse to get to rgb.

[Malini Bakthavatchalam]

Sir, Also I want to know why there is a roi_mask3 ? roi_mask3 = repmat(roi_mask, [1 1 3]) reason for this line ?

[Walter Roberson]

The code flow goes like this:

roi_mask = createMask(ROI,img);

The above creates a 2D mask -- roi_mask(J,K) true means that the RGB pixel img(J,K,:) is part of the image.

roi_mask3 = repmat(roi_mask, [1 1 3]);

The above "promotes" the 2D mask to 3D. Where roi_mask(J,K) was set, roi_mask3(J,K,1:3) will be set. 3 copies of the mask, one for the red layer, one for the green layer, one for the blue layer.

roi_img = nan(size(img));

img is RGB , rows x columns x 3, so roi_img is initialized to all nan, rows x columns x 3.

roi_img(roi_mask3) = img(roi_mask3);

The locations that are set in roi_mask3 are copied to roi_img.

Now, imagine that I had not promoted the mask to 3D, if I just had the 2D roi_mask . Then if you were to do

roi_img(roi_mask) = img(roi_mask)

then because roi_mask is acting as a logical index and roi_mask is only 1/3 as large as roi_img, at most 1/3 of the values could be copied between img and roi_img.

So the question then becomes how to apply the 2D roi_mask to each of the three color planes. You cannot do

roi_img(roi_mask,2) = img(roi_mask,2);     %WILL NOT WORK
roi_img(roi_mask,:,2) = img(roi_mask,:,2);  %WILL NOT WORK EITHER

To work pane-by-pane you would have to do

[r, c, p] = size(img);
roi_img = nan(r,c,p);
for P = 1 : p
    this_pane = img(:,:,p);
    temp = nan(r,c);
    temp(roi_mask) = this_pane(roi_mask);
    roi_img(:,:,P) = temp;
end

which is a pain. Far easier to just repmat() the mask to 3D so that it becomes the same size as the original image, after which you just do a simple logical indexing copy roi_img(roi_mask3) = img(roi_mask3)

[Malini Bakthavatchalam]

Wow this is a great explanation to beginners like me .... thanks a lot sir, I will work on the roi_mask image histogram and will let you know my results .

[Malini Bakthavatchalam]

img = im2double(imread('dkloriginalimgwithbcg.jpg')); 
 clf
 himage = imshow(img);
ROI = drawfreehand;
roi_mask = createMask(ROI,img);
roi_mask3 = repmat(roi_mask, [1 1 3]);
roi_img = nan(size(img));
roi_img(roi_mask3) = img(roi_mask3);
%lum_img = rgb2gray(roi_img);
% meanL = mean(lum_img(roi_mask), 'all', 'omitnan');
subplot(2,3,1)
himg = imshow(roi_img);
himg.AlphaData = double(roi_mask);
title('masked image')
subplot(2,3,2)
imhist(roi_img(roi_mask3));
title({'histogram of color image within ROI', 'whatever that means'});
 medianL = median(roi_img); % median pixel value of roi_img
 LHR = medianL; %(lower half rectified)
 UHR = medianL; %(upper half rectified)
 UHR(roi_img >= medianL) = medianL; % set image with half of the pixels lesser than the median value to be set to median value, & other lesser pixels than median are not modified 
 LHR(roi_img <= medianL) = medianL;% set image with half of the pixels lighter than the median value to be set to median value, so the other lighter pixels than median are not modified. 
 for i = 1:length(roi_img(:,1))
     j =1: length(roi_img(1,:))
     if LHR(i,j)<= medianL
         LHR(i,j)= roi_img(i,j);
     else UHR(i,j) = roi_img(i,j);
     end
 end
 subplot(2,3,3)
 imshow(UHR)
 subplot(2,3,4)
 imshow(UHR)
subplot(2,3,5)
imhow(LHR)
subplot(2,3,6)
imshow(LHR)

Sir, I am trying to do the rectification here, but it shows the error. I am also attaching an example which i did before, but in that image background pixels were getting calculated so the proper method to get image with rectification only in foreground.

This is my error

Unable to perform assignment because the left and right sides have a different number of elements.

Error in rectificationproper (line 39)

UHR(roi_img >= medianL) = medianL;

[Walter Roberson]

Once more you are treating medianL as if it were a median image. You have said before you wanted a median luminosity. You need to obtain the luminosities somehow, and take the median over all of that, getting back a scalar. It looks to me as if LHR and UHR are intended to work in the luminosity pane, and so should not be copying values from roi_img, which is a 3D array of RGB.

I already showed taking the mean of the brightness of the roi_img,

%lum_img = rgb2gray(roi_img);
% meanL = mean(lum_img(roi_mask), 'all', 'omitnan');

which is easily adjusted to

lum_img = rgb2gray(roi_img);
medianL = median(lum_img(roi_mask), 'all', 'omitnan');

and then your loop for LHR and UHR probably need to copy values from lum_img not roi_img.

 for i = 1:length(roi_img(:,1))
     j =1: length(roi_img(1,:))

Your j is a vector.

     if LHR(i,j)<= medianL

so LHR(i,j) is a vector. So you are comparing a vector of values to the scalar medianL . The if statement will be considered true only if all of the entries in the vector are <= medianL. The code you used will not process the body of the if statement only for the parts of LHR(i,:) that are <= medianL .

You should either be doing logical indexing or else you should be using two for loops.

If you switch to logical indexing you do not need loops at all there:

    mask = LHR <= medianL;
    LHR(mask) = lum_img(mask);
    

[Malini Bakthavatchalam]

Your explaining me in much better ways. I am just given this work, i am figuring out ways to do it. All i have to do it is in the color image and not in gray image. A senior in my lab already worked in grey levels and it worked fine for him, so I am given this work to see whether this kind of rectification works in color as well.

so I am using

     mask = LHR <= medianL;
    LHR(mask) = lum_img(mask);

to rectify my pixels, and now i want to see how the rectified face looks lik compared to the original. But in this case, my 1* 240960 double & lum_img(mask) is 502*480 logical array. I want to look at my rectified image using imshow. Also my LHR histogram involves value above median too. Which I dont want, i want to look the difference in the image with new rectifaction. But that is not happening in my histogram sir. Like the picture I attached before, it has three histograms... and images accordingly. I am looking at brightness level changes in the image with rectification code.

[Walter Roberson]

If you want to work with medians and a color image, then you need to define what it means to take the median of color. And since the median of color would have three components, you need to define what it means for a RGB value to be "above" or below that RGB median, since there are 8 possibilities.

Sigh, guessing again:

mask = lum_img < medianL;
mask3 = repmat(mask, [1 1 3]);
LHR = nan(size(img));
LHR(mask) = img(mask);
h = imshow(LHR);
h.AlphaData = double(mask);

[Malini Bakthavatchalam]

Sir, I have a thought, just want to share this to see whether it makes sense, for example if I convert my image to gray and do the rectification for my project, like looking at the lightness distribution in the gray scale and after all the rectifications, could i convert the rectified gray image to color will that make sense ?

[Walter Roberson]

could i convert the rectified gray image to color

If you convert an RGB to grayscale, and you modify the grayscale values, then you cannot convert back to RGB. There are on average 65536 different RGB colors that map to the same grayscale. (In practice, the distribution is not equal; there are over 200,000 different RGB colors that map to some of the darker grayscale colors; the brighter grayscale values have fewer RGB that can create them.)

If, however, you just use the grayscale value to determine which locations should belong to one set ("high") or another ("low"), then you are building a mask, and you can use that mask to select color locations. This is exactly what I do in the "Sigh, guessing again" code above: it uses the grayscale brightness to select RGB pixels as being below median brightness and displays only those pixels.

[Malini Bakthavatchalam]

@Walter Roberson: Sir, Now i spoke to my professor he suggested me like for example we have 3 color planes and three axis, can we get the histogram of x axis with the white point as the median? Like in my image x axis has the color range from red to green somewer there will be a middle white point. so that white point should be set as median. And he wants to look at the image and histogram of the axis...

[Walter Roberson]

Like in my image x axis has the color range from red to green somewer there will be a middle white point.

No, that is not true that there is a white point in the middle.

The green part starts with no blue component.

The red part has no blue component.

If there were hypothetically a white point in the middle, then because white is equal parts red, green, and blue, then you have two possibilities:

1. That the "white point" you refer to is a point with red, green, and blue components all 0, which would normally be called black, but you are calling it white because it has equal red, blue, and green (which is to say, none at all of any of them.) This is contrary to the common meaning of white.
2. Or... through some process, the transition from green to the hypothetical white point in the middle adds blue, and the transition from the hypothetical white point in the middle to red subtracts blue again. This would be... odd... but not impossible. It would have to be justified.