Animal detection in thermal images

Hello everyone. I'm a newbie to MATLAB and I'm kinda struck with a project which detects the animals from the given thermal image.

The tasks are as follows:

1. Detect the animals and mark it with bounding boxes along with its temperature
2. Enhancing the image when the contrast is not so good (background and the animal are not distinguishable )

For the above task, can someone plz suggest the methods applicable for detection. So far, I have tried to pre-process the images by masking technique so that I can improve the contrast of the image.

clc;clear;close;
img = imread('C:\Users\DIVYA MEENA\Desktop\1 (79).jpg');
img_gray = rgb2gray(img);
BW = imbinarize(img_gray);
maskedRgbImage = bsxfun(@times, img, cast(BW, 'like', img));
imshow(maskedRgbImage)

However, the above code dint help much.

Any help will be highly appreciated, Thanks in advance and sorry for the long question.

PS: I have attached my sample images.

Answer 1

[Image Analyst]

It's best if you can get a background image with no animals in it, then subtract that from the frame with animals (possibly) in it. You'll need to determine this dynamically since the no-animals background image may change with time as the room changes temperature. So maybe you could check for a change, and if there has been no change for, say, 15 minutes, then declare that frame as a background image and subtract it from future frames.

[S DIVYA MEENA]

Thank you so much Image Analyst for your time and reply. Following your idea, I got the results.

close all;
I1=imread('C:\Users\DIVYA MEENA\Desktop\flira.jpg');
I2=imresize(I1,[500 500]);
I3 = I2;%rgb2gray(I2);
I1=imread('C:\Users\DIVYA MEENA\Desktop\bg.jpg');
I2=imresize(I1,[500 500]);
I4 = I2;
J = imsubtract(I3,I4);
subplot(1,3,1), imshow(I3)
subplot(1,3,2), imshow(I4)
subplot(1,3,3), imshow(J)
title('image substraction');

My next part is to put bounding boxes around the animals or highlight the edges (boundary) of the animal , so that the animal gets highlighted (indicating detection).

Also I would want to find the temperature for each of the animals in the frame. Any help with this. Plz.

[Image Analyst]

That background image doesn't exactly look like the background without the animal in it.

See my Image Segmentation Tutorial in my File Exchange to find the objects and find bounding boxes.

[S DIVYA MEENA]

I couldnt get a much better background image. So , I tried a different technique -Gamma correction, for improving the contrast of image.

I = imread('C:\Users\DIVYA MEENA\Desktop\1 (27).jpg');
J = imadjust(I,[],[],3.5);
imshowpair(I,J,'montage')

Next I have to detect those three animals in the above image (and represent the detetion using bounding box or some other means).

Finally, I want to find the temperature of those animals individually. Any help sir. TIA

[Image Analyst]

See attached demo.

[S DIVYA MEENA]

Sir your code was very useful. Thank you.

By the way, I have one other issue. I found the following code to be applicable for my scenario.

I = imread('C:\Users\DIVYA MEENA\Desktop\c6.jpg');
I = rgb2gray(I);
whos I
range = [min(I(:)) max(I(:))]
figure
imshow(I,[])
colormap(gca,hot)
title('Original image')
smoothValue = 0.01*diff(range).^2;
J = imguidedfilter(I,'DegreeOfSmoothing',smoothValue);

figure
imshow(J,[])
colormap(gca,hot)
title('Guided filtered image')
thresh = multithresh(J,1)
L = imquantize(J,thresh);
L = imfill(L);

figure
imshow(label2rgb(L))
title('Label matrix from 3-level Otsu')
props = regionprops(L,I,{'Area','BoundingBox','MeanIntensity','Centroid'});

% Find the index of the background region.
[~,idx] = max([props.Area]);

figure
imshow(I,[])
colormap(gca,hot)
title('Segmented regions with mean temperature')
for n = 1:numel(props)
    % If the region is not background
    if n ~= idx
        % Draw bounding box around region
       rectangle('Position',props(n).BoundingBox,'EdgeColor','c')
       
       % Draw text displaying mean temperature in Celsius
       T = [num2str(props(n).MeanIntensity,3) ' \circ C'];
       text(props(n).Centroid(1),props(n).Centroid(2),T,...
           'Color','c','FontSize',12)
    end
end

Following figures are some of the output screenshots for the above code:

The above output is exactly what I expect. But I dont understand what that 125 C denotes? I wanted to know the body temperature of the animal that is detected within the bounding box. Does this mean, that dog has 125 C temperature? Too strange..!!

The input image captured by the Flir camera is given below.

From the above image (input), I guess that the body temperature of the dog is somewhere around 35 to 37C. Then why is the output image says it as 125 C.??

Can you plz clarify me on this sir??

[Image Analyst]

YOu'd have to check the FLIR camera manual but I thought the temperature is the temperature within the little circle in the crosshairs. Like the grass is 30.1 degrees. I don't know how the red image came up with 125. Maybe it gives the peak temperature within the box, though 125 is more than boiling water! So either the instrument is not calibrated properly, like you've inputted the wrong emissivity, or you've got one very hot dog!

[S DIVYA MEENA]

That cleared my doubt. Indeed, you are right. That 30.1 C is the temperature of grass or something and not that of Dogs'. And Big thanks for that insight you gave me on the "Emissivity" of the Flir camera. I'll check into that. Cuz the normal body temperature of dogs is 101 - 102.5 C (Just now got to know from Google LOL ). So, that temperature from input image is definetly wrong and might be that 125 C could be approximately right. Thanks much Sir :)

[S DIVYA MEENA]

Sir I'm still struggling with this bounding boxes..

In the following figures, how do I put separate bounding boxes for each of the animals.

All that I want is the hotter objects to be put in separate bounding boxes instead of one box covering the whole region. And also, I don't want that temperature at all. Its too erroneous. Could you plz plz modify the code for me (separate bouunding boxes and no temperature). I've attached the code.

Just in case, if you need the input image, I've attached that too (File name: input 1 and input 2)

And the contrast enhanced (Gamma correction) image is also attached ( File name: enhance 1 and enhance 2). The input which I passed for the above program is "Enhanced mage " and not the original image.

I'm waiting for your reply Sir.

[Image Analyst]

Threshold the image, then call regionprops() and ask for the bounding box.

binaryImage = temperatureImage > someValue;
props = regionprops(binaryImage, 'BoundingBox');
hold on; % Don't let boxes blow away image.
for k = 1 : length(props)
    thisBoundingBox = props(k).BoundingBox;
    rectangle('Position', thisBoundingBox);
end

[S DIVYA MEENA]

Sir.. I dont understand what that "temperatureImage" in your first line mean. If you could plz check the below code and help me,..

temperatureImage = imread('C:\Users\DIVYA MEENA\Desktop\c6.jpg');
binaryImage = temperatureImage > 23;
props = regionprops(binaryImage, 'BoundingBox');
hold on; % Don't let boxes blow away image.
for k = 1 : length(props)
    thisBoundingBox = props(k).BoundingBox;
    rectangle('Position', thisBoundingBox);
end

I assumed "temperatureImage" to be original image. Also the output is not displayed. All that I get is plain axes. How should I display the image with bounding box. Plz find the attached input image.

[Image Analyst]

Attach your background image also - the one with no animals in it, if you have one.

[S DIVYA MEENA]

Sir this is the only background image I have.

Furthermore, I have photos taken throughout the day (Dawn, noon and night time). Background will definetly change for each of the test images. How will then background subtraction be useful sir?

As of now, can you try something with the attached background image. ? If I get some more, I'll use it in place of this bg image.

[Image Analyst]

Unless you have strong shadows that will cause uneven temperature variation, you might be able to assume that all pixels in the background scene are just a scaled version of each other, like the noon image has all pixels with twice as much intensity as the night one, more or less. Then you can do some smart comparisons to try to find the "best" background image.

For the animals image, can we assume that the animals will not take up the majority of the field of view? If so, we can subtract all backgrounds from that and determine which background is closest to the animal background and use that.

Or can we just look at what's there and assume that if there are no animals, the background should be the same and if there are animals, you have a bright tail on the background histogram so you can use the triangle threshold (attached) to find out a threshold.

[S DIVYA MEENA]

Thank you so much Sir. That was a big help.

[S DIVYA MEENA]

Sir. One final question. Is there a way to directly find the temperature of the attached thermal image. I have the temperature readings in the image.

How should I find the temperature of the hottest object in my image? I have the code to detect the hottest spot in the image but I dont know how to read its temperature?

[Image Analyst]

I'm going to assume that you've integrated my prior code I attached into your code - the code where I took the image, extracted the image and the color bar, and produced a thermal image from that. So, once you've done that, you can find the highest temperature from

maxTemperature = max(thermalImage(:))
% Find areas a little cooler and hotter than that
threshold = maxTemperature - 4; % Whatever.
mask = thermalImage > threshold;
imshow(mask);

Actually, you'd be better off getting the one camera model better. I just spent all day with two FLIR engineers a couple of weeks ago and they said there is a better model for like a hundred dollars more that will give you the temperature image directly instead of a gray level image. It would be well worth it.

[S DIVYA MEENA]

Sir I have really messed up with all the code. I dont get as where to fix this piece of code. Can u plz give it as one complete code? If u don't mind..

[S DIVYA MEENA]

And thank you for the information on Flir camera. We just bought a refurbished Flir camera for 2.5L. Bad luck. I dint know about the other model earlier.

[Image Analyst]

I don't have enough time to donate it to you. I usually spend less than 5-10 minutes on a solution. It's your project so I think it's you who should get this going and just give me small parts that need fixing.

Can your camera give the actual thermal image out (great for you)? Or does it only give you basically a screenshot (bad for you)? Hopefully since it's refurbished you were able to get a nice advanced model for the same price as the almost useless entry model.

[Walter Roberson]

Google says the body temperature of dogs is roughly 101 F, not 101 C !!

[S DIVYA MEENA]

Thank you Image Analyst for your tremendous help. and yeah, ill stitch the code and hopefully will post it here back.

My model is flir e40. I get thermal images with temperature readings.

[S DIVYA MEENA]

Thank you for pointing out Walter. That was a big blunder..