FFT on Image Time Series

I am working on a stream of images ( time series), where the images are captured by camera after illuminating with a sinusoidally modulated light source, I am interested in the response of the object captured by camera to the illuminating light source. Am I to use a one dimensional fft on each pixel location in the time series? two dimensional fft on each pixel location in the time series? or am I to use two dimensional fft on the whole image

Answer 1

[Jonas]

if the object of your camera is more or less static and illuminated with a temporally modulated light source and you are interested in how the object changes its appearance over time, i would use a 1d fft on each pixel location over time. this results into e.g. 200x200 = 40000 complex valued ffts over time if you image is e.g. 200x200. you then have to think about in how to combine those spectra. if you are interested in magnitude, you may e.g. combine the spectra by using e.g. mean, median or max of the magnitude spectra.

take the abs first before using e.g. mean to avoid loss of information due to contrary phase (e.g. imagine two pixels changing values sinusoidally, but one starting black, the other white. due to the 180° phase difference the spectra would delete each other if you dont take abs before e.g. mean)

also delete the mean value of each (not all) Pixel value before fft, since the DC fft coefficient (the mean value, 0 Hz) is usually much bigger in magnitude than the non 0 Hz values. this can be done pixel wise by e.g. vid-mean(vid,3) if vid is a 3d matrix containing the images stream with temporal dimension along dimension 3

[faisal sani]

Hello, Jacob, I was able to come up with the code below, it looks a bit reasonable, however, I need to repeat it for every pixel, will I also be able to create an image again out of the fft values?

[faisal sani]

clc;

clear all;

image_folder = 'C:\Users\g44136fb\Documents\Cultivar-SMFI_02072022-1429\P49152_L smfi';

filenames = dir(fullfile(image_folder,'*.png'))

total_images = numel(filenames);

pixel_value = 1:total_images;

samples = 1:1:total_images;

for n = 1: total_images

f = fullfile(image_folder, filenames(n) .name);

our_images = imread(f);

pixel_values = our_images(600,600); %choosing arbitrary pixel location at 600 by 600

pixel_value(n) = pixel_values;

end

pixel_value_filt = filloutliers(pixel_value, 'linear'); %removing saturated pixels corrupted by ambient light

plot(samples,pixel_value_filt)

Y = abs(fft(pixel_value_filt))

figure

Fs=2.604

f = Fs*(0:(n/2))/n;

P = abs(Y/n).^2;

plot(f,P(1:n/2+1))

title('pixel in Frequency Domain')

xlabel('Frequency (f)')

ylabel('|P(f)|^2')

[Jonas]

'create an image from the fft values'

you won't get a 2d image but of course you can create a simple frequency plot like you did already in your code.

depending the memory you have available, you could first merge all images intoba big matrix, calculate all fft values using fft(stickedMatrix,numOfImages,3) to calculate fft of each pixel along time domain

[faisal sani]

Okay,

Thank you so much. So after creating the stickedMatrix and computing the fft for each pixel, can I create an image of say median or minimum magnitude of the spectrum?

[Jonas]

short demo

% use cameraman as exemplary image to create video
im=im2double(imread('cameraman.tif'));

% create video by sticking multiple instances along dimension 3
vid=repmat(im,[1 1 200]);

% create sinusoidal illumination change vector with frequency 10
iluChange(1,1,:)=0.1*sin(2*pi*10*linspace(0,1,200));

% add illumination change and some noise
vid=repmat(im,[1 1 200])+iluChange+0.05*rand(size(vid));
vid=vid-mean(vid,3);
% full lenght fft
pixelwiseFFT=abs(fft(vid,200,3));

% you could then calc e.g. the median abs value per layer

for layer=1:size(vid,3)
    currLayer=pixelwiseFFT(:,:,layer);
    finalResult(layer)=median(currLayer(:));
end

[Jonas]

this finalResult fft vector must be normalized etc and the frequency vector has to be formed, but i see in your own code that you can do that by your own

[faisal sani]

The part below, computes the pixel layer by layer, however, I want to compute a single pixel fft across all the layers, then repeat it for all the pixels

% you could then calc e.g. the median abs value per layer

for layer=1:size(vid,3)

currLayer=pixelwiseFFT(:,:,layer);

finalResult(layer)=median(currLayer(:));

end

[faisal sani]

This is the modification I have made, finally I am having variants of fft based images

finalResult_median = zeros(size(our_images_series,2),size(our_images_series,1));

finalResult_max = zeros(size(our_images_series,2),size(our_images_series,1));

finalResult_min = zeros(size(our_images_series,2),size(our_images_series,1));

k = 1; % counter variable in for loop

for r = 1:size(our_images_series, 1) % for number of rows of the image

for c = 1:size(our_images_series, 2) % for number of columns of the image

finalResult_median(k)= median(pixelwiseFFT(r,c,:));

finalResult_max(k) = max(pixelwiseFFT(r,c,:));

finalResult_min(k) = min(pixelwiseFFT(r,c,:));

k = k+1; % increment counter loop

end

end

[Faisal Sani Bala]

I am working on a stream of images ( time series), where the images are captured by camera after illuminating with a chirp (non-stationary sinusoidal) light source, I am interested in the response of the object captured by camera to the illuminating light source. Is it possible to develop a single image from the image series based on the wavelet transform for each pixel series?