How Can I load 200 images or more image frames via parfor loop?

My code is below and having the error that parfor loop can not be run.

At the end my image matrix should be : row x col x all frames...... not in a cell.

Thanks for the help !

% Define the range of images to load
start_idx = 276;
end_idx = 280;

% Start parfor loop
parfor i = start_idx:end_idx
    % Generate the file name based on the index 'i'
    filename = sprintf('_2023-07-13-13-46-12_%04d.tif', i);
    
    % Construct the full path to the image
    full_path = fullfile(image_dir, filename);
    
    % Check if the file exists
    if exist(full_path, 'file')
        % Read the image and store it in the 3D array
        tmp_img = imread(full_path);
        k = i - start_idx + 1
        images(:,:,k) = tmp_img(:,:,1);
    else
        disp(['Image ' num2str(i) ' not found.']);
    end
end

Answer 1

[Walter Roberson]

        k = i - start_idx + 1
        images(:,:,k) = tmp_img(:,:,1);

Inside parfor, the use of the loop control index must be explicit in the line that uses the parfor variables. parfor is not able to simulate the code to prove that k will be a reasonable value with no overlaps.

You will need to instead code

       images(:,:,i - start_idx + 1) = tmp_img(:,:,1);

[DGM]

Wouldn't there still be some other restrictions? I'm not remotely familiar with using parfor(), but even with these changes, it still complains about the indexing. I'm assuming that's because the array size is not consistent.

Is there even any practical advantage to using parfor() for file I/O? I guess I might as well try to answer that...

[DGM]

This is how I managed to get it to work.

% Define the range of images to load
start_idx = 1;
end_idx = 50;

% define some things
image_dir = 'bodyhuesweep1';
basefilename = 'character_%02d.png';

% preallocate on the following assumptions:
%  all images have the same geometry
%  all images have the same class
%  all images are single-channel grayscale 
filename = sprintf(basefilename,start_idx);
full_path = fullfile(image_dir, filename);
f1 = imread(full_path); % getting class from imfinfo is inconsistent
nframes = end_idx-start_idx+1;
sz0 = [size(f1,1:2) nframes];
images = zeros(sz0,class(f1));

tic
% Start parfor loop
goodframe = true(nframes,1);
fidx = start_idx:end_idx; % input frame (filename) indexes
parfor k = 1:nframes % these are output frame indexes
    % Generate the file name based on fidx(k)
    filename = sprintf(basefilename,fidx(k));
    
    % Construct the full path to the image
    full_path = fullfile(image_dir, filename);
    
    % Check if the file exists
    if exist(full_path, 'file')
        % Read the image and store it in the 3D array
        tmp_img = imread(full_path);
        images(:,:,k) = tmp_img(:,:,1); % output indexing is simplified
    else
        disp(['Image ' num2str(k) ' not found.']);
        goodframe(k) = false;
    end
end
% remove any blank frames from failed reads
images = images(:,:,goodframe);
toc

With 50 600x600 input images being read from SSD, the parfor loop takes ~30% longer to populate the output. So in my limited tests on junk hardware, I don't see any advantage to using parfor here. YMMV. I don't know what the difference would be if the output were not preallocated, but I'd expect it to be worse either way.

Given that the output is preallocated, any failed reads will result in blank pages in the output. Those need to be removed afterward by keeping track of the indices that had failed reads.

[Walter Roberson]

If you have a spinning disk (that is not a RAID) then optimal bandwidth is typically when there are either two or three simultaneous requests per drive. The idea is that while you wait for the proper sector to rotate into place to be read, you might potentially have an opportunity to read a sector for a different request. (The optimal strategy can vary if you have multiple read heads on the drive that at different angular positions.)

Likewise, the optimal per controller is typically two simultaneous drives. So instead of going serial on one drive on one controller channel, you can typically do better with two outstanding requests to two different drives on the same controller. Spinning drives are limited by rotation time, and by memory bandwidth once the data is read in.

But if you have two spinning drives and two controllers, you are usually better off putting each drive on its own controller.

With SSDs, as they do not have to spin and sector access is near-constant time, the optimization is different.

[DGM]

I did try it on disk as well as SSD, but the results were basically the same. Maybe the number of files or overall volume is too small to reveal the difference if any exists with my configuration.