How to load and and open multiple files in one loop?

Hello! I have 24 data files (spectral reflection from various substrates from using TriOS) which contain spectral info about my chosen substrates and these come in pairs witth the first file being a white plate calibration file and straight after that its the substrate reflection. I am having a little bit of trouble writing my loops as I would like to load them all in at once and process like that but I am struggling with figuring this out, any help is much appreciated! I will paste what I have so far below, thanks!

You can see some descriptions % below, thank you for taking the time to have a look, is there a way to load all my files in at once? Here also is an example of the naming format : _Spectrum_Calibrated_2021-03-22_15-29-49_788_SAM_811b_plate.dat

I keep getting the following error too and I am unsure how to fix it i tried using fopen but then it had an issue with textscan:

Error using fgets
Invalid file identifier. Use fopen to generate a valid file identifier.

I wanted to attach sample files but the format isnt supported, they are in .dat

Thank you so much in advance!:)

folderData =  'E:\mar518\';
filePattern = fullfile(folderData, '*dat'); 
datfiles = dir(filePattern);
nFiles = length(datfiles);

for i = 1:nFiles
    filename = fullfile(datfiles(i).folder, datfiles(i).name);
end

fid = fopen(Ftarget1ref,'r');  % open the file and assign a file handle
for i=1:43; fgets(fid); end;  % reads 42 lines from the file and does not save them - this is the header data.
for i=1:196
    tmp = fgets(fid);  % get the next line as a string
    tmp2 = textscan(tmp,'%f%f%d%d') % break the line up into it's 4 column entries - they are now 'cell' format
    wv(i) = cell2mat(tmp2(1)); % conver the wavelength at this line into a number and store it
    reference1(i) = cell2mat(tmp2(2)); % convert the radiance at this line into a number and store it
end
fclose(fid);  % close the file

Answer 1

[Benjamin Großmann]

I do not see the definition of Ftarget1ref. Moreover, the variable "filename" is overwritten in every step of the for loop.

I recommend the following for batch file processing:

1) Write a function that does all the reading/loading for one file. A file name is the functions input:

function [wv, reference] = spectrumFileReader(fileName)
    fid = fopen(fileName,'r');  % open the file and assign a file handle
    for i=1:43; fgets(fid); end;  % reads 42 lines from the file and does not save them - this is the header data.
    for i=1:196
        tmp = fgets(fid);  % get the next line as a string
        tmp2 = textscan(tmp,'%f%f%d%d') % break the line up into it's 4 column entries - they are now 'cell' format
        wv(i) = cell2mat(tmp2(1)); % conver the wavelength at this line into a number and store it
        reference1(i) = cell2mat(tmp2(2)); % convert the radiance at this line into a number and store it
    end
    fclose(fid);  % close the file
end

2) Test the function using one file as input

fileName = '_Spectrum_Calibrated_2021-03-22_15-29-49_788_SAM_811b_plate.dat';
[wv, reference] = spectrumFileReader(fileName)

If everything works for a single file, proceed with step 3 to process all of your files.

3) Use a cell array of file names and cellfun or a for loop to process all of your files

3.1 Using cellfun

folderData =  'E:\mar518\';
filePattern = fullfile(folderData, '*dat'); 
datfiles = dir(filePattern);

% create a cell array of file names
fileNameCell = fullfile({datfiles.folder}, {datfiles.name});

% use cellfun to read all of the files in fileNameCell
[wv_cell, reference_cell] = cellfun(@(fileName) spectrumFileReader(fileName), fileNameCell, 'UniformOutput', false);

3.2 With for loop

%% Alternative using for loop:
% wv = {};
% reference = {};
%
% for fileName = fileNameCell
%     [wv{end+1}, reference{end+1}] = spectrumFileReader(fileName);
% end

[Daryna Butash]

Amazing thank you so much for your help I was getting a bit confused and this has definitely cleared that and it works pretty well thanks again!:)

[Benjamin Großmann]

You are welcome!