How to put NaN when concatenating *.dat files in a single matfile

I have a collection of (1 x 3) *.dat files that I need to combine into a single matfile (n x 3). I tried using this solution (by this user) which worked! However, I needed to preserve the row counts as it is imperative that I do so. I modified his code by addressing the size/dimension difference through:

if size(dataArray, 2) ~= 3
    dataArray = nan(1,3);
end

However, I cannot seem to address .dat files that seem to be either corrupted or empty. When running the loop, I get this error:

Error using dlmread (line 147)
Empty format character vector is not supported at the end of a file.

Error in csvread (line 48)
m=dlmread(filename, ',', r, c);

Error in untitled (line 19)
dataArray = csvread(fullFileName); % Or whatever function you want.

And upon checking the .dat files that show this error, all of them looks like this (See screenshot attached).

As much as I want to skip those files, I need to put NaN in the rows that they need to be. I have attached the full code below:

% Specify the folder where the files live.
myFolder = 'something'; % Wherever...
% Check to make sure that folder actually exists.  Warn user if it doesn't.
if ~isdir(myFolder)
  errorMessage = sprintf('Error: The following folder does not exist:\n%s', myFolder);
  uiwait(warndlg(errorMessage));
  return;
end
% Get a list of all files in the folder with the desired file name pattern.
filePattern = fullfile(myFolder, '*.dat'); % Change to whatever pattern you need.
theFiles = dir(filePattern);
for k = 1: length(theFiles)
  baseFileName = theFiles(k).name;
  fullFileName = fullfile(myFolder, baseFileName);
  fprintf(1, 'Now reading %s\n', fullFileName);
  % Now do whatever you want with this file name,
  % such as reading it in, plotting it, using it, or whatever.
  dataArray = csvread(fullFileName); % Or whatever function you want.
  
 if isempty(dataArray) || any(isnan(dataArray(:))) 
    continue; % Skip appending if this file is empty or has nans in it.
end
if size(dataArray, 2) ~= 3
    dataArray = nan(1,3);
end
 
  % Append data
  if k == 1
     allDataArray = dataArray;
  else
     allDataArray = [allDataArray; dataArray]; % Must have same number of columns
  end
end
xlswrite(outputFilename, allDataArray, 'All Data', 'A1');

[Image Analyst]

Make it easy for us to help you. Attach 3 files : two good ones and a bad one.

[Jeffrey Maloles]

I attached the files inside this compressed folder.

Answer 1

[Nihal]

Hello Jeffery,

I see that you are encountering difficulties when attempting to merge various data files. After reviewing your code, I noticed that the "csvread" function is generating errors for certain "BAD FILES." To address this issue, I recommend implementing a try-catch method to capture the error and insert "nan" rows for those specific files.

Here is the corrected code:

% Specify the folder where the files live.
myFolder = 'untitled folder'; % Wherever...
% Check to make sure that folder actually exists.  Warn user if it doesn't.
if ~isdir(myFolder)
  errorMessage = sprintf('Error: The following folder does not exist:\n%s', myFolder);
  uiwait(warndlg(errorMessage));
  return;
end
% Get a list of all files in the folder with the desired file name pattern.
filePattern = fullfile(myFolder, '*.dat'); % Change to whatever pattern you need.
theFiles = dir(filePattern);
for k = 1: length(theFiles)
  baseFileName = theFiles(k).name;
  fullFileName = fullfile(myFolder, baseFileName);
  fprintf(1, 'Now reading %s\n', fullFileName);
  % Now do whatever you want with this file name,
  % such as reading it in, plotting it, using it, or whatever.
  try 
  dataArray = csvread(fullFileName); % Or whatever function you want.
     if isempty(dataArray) || any(isnan(dataArray(:))) 
        continue; % Skip appending if this file is empty or has nans in it.
     end
  catch err
            dataArray = nan(1,3);
  end
  % Append data
  if k == 1
     allDataArray = dataArray;
  else
     allDataArray = [allDataArray; dataArray]; % Must have same number of columns
  end
end
xlswrite("res", allDataArray, 'All Data', 'A1');

For a more comprehensive understanding, I suggest referring to the following documentation: https://www.mathworks.com/help/matlab/ref/try.html

I hope this information proves helpful in resolving your concern.