Loop files, create a timetable and add file names to variable names

Hi,

Wondering if anyone is familiar with how adding file names to the variable names in a loop when creating a timetable?

I have 800 files, each of them have the same number of columns but a different number of rows. Rows are time steps. Columns are variables.

When trying to run the below script I get an error ‘Duplicate variable name’, this happens when building up the final timetable as the variable names are the same. What I need is to add the file name to each column of the timetable, at each loop, plus the variable name Data1, Data2, Data3…. To get something like this…

Timetable…

 Time (column 0) %First column in timetable
 FileName1_Data1 (Variable Name column 1)
 FileName1_Data2 (Variable Name column 2)
 FileName1_Data3 (Variable Name column 3)
 FileName2_Data1 (Variable Name column 4)
 FileName2_Data2 (Variable Name column 5)
 FileName2_Data3 (Variable Name column 4)
 FileName3_Data1 (Variable Name column 6)
 FileName3_Data2 (Variable Name column 7)
 FileName3_Data3 (Variable Name column 8)
 ….
 FileName800_Data1 (Variable Name column ...)
 FileName800_Data2 (Variable Name column ...)
 FileName800_Data3 (Variable Name column ...)

Here is what I have done… …

    filePattern = fullfile(myFolder, '*.csv');
    theFiles = dir(filePattern); %800 files

    FinalM=[]; %Empty matrix

    %Create timetable T for synchronization
    t1 = datetime(2002,3,10,0,0,0); t2 = datetime(2017,12,31,0,0,0);
    Time_All = (t1:t2)';
    column=(1:1:5776)';
    T = timetable(Time_All,column);

 for k = 1 : length(theFiles)
    baseFileName = theFiles(k).name;
    fullFileName = fullfile(myFolder, baseFileName);
    fprintf(1, 'Now reading %s\n', fullFileName);

    %Create FileName which I do not know how to use to include in the Variables %(columns) name in the Final timetable
    FileName=strcat(theFiles(k).name(1:end-4));

    aa=csvread(fullFileName, 2,0);

    %Create time from first 6 columns (Year, month, …) 
    Time=datetime(aa(:,1),aa(:,2),aa(:,3),aa(:,4),aa(:,5),aa(:,6));

    %Create Data1, Data2, and Data3
    Data1=aa(:,7); 
    Data2=aa(:,8); 
    Data3=aa(:,9);

    Tab=timetable(Time,Data1, Data2, Data3);

    %Obtain some statistics
    Dmean=rmmissing(retime(Tab,'Daily',@nanmean));
    Drange=rmmissing(retime(Tab,'Daily',@range));

    %Synchronize the data based on the largest time series ‘Time_All’ created above
    FT=synchronize(Dmean, Drange,Time_All,'firstvalue');

    %Create final timetable
    FinalM=[FinalM innerjoin(FT,FT)];
 end

Any suggestions or comments are welcome.

Answer 1

[Walter Roberson]

After you create FT then

FT.Properties.VariableNames = strcat( FileName, '_', FT.Properties.VariableNames );

[Robert]

Thanks, Walter!

Answer 2

[Jose Marques]

Hello Robert.

If I understand correctly, you may try:

FileName=[theFiles(k).name(1) theFiles(k).name(2) theFiles(k).name(3) theFiles(k).name(4)];

[Robert]

Thanks, Jose!