Import multiple .txt files by dates from user input

Hi!

I have a folder with appr. 47000 txt-files. I would like to open only a few of them. I attach a screenshot of the folder and the filenames.

I have created an app with appdesigner where the user can select startdate/starttime and enddate/endtime.

Now I would like to open only the .txtfiles relevant to the user input.

What would be the best way to approach this?

I have been trying something like this:

foldername='%This is the name of my folder'; %adds the path of the archive
filepattern=fullfile(foldername, '*.txt);
D = dir(filepattern);

This would give me a struct with all .txt files.

What I would like to do is to select only the files matching periodDTstr below:

period = [tstart:tend]; %tstart and tend is datetime values from app
periodstr=datestr(period,'yyyymm');
periodstrday=datestr(period,'yyyymmdd');
w=unique(periodstr,'rows');
numfiles=length(w(:,1));
Starttime=datestr(timein,'HHMM'); %timein is user input from app
Stoptime=datestr(timeout,'HHMM'); %timeout is user input from app
StartDT=strcat(periodstrday(1,:),Starttime);
StopDT=strcat(periodstrday(end,:),Stoptime);
StartDTstr=datetime(StartDT,'InputFormat','yyyyMMddHHmm');
StopDTstr=datetime(StopDT,'InputFormat','yyyyMMddHHmm');
periodDT=[StartDTstr:hours:StopDTstr];
periodDTstr=datestr(periodDT,'yyyymmddHH');

Could you please help me with this?

Best regards

/Linus

Answer 1

[Stephen23]

"What would be the best way to approach this?"

Your approach of manipulating dates as strings is inefficient because it requires multiple class conversions.

Using DATESTR et al is discouraged in favor of more capable DATETIME and DURATION objects. Rather than creating a looooong list of every hour, it is much simpler and more efficient to use the inbuilt ISBETWEEN.

Try something like this:

DS = datetime(2021,06,09); % date start
DE = datetime(2021,06,20); % date end
TI = duration(7,0,0); % time in
TO = duration(9,0,0); % time out

Convert filenames to DATETIME:

P = '.'; % absolute/relative filepath to where the files are saved
S = dir(fullfile(P,'*.txt'));
D = regexp({S.name},'\d{10}','match','once');
T = datetime(D, 'InputFormat','yyMMddHHmm');

Filter filenames:

X = isbetween(T,DS,DE) % dates

X = 1×6 logical array
   0   1   1   1   1   0

Y = isbetween(timeofday(T),TI,TO) % time of day

Y = 1×6 logical array
   1   0   1   1   1   1

Z = S(X&Y); % Z is your output!
Z.name

ans = 'Hello_World_2106090800.txt'

ans = 'Hello_World_2106130800.txt'

ans = 'Hello_World_2106190800.txt'

[Linus Dock]

Hi! Many thanks!

I get the following error: Undefined function 'isbetween' for input arguments of type 'duration'.

I'm on version R2018b.

Otherwise I think i get the idea!

/Linus

[Stephen23]

ISBETWEEN was introduced in R2014b, but perhaps its functionality changed.

You can replace ISBETWEEN with logical comparisons:

X = T>=DS & T<=DE; % dates
tod = timeofday(T);
Y = tod>=TI & tod<=TO; % time of day

[Linus Dock]

Thank you very much Stephen!

/Linus