Subset timetable by specific time windows/periods

I have a large dataset which looks like this:

The first column is datetimes and the first row is frequency bins. The data is dB values. (1,1) is irrelevant and could be 0/NA...

I then have a table of datetimes:

where the first column is the associated datetime variable from the large dataset, second is sunrise, third is end of dawn, fourth is beginning of dusk, and fifth is sunset. The first row is irrelevant and if possible I would change it to useful variable names.

I would like to subset the large dataset using these values.

For example, if the datetime (2,1) in the dataset falls between either duskdawntime(2,2:3) or duskdawntime (2,4:5), I would like to take the corresponding row of data from the large dataset and store it in a separate dusk or dawn table. Since these time windows change every day, it is important that each row of data is distinct i.e. we compare row 2 filename with row 2 sunrise time and row 2 sunset time.

(I calculated end of dawn and beginning of dusk by adding/removing 1hr from sunrise/sunset times but I may change the size of this window)

So far, I am thinking something like this:

filenumber=height(dataset);

dawnoutput=[]; %new empty arrays for dawn and dusk
duskoutput=[];


for i=1:filenumber %where filenumber is the data row, changes on each iteration
    
    filename=times(filenumber,1); %datetime of filename from first column in datetime matrix
    sunrise=times(filenumber,2); 
    dawnend=times(filenumber,3);
    duskstart=times(filenumber,4);
    sunset=times(filenumber,5);
    
    if (filename >= sunrise) && (filename <= dawnend)
        dawnoutput(filenumber,:)=noises(filenumber,:)
        
    elseif (filename >= dawnend) && (filename <= sunset)
           duskoutput(filenumber,:)=noises(filenumber,:)
            
            
    end
end

...but I am not sure if this is the most refined way.

Is this possible? Thanks in advance for any suggestions/help

Answer 1

[Peter Perkins]

This post is more than a year old, but I'm responding in hopes that it might help anyone who looks at this.

First, I don't understand what the 8x72000 table is supposed to represent. It has only 8 timestamps. Maybe that's just a subset, I don't know, but the problem as stated seems to imply that it should have as many rows as the datetime matrix. Also I would recommend not storing that first row, it's just going to cause problems. Set the var names to things like Freq1 or whatever, and remove the first row. Similarly, get rid of the first row in the datetime matrix, and convert that to a table with var names Sunrise, DawnEnd, etc. And it's not clear what the rows in that matrix contain; it appears that this has many timestamps from each day, and the sunrise/etc. times are repeated many times. That seems unnecessary.

Here's what I would do for how I interpret this question:

Assume you have one timetable containing the "every 10sec" noises (frequency binned?) data, and one timetable containing sunrise/etc. times for each day of interest (this timetable's rowtimes are the days). For each row of the noises timetable, get the day of the row time, then use those to index into the sunrise/etc. data, and compare the noise timestamps to the dawn dusk times. No loops needed.

day = dateshift(noiseTimeTbl.Time,'Start','day'); 
dawn = dawnduskTimeTbl(day).Sunrise <= noiseTimeTbl.Time & noiseTimeTbl.Time <= dawnduskTimeTbl(day).DawnEnd;
dawnNoiseTbl = noiseTimeTbl(dawn,:);
dusk = dawnduskTimeTbl(day).DuskStart <= noiseTimeTbl.Time & noiseTimeTbl.Time <= dawnduskTimeTbl(day).Sunset;
duskNoiseTimeTbl = noiseTimeTbl(dusk,:);