Extract consecutive data points between 2 thresholds

For a project, i have data in the form of wind speed (m/s) recorded with 1 minute intervals, starting from 2009 through to 2021. For the time of each recording I have created a datetime array in the form of yyyy:MM:dd_HH:mm:ss, and for the wind speed recorded at each minute i have created a double array.

The wind speed ranges from 0 through to 40+ m/s. For the project, any wind speeds above 30m/s will cause damage to some machinery and hence, occurances where windspeed exceeds 30m/s must be recorded. Additionally, for each case where wind speed data exceeds 30m/s, i need to find the time taken (in minutes based on the datetime data) for the wind speed to go from 20m/s to the 30m/s limit. So the extracted data points between 20 and 30 m/s must be consequtive, which will allow me to find the time taken for cases where wind speed gradually increases from 20 to 30 m/s. Im not too fussed on the method used to extract the data with said conditions, so any sort of direction/help would be greatly appreciated.

[Walter Roberson]

If one reading was 19.5 and the next reading was 30, what would you want to have happen with the data? The peak wind might have been steadily increasing, but due to microcells or whatever, you could perhaps have a brief drop below 20 during a trend.

[Joshua Pretorius]

I think an approach to this is setting a minimum amount of time needed to be spent above or below a threshold to cause any significant affect to the trend. Lets say we have a range of time spanning over 10 minutes, where the first minute of this range is at 20m/s and the final minute of the range is at the 30m/s limit. Given that the fourth minute, for example, has a corresponding wind speed of 19.5 m/s, then that minute should essentially be ignored as the bulk of the readings sit between the 20-30 m/s margin. So for this example, the 10 minute period starting at 20 and ending at 30, with a single outlier, would still be recorded as taking 10 minutes to go from 20 to 30 m/s.

A way to look at it is that for damage to occur, the wind speed must be above 30 m/s for more than just a minute. So in effect, single outliers should not be deemed as detrimental to the trend, if that makes sense.

Thanks very much.

[Walter Roberson]

Should there be a sliding window (what width?) in which the condition is set true if mean() over the window is > 30, or median() over the windows is > 30, or if count of (data>30) is > 1 ?

[Joshua Pretorius]

Hi Walter, sorry for the late reply.

Im able to use a moving average to smoothen the data set and get rid of instances where the wind speed quickly drops below the threshold. Based on different trials, a moving average of 5-10 does a good job.

Answer 1

[KSSV]

Let t be your dates in datetime format and w be your wind data.

% get indices of wind lying in [20, 30]
idx = w>= 20 & w <= 30 ;  % results into logical indexing
t1 = t(idx) ;   % extract those dates 
w1 = w(idx) ;   % extract the wind data 

[Walter Roberson]

This does not handle the idea that the points must be consecutive.

[KSSV]

From the obtained dates, calculate the diff, wherever diff is 1, those indices can be picked.