How to calculate the quantiles/percentiles values for each hour?

I calculate with this code the quantile points of all the data.

Data=[1.5 2.3 2.5 4 6.2 7.1 8];
First_data = quantile(Data, [0 0.25 0.50 0.75 1])';
QuantilePoints = {'Minimum', '25th percentile', '50th percentile', '75th percentile', 'Maximum'}';
T = table(QuantilePoints, First_data);
disp(T)

How can I calculate the quantile points for every hour? The hour is in the datetime format like below.

Time_datenum={'20-Apr-2020 11:06:00','20-Apr-2020 11:20:10','20-Apr-2020 11:45:30','20-Apr-2020 12:07:00','20-Apr-2020 12:35:40','20-Apr-2020 12:40:50','20-Apr-2020 13:07:00'};
Time_datetime = datetime(Time_One,'InputFormat','dd-MM-yyyy HH:mm:ss');

Answer 1

[dpb]

Use grouping variables and splitapply or put into a table or timetable and then rowfun and/or retime (timetable) are also options.

Earlier today I showed an example on another variable, but principles still apply <Splitting-data-array-into-sub-arrays#>

[Kofial]

Can you show smth more concrete? I didn't see much in common with the link you suggested. I need not only the median but also 25% and 75 % for every hour.

[dpb]

What's not in common? Calculating based on grouping variables. The only thing really any different at all is that your function has two outputs, but there's an example of that at the documentation for rowfun

Your grouping variable will, of course, be hours() of your datetime date variable

Give it a go and see -- you can get farther than you think you can and you'll learn a lot more by doing than just picking up somebody else's code as well...

[Maria Jacob]

I have a similar issue, and I must confess that I haven't loooked at it into too much detail. However, I tried using retime to calculate the 5th percentile on my timetable, the problem is, retime can only use functions that accept empty inputs, like median (that you show on the example on the link). I will look into rowfun, but if you have any suggestions...

[dpb]

>> prctile([],5)
ans =
   NaN
>> 

accepts an empty array; just returns NaN...

[Maria Jacob]

Thank you. I know prctile can take an empty array, but you still need to tell it the % hat you want to calculate, so in that sense it's not an empty input. From what I understand, and given that it gives me an error when I try to use it, retime only takes function that accept not putting any parameter inside the ().

What I will try now is creating a custom function that calls prctile to calculate the % I want. I'll let you guys knows if it works...

[dpb]

Let's see your code and the returned error -- I'm not buying it being it impossible (yet, anyways... :) )

You will need to use an anonymous function as the functional input to retime that supplies the wanted percentile(s), yes, you can't just use @retime by itself, no. (I had thought that obvious so hadn't mentioned it explicitly...I suppose owing to too much familiarity myself and presuming too much for newcomers.)

ADDENDUM:

OK, I just tried it with one of the example datasets for timetable(after I made the initial time span a couple days longer) and

>> ttP=retime(TT2,'daily',@(x)prctile(x,5))
ttP =
  3×3 timetable
    MeasurementTime     Temp     Pressure    WindSpeed
    _______________    ______    ________    _________
    18-Dec-2015        37.523     29.997      6.5199  
    19-Dec-2015        40.037     29.936      6.7343  
    20-Dec-2015        41.541     29.898      7.1103  
>> ttP=retime(TT2,'daily',@(x)prctile(x,[5 95]))
ttP =
  3×3 timetable
    MeasurementTime          Temp              Pressure           WindSpeed    
    _______________    ________________    ________________    ________________
    18-Dec-2015        37.523    39.909    29.997    30.091    6.5199    12.544
    19-Dec-2015        40.037     41.42    29.936    29.992    6.7343    7.0799
    20-Dec-2015        41.541    42.348    29.898    29.931    7.1103    7.3119
>> 

worked just fine -- I converted the hourly data to find percentiles by day so there would be a consolidation operation since the base timetable had been retimed to the hourly basis to begin with. For a timetable by finer resolution than an hour, then 'hourly' will work same way for it.

And, it handles both the one- or multiple- percentiles automagically as well; just creates an array for each variable of as many elements as requested.

[Maria Jacob]

Thanks for checking. I literally started using timetables just a few weeks ago, and due to other commitments I haven't been able to dedicate it a lot of hours.

So yes, what I was saying is doing retime(data,'monthly',@prctile) is not allowed, because I need to specified what % I want. Sorry for not being more clear.

With respect to creating the custom function, what I tried is using rowfun, in the following way (which is a copy of an example in the Matlab documentation):

5prc = @(x) prctile(x,5);

B = rowfun(5prc,data,'GroupingVariable','month(data.Time)','OutputVariableName','5th prc');

And I get a parsing error at ) for the definition of 5prc, it literally says: Parse error at ')': usage might be invalid MATLAB syntax.

Regardless, I will try your way, since you have shown it works. I wish I had more time to tried myself before asking, sorry...

EDIT:

OMG, I'm an IDIOT!! It's the 5...

[dpb]

Chuckles...we all do such things from time to time...