MATLAB Answers

How to find Trend of a Uni variate Time series?

23 views (last 30 days)
bushra raza
bushra raza on 15 Dec 2018
Commented: bushra raza on 19 Dec 2018
Hi,
I have time series data of maximum water level in cm on an hourly basis for a number of years 1961-2016. the data sets are categorized as observed set from water gauges and simulated data from some model. The observed data set has no missing value but there is a long range of data about 10 years having same negative value.(i dont know why? )should i consider it as NaN? I want to calculate the trend of these two uni variate time series? i also need to compare these two series? or one can say how well the simulated time series fit with observed time series?
i have converted my time series into timetable, then i compared them using cross correlation and Nash Sutcliffe coefficient.
tried to plot this data, as well. here is its plot. S1 is observed data plot, and S2 is Simulated data plot.
please guide me how can i find the trend of these series? observed data is attached for your convience. i am badly stuck after reading all helping texts in this regard?
plot_Observed_and_SimulatedData.jpg

  1 Comment

dpb
dpb on 15 Dec 2018
There frankly doesn't appear to be any significant overall trend in those data...
Something is out of whack in a comparison since the mean of the second series is essentially 0 while ~500 for the measurements??
Clearly the measuring device was totally non-functional for a long period or somehow else the data file was corrupted. That section should clearly simply be ignored entirely as being meaningless.

Sign in to comment.

Accepted Answer

Rick Rosson
Rick Rosson on 15 Dec 2018
Edited: Rick Rosson on 15 Dec 2018
Have you tried looking for patterns in the data by day of the week, hour of the day, month of the year, or whether the date is a holiday? Time series data often exhibits seasonality effects on different time scales depending on the nature of the observed phenomenum.
doc datetime
doc boxplot
doc scatter
doc histogram

  4 Comments

Show 1 older comment
Rick Rosson
Rick Rosson on 16 Dec 2018
t = TimmendorfData.timmendorf_time;
x = TimmendorfData.timmendorf_water;
mo = month(t);
da = weekday(t);
hr = hour(t);
figure;
boxplot(x,mo);
grid on;
figure;
boxplot(x,da);
grid on;
figure;
boxplot(x,hr);
grid on;
dpb
dpb on 16 Dec 2018
I really can't comment w/o knowing much more than we do other than you have no data for that period, period. From afar, I can't see that it makes any sense to do anything other than compare the data that you do have over the time frames it exists; interpolating to make something up is simply that; there still isn't any data for that period of time available.
If you were missing a day's reading here and there, "well, maybe" you could do something, but when there are 10 years of data missing, that's kinda' a large chunk to just create out of thin air. Now, granted, whatever this is seems to be pretty consistent overall, but still...
NaN has the unfortunate habit of propogating through numerical calculations and turning everything into NaN, though, so that's probably not a viable solution other than for plotting in which it is simply ignored.
Many of the statistics calculations will ignore NaN as well; I'm not certain that the fitting toolbox has such an implementation. You can, of course, use logical addressing to select isfinite() subsets, and overall model-fitting wouldn't be terribly affected, but as others have noted, given you have what appear to be seasonal patterns, those would be drastically altered by the break if you're attempting to model the actual calendar series. If it is simply a model on the seasonal basis, independent of absolute time, then that may also work ok; you'll just have no way to do any verification that there wasn't something different going on during that period of time (albeit, given the rest, it would seem a reasonable assumption to make).
bushra raza
bushra raza on 19 Dec 2018
Hi,
thank you Mr.Rick and @dpb
yes sir, there is a long gap of 10 years in data set. i had put forward a request to correct this gap or provide another data set.
in the meanwhile, i am working on the suggested options by Mr.Rick.

Sign in to comment.

More Answers (1)

Rick Rosson
Rick Rosson on 16 Dec 2018
Another option is to look at the data in the frequency domain to estimate the relative significance of seasonality at different time scales.
doc fft
doc fftshift
doc abs
doc pwelch
doc spectrogram

  1 Comment

bushra raza
bushra raza on 16 Dec 2018
thanx alot ,sir.
please let me try these options, then come back here.

Sign in to comment.

Sign in to answer this question.


Translated by