MATLAB Answers

0

how to merge interpolated data into an existing timetable?

Asked by bushra raza on 25 Dec 2018
Latest activity Commented on by Peter Perkins
on 2 Jan 2019
Hi,
i have a long uni variate timeseries data with 1 minute resolution. Here is an attachment of one part of this data set,shifted into timetable.
this attached data has one time slot ( 28 Mar 2017 21:49:00 to 10 Apr 2017 10:28:00) having negative value, which should not be negative. so i have converted this negative value to NaN, and then this timerange data is interpolated. i have succesfully interpolated this timeslot data. but i am not getting a way to put this interpolated data into the existing time table. caz i need full timetable at the end, with the interpolated values as well . Here is my code, please can any one guide me by looking at my code, what am i missing?
one thing more, when i run this code , it displayed clearly the interpolated data with the existing data in the plot, but with a warning "Warning: Columns of data containing NaN values have been ignored during interpolation. " how to get rid of this warning?
Looking forward to any guidance anxiously...
load OneMinute_ObsData;
plot(OneMinute_ObsData.timmendorf_time,OneMinute_ObsData.timmendorf_waterlevel);
for i= 1:size(OneMinute_ObsData,1)
if( OneMinute_ObsData.timmendorf_waterlevel(i,1)< 0)
OneMinute_ObsData.timmendorf_waterlevel(i,1) = NaN;
end
end
Timerange_for_interpolation=[datetime('2017-3-28 21:49:00'):minutes(1):datetime('2017-4-10 10:28:00')]';
V = interp1(OneMinute_ObsData.timmendorf_time, OneMinute_ObsData.timmendorf_waterlevel, Timerange_for_interpolation, 'spline');
% Plot the interpolated points.
hold on
plot(Timerange_for_interpolation,V,'r');

  0 Comments

Sign in to comment.

2 Answers

Answer by Peter Perkins
on 27 Dec 2018
 Accepted Answer

Really this is best done with fillmissing. It's just two lines:
>> X = [1;2;-3;-4;5;6];
>> tt = timetable(X,'RowTimes',datetime(2018,12,27,0,0:5,0))
tt =
6×1 timetable
Time X
____________________ __
27-Dec-2018 00:00:00 1
27-Dec-2018 00:01:00 2
27-Dec-2018 00:02:00 -3
27-Dec-2018 00:03:00 -4
27-Dec-2018 00:04:00 5
27-Dec-2018 00:05:00 6
>> tt.X(tt.X<0) = NaN
tt =
6×1 timetable
Time X
____________________ ___
27-Dec-2018 00:00:00 1
27-Dec-2018 00:01:00 2
27-Dec-2018 00:02:00 NaN
27-Dec-2018 00:03:00 NaN
27-Dec-2018 00:04:00 5
27-Dec-2018 00:05:00 6
>> tt1 = fillmissing(tt,"spline")
tt1 =
6×1 timetable
Time X
____________________ _
27-Dec-2018 00:00:00 1
27-Dec-2018 00:01:00 2
27-Dec-2018 00:02:00 3
27-Dec-2018 00:03:00 4
27-Dec-2018 00:04:00 5
27-Dec-2018 00:05:00 6

  2 Comments

Thank you so much sir.
since many days, i am working on it, and even then i interpolated the values, but my data set was showing some missing data in summary(timetable) command. and i was not getting a way out.
your solution is superb, really worked for me. :)
Sir, can you please guide me in my next step?
i have to calculate the trend of my timeseries data.
this one minute resolution data is further meged in an hourly data, and the whole data set is now in hourly resolution.
please give me some hint, i tried box plot and linear trend line , but unfortunately i could not get a way to interpret it.
how to calculate the trend values for my timeseries data set?
That's a different question that deserves its own thread.

Sign in to comment.


Answer by bushra raza on 26 Dec 2018

Hi,
i have got the solution myself. just few more lines of code.
here is the code, May it be helpful for some one else as well.
load OneMinute_ObsData;
% plot(OneMinute_ObsData.timmendorf_time,OneMinute_ObsData.timmendorf_waterlevel);
for i= 1:size(OneMinute_ObsData,1)
if( OneMinute_ObsData.timmendorf_waterlevel(i,1)< 0)
OneMinute_ObsData.timmendorf_waterlevel(i,1) = NaN;
end
end
timmendorf_time=[datetime('2017-3-28 21:49:00'):minutes(1):datetime('2017-4-10 10:28:00')]';
timmendorf_waterlevel = interp1(OneMinute_ObsData.timmendorf_time, OneMinute_ObsData.timmendorf_waterlevel, timmendorf_time, 'spline');
%make a new timetable of interpolated values
TT = timetable(timmendorf_time,timmendorf_waterlevel);
OneMinute_ObsData = [OneMinute_ObsData ;TT];
%%Sort in Time Order
sorted = issorted(OneMinute_ObsData); % logical 0
if (sorted == 0)
OneMinute_ObsData = sortrows(OneMinute_ObsData);
end %if end
plot(OneMinute_ObsData.timmendorf_time,OneMinute_ObsData.timmendorf_waterlevel);

  0 Comments

Sign in to comment.