How to replace messy data by using linear interpolation of nearby messy data

Hello!! I would like to ask for some help. I have been working with the data processing for weeks. I am quite new to this. I have been struggling to find the solution to my problem. My problem is that I have to import data from txt file which 90 % of them are alright, but some contains messy data. I will show an example of input text file in the attached file. I would like to clean this data by replacing it by the linear interpolation of the nearby existing values. My messy data is either in form of ********** or 0.

I have been seeking for an answer for a while, but no solution has been found so far.

Note that the actual input file that I work with is in format of .out not .txt but I cannot attach it here.

Thank you in advance

[KSSV]

YOu can replace the messy data with NaN and use fillmissing

Answer 1

[KSSV]

Replace all ****** values in the text file with NaN's and follow the below code:

data = importdata('data.txt') ;
data = data.data ;
t = data(:,1) ; 
s1 = data(:,2) ; 
s2 = data(:,3) ;
%% Remove outliers in s1
stdDev = nanstd(s1) ;% Compute standard deviation
meanValue = nanmean(s1) ; % Compute mean
zFactor = 1.5; % or whatever you want.
% Create a binary map of where outliers live.
outliers = abs(s1-meanValue) > (zFactor * stdDev);
s1(outliers) = NaN ;
% GEt NaN filled 
idx = isnan(s1) ;
s1(idx) = interp1(t(~idx),s1(~idx),t(idx)) ;

%% Remove outliers in s2
stdDev = nanstd(s2) ;% Compute standard deviation
meanValue = nanmean(s2) ; % Compute mean
zFactor = 1.5; % or whatever you want.
% Create a binary map of where outliers live.
outliers = abs(s2-meanValue) > (zFactor * stdDev);
s2(outliers) = NaN ; 
% GEt NaN filled 
idx = isnan(s2) ;
s2(idx) = interp1(t(~idx),s2(~idx),t(idx)) ;

[Danupon Subanapong]

Hi KSSV!! Thank you for your answer. But it only changes the first messy data (******) not all. And the function importdata only imports some part of data from row no.1 to the first messy data row. Could you please explain more if I did anything wrong with the code?

[KSSV]

i jave asked you to replace those stars with nan...did you?

[Danupon Subanapong]

Ohh Im sorry it did work.

[KSSV]

YOu can also have a look on fillmissing.