Unrealistic fitting confidence levels in noisy data

1 view (last 30 days)
The fitting of simple exponential is giving a confidence level which is not realistic providing how noisy are the data. Any explanation please ?
Thank you. Code:
Data = xlsread('data_test.xlsx','Sheet1','A1:B3990');
Time = Data(:,1) - 2.6E-6 ; % time offset
Signal = Data(:,2)-0.078; % amplitude offset
x = Time;
y = Signal;
y = y/(0.8*max(y)); % data normalization to about 1
format long % just to get more precision digits
mdl = fittype(' a*(1-exp(-b*x)) ','indep','x')
mdl =
General model: mdl(a,b,x) = a*(1-exp(-b*x))
fittedmdl = fit(x,y,mdl,'StartPoint', [max(y) 1E5])
fittedmdl =
General model: fittedmdl(x) = a*(1-exp(-b*x)) Coefficients (with 95% confidence bounds): a = 0.9589 (0.955, 0.9628) b = 5.557e+06 (5.373e+06, 5.74e+06)
figure
plot(x,y, 'bp', 'DisplayName','data')
hold on
plot(fittedmdl)
grid
xlabel('Time /s')
ylabel('Intensity /a.u')
ax = gca;
ax.FontSize = 15;
As you can see the error on (b) is very small (only 3% !) which is not realistic looking how noisy are the data:
coefficientValues = coeffvalues(fittedmdl);
a = coefficientValues(1);
b = coefficientValues(2);
Tau = (1/b)
Tau =
1.799653625749003e-07
ConfIntervals = confint(fittedmdl);
b_err = (ConfIntervals(2,2) - ConfIntervals(1,2))/2;
DeltaTau = Tau * (b_err/b)
DeltaTau =
5.935716982509685e-09
Error = (DeltaTau / Tau)*100
Error =
3.298255229552455

Answers (1)

Bjorn Gustavsson
Bjorn Gustavsson on 1 Sep 2022
This is most likely due to the large number of data-points you have. Compare with the simpler case of the uncertainty of the average of a number of random samples from a normal-distribution with a large standard deviation - it decreases roughly as the square-root of the number of samples. Here you have a similar situation. One thing you can do is to plot the distribution of the residuals, and investigare how it varies as you vary the parameters - and check how much it starts to skew off zero-centred and symmetric. Another thing you should look at are the residuals - to my naked eye there seems to be some systematic variation in the signal that aren't accounted for - some higher-frequency variations that do not look like random noise.
HTH

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by