plot sfit shows an offset between data and the fit but the residuals are very small.

I have been trying to fit my data to a custom function with 10 fitting parameters using

% [fitres,gof] = fit( [xData, yData], zData, ft, opts )

where ft is the custom function I defined, opts specifies the parameter bounds, initial guess, fitting method and the stop criteria. I can plot the fitting results as follows

plot(fitres,[xData yData],zData)

which generates the figure below

There is clearly an overall offset between the data and fit. But we could check the goodness of fit

gof = 

  struct with fields:

           sse: 0.0576
       rsquare: 0.9850
           dfe: 269
    adjrsquare: 0.9845
          rmse: 0.0146
          

and also the residuals plot

plot(fitres,[xData yData],zData,'Style','residuals') 

I can also plot fit results with the data manually using surf

surf(reshape(feval(fitres,[xData yData]),[31 9]),'linestyle','none')
hold on
surf(reshape(zData,[31 9]),'facecolor','none')

where the solid surface is the fit, the grid is the data.

All of the three tests above showed a relatively reasonable agreement between the data and the fit. Can anyone help to explain the problem with the plot(fitres,[xData yData],zData) function?

The data can be found in the attachment.

[the cyclist]

Can you upload the data? You can use the paper clip icon in the INSERT section of the toolbar.

[Jingnan Cai]

Thanks for the comment! I uploaded the data to the forum.

[Matt J]

We can't try to reproduce what you see without the model function as well.

[Jingnan Cai]

I didn't realize that ploting requires the original function. It has been uploaded as well.

[Matt J]

Your posted plots are all uploaded images. Please instead generate them with the Run button, so that we know they come from the given code, and that the code is complete.

[Jingnan Cai]

I uploaded all the functions involved in the code. The online Matlab kernel is too slow to generate any of the plots. It should be possible to reproduce all the plots attached after downloading the data and the funcitons.

[Matt J]

Nope, it's not. ft and opts are missing. Please add all code needed to execute the fit.

[Jingnan Cai]

I updated the data file yesterday. Please download again and overwrite the previous version.

And I feel like this should be an issue with plotting not fitting.

[the cyclist]

It's a mystery to me. If I extract the fit coefficients from the fitres object, and manually plot, it show them close together:

load test_data
Def = 0.2908;
gmf = 1077;
lossf = 0.02632;
lossother = 1.303;
losstls = 0.6968;
mu = 0.7957;
t1f = 4.784e+06;
t2f = 0.001997;
tmin = 0.006372;
vol = 60.73;

figure
hold on
scatter3(xData,yData,zData)
scatter3(xData,yData,tls_full_fit(xData,yData,lossf,losstls,lossother,t2f,t1f,gmf,vol,Def,mu,tmin))
view(-45,30)

Answer 1

[Matt J]

One thing that seems to be creating problems is that your model function returns different results depending on whether the (x,y) pairs given to it are vectors or scalars. This violates the assumptions of the Curve Fitting Toolbox. The model function is supposed to be a point-wise operation. In other words, the use of mdl(X,Y) or arrayfun(mdl,X,Y) below is supposed to give the same result, but does not:

load('test_data.mat')

Def = 0.2908;
gmf = 1077;
lossf = 0.02632;
lossother = 1.303;
losstls = 0.6968;
mu = 0.7957;
t1f = 4.784e+06;
t2f = 0.001997;
tmin = 0.006372;
vol = 60.73;


[X,Y]=deal(xData,yData);

mdl= @(x,y)tls_full_fit(x,y,lossf,losstls,lossother,t2f,t1f,gmf,vol,Def,mu,tmin);

discrepancy = max(abs( mdl(X,Y)  - arrayfun(mdl,X,Y)   ) ) 

discrepancy = 0.0896

[Jingnan Cai]

Thanks for the insight! The problem is indeed within the model function definition in line 13

gmax=g2(1,end)*gmf*exp(-tmin./T);

where I used g2 calculated from xData as a covenient scaling factor. This code works when the input is the full vector/matrix of data. However, when one uses arrayfun that evaluates the function elementwise, g2(1,end) will change for each element of the data. I just need to change the defintion of gmf to include the scaling to avoid this issue.

Thanks again for the detective work!