Various types of RMSE in Regression Learner. Is it a bug?

I use the Regression Learner to produce a simple regression model. There are the following information related to the model in the Regression Learner window:

RMSE=3.5007

MSE=12.255

Observations=196.

But, when I exported the model to the workspace I see other value for RMSE: RMSE=3.53.

So, the same model has 2 different RMSEs. The first one is RMSE=3.5007. It is calculated as

RSME=sqrt( sum_of_squared_error / number_of_observations).

The second one (RMSE=3.53) is calculated as

RSME=sqrt( sum_of_squared_error / (number_of_observations - number_of_coefficients) ).

Why is necessary to use 2 various kinds of RMSE in the Regression Learner? May be it is a bug?

--

I checked also Curve Fitting Tool. It use the second variant of RMSE.

Answer 1

[Hiro Yoshino]

They should be matched together however it is not necessarily a bug because ...

RMSE is an estimator of standard error, or standard deviation.

Then, there are two types for it: biased and unbiased estimators.

Your first RMSE is a biased estimator and the second one is an unbiased counterpart.

The First one is derived from maximum likelihood method and the other one is from statistical approach.

Good point of the unbiased estimator is that its expectation matches the true value of the standard deviation.

(Obviously it is not the case with the biased one)

That being said, the point you're making is absolutely reasonable. They should not be confusing at least.

Thanks very much for raising a flag.

Answer 2

[Javier Valdes]

Hi all,

Definitively it is a little confussing situation.

When I get the RMSE from a fitted linear regresion model, the value is different that when I compute it using the built-in rmse function in matlab with the exact same input data.

I would like to suggest to MATLAB developers to include a discrimination between the two alternatives to compute rmse, in a simmilar way as it is discrimante between mdl.Rsquared.Ordinary an mdl.Rsquared.Adjusted

>> mdl.RMSE

ans =

0.081319469137617

>> rmse(mdl.Variables.y(:),mdl.Fitted(:))

ans =

0.079281062990489