Model calibration in Matlab: find minimum RMSE

Hi all,

I have a model that runs a mass balance profile for a glacier (i.e. the net result between melt and snow accumulation). However, I have a set of 7 variables to calibrate the model. I was wondering if there is some kind of small program I can write to run the model several times with all different values for the parameters within a certain range, in order to find the minimum RMSE. For example, parameter 1 from 0.4 to 0.6, parameter 2 from 0.34 to 0.42, etc. I was thinking about a random walk method but it is hard to write these kind of things. It would be very much appreciated. Can anybody help me getting started?

Thanks!

Answer 1

[Jeff Miller]

It seems like you have to start by writing the function to compute RMSE for a given set of parameter values, e.g.,

function thisRMSE = myRMSE(x)
% x is a vector of 7 parameter values.
% thisRMSE is the RMSE for those values.
% ...
end

After you have that, you can use fminsearch, which will probably do a better job of finding the minimum RMSE than a random walk.

StartingGuess = [0.5, 0.38, ...]; % Your initial best-guesses for the 7 parameter values.
[BestGuess, BestRMSE] = fminsearch(@myRMSE,StartingGuess);

If the solution space is nasty, there is a good chance that fminsearch will get caught in a local minimum (as too would your random walk process). The only thing you can do about that is to run fminsearch repeatedly from lots of different starting guess vectors, keeping whichever final 'BestGuess' really produces the lowest RMSE. You could even do that in a loop, generating the different starting guesses randomly.

HTH

[yoni verhaegen]

@Jeff Miller: thanks for the suggestion, but I can't get Matlab to run it properly. Is there a possibility that I can send you the code so you can check what I did and what is wrong about it?

Thanks

[Jeff Miller]

No, sorry, I can't accept that responsibility. But you can post a lot more detailed explanation of the problems with your code, and maybe get more detailed suggestions for fixes. For starters, what do you mean that it doesn't run "properly"?

[Pablo Cordoba]

Hello @Jeff Miller, I am trying to use the fminsearch function as you explained to find the minimum RMSE for a model output but I get an error:

The starting guess are some random parameter values that I have chosen to test the fminsearch function.

StartingGuess = [1,0.2,30,445,5];

And I obtain my RMSE like this:

sqrdError = (target_data{:,2} - model_data{1:end,2}).^2 ; % Squared Error

RMSE = sqrt(mean(sqrdError)); % Root Mean Squared Error

Then I run the fminsearch function like you did:

[newGuess, newRMSE] = fminsearch(RMSE,StartingGuess);

But when I run it I get this error that I don't know how to fix:

Error using fcnchk (line 107)

FUN must be a function, a valid character vector expression, or an inline function object.

Error in fminsearch (line 174)

funfcn = fcnchk(funfcn,length(varargin));

Error in modelCalibration (line 56)

[newGuess, newRMSE] = fminsearch(RMSE,StartingGuess);

[Jeff Miller]

The first parameter for fminsearch has to be a function, but it looks like you are trying to give it a computed value. Take the code that you used to compute model_data and RMSE (from any arbitrary set of parameters) and move that code into a separate function that can be called by fminsearch. fminsearch will call your function with lots of different parameter combinations and try to find the best one (min RMSE).

See https://au.mathworks.com/help/matlab/ref/fminsearch.html and look at the example "Minimize a Function Specified by a File"