MATLAB Answers

0

fitrgp: hyperparamter optimization method maximum likelihood & cross-validation

Asked by seung ho yeom on 8 Jan 2019
Latest activity Answered by Don Mathis on 10 Jan 2019
Hi,
I am wondering how fitrgp optimizes hyperparameters.
According to the instructions in this link : https://kr.mathworks.com/help/stats/fitrgp.html
gprMdl2 = fitrgp(x,y,'KernelFunction','squaredexponential',...
'KernelParameters',kparams0,'Sigma',sigma0);
This code: ' The marginal log likelihood that fitrgp maximizes to estimate GPR parameters has multiple local solution '
That means fitrgp use maximum likelihood estimation (MLE) to optimize hyperparameter.
But in this code,
gprMdl2 = fitrgp(x,y,'KernelFunction','squaredexponential',...
'OptimizeHyperparameters','auto','HyperparameterOptimizationOptions',...
struct('AcquisitionFunctionName','expected-improvement-plus'));
This code : ' Find hyperparameters that minimize five-fold cross-validation loss by using automatic hyperparameter optimization.'
That means fitrgp use cross-validation (CV) to optimize hyperparameters. is this right?
So when train GPR models, there are MLE and CV methods to optimize hyperparameters. Just use fitrgp, use MLE to optimize hyperparameters, and if i put options separately in fitrgp like above code, i can optimize hyperparameters using CV instead of MLE. Did i get it right?

  0 Comments

Sign in to comment.

1 Answer

Answer by Don Mathis on 10 Jan 2019
 Accepted Answer

The hyperparameters and the objective function are different in the 2 cases.
  • When you do pass 'OptimizeHyperparameters', it will optimize the parameters you specify, which is some subset of {'BasisFunction','KernelFunction','KernelScale','Sigma','Standardize'}, using Bayesian Optimization, minimizing the out-of-sample MSE as measured using cross-validation.
There is some overlap between the 2 sets of parameters. Sigma is the same, and KernelScale corresponds to the length scales inside the kernel functions.

  0 Comments

Sign in to comment.