## objective function in Bayesian Optimization Algorithm like fitrsvm and fitrgp

### Dimitri (view profile)

on 13 May 2019
Latest activity Edited by Don Mathis

on 14 May 2019

### Don Mathis (view profile)

Hello,
What is the mathematical objective function in the bayesian optimization algorithm? The explanation says that the algorithm like fitrsvm tries to minimize the log(1 + cross-validation loss) but what is the real mathematical formula?
Is it possible to change the objective function to just the MSE?
Thank you!
Dimitri

R2019a

### Don Mathis (view profile)

on 13 May 2019

This page says that the loss defaults to MSE. So that's the loss that's used in the log(1+cvloss) formula. Cross validated loss is the loss summed over all the held-out validation sets. The default when using optimization is 5-fold cross-validation.
There's not an option to change the hyperparameter optimization objective function from log(1+cvloss). You would need to edit the source code to do that. The source file is matlab\toolbox\stats\classreg\+classreg\+learning\+paramoptim\createObjFcn.m. Look for the call to the log1p function.

Dimitri

### Dimitri (view profile)

on 14 May 2019
Hello,
Thank you for your answer. Let´s take this example. The Mdl.HyperparameterOptimizationResults.MinObjective is 2.8666. If I take the Mdl and use L=loss(Mdl,X,Y) the loss is 16.1764. If I compute the log1p(L)=2.8435. Why is there a difference in the results?
Don Mathis

### Don Mathis (view profile)

on 14 May 2019
Because loss(Mdl,X,Y) is the loss of the final model on the full dataset, while the MinObjective is the log of 1 plus the out-of-sample cross-validated loss. See the kfoldLoss method for documentation of that. If you used 5-fold cross-validation, the kfoldLoss is the summed loss of 5 different models, each on 1/5 of the dataset. It is not the loss of the final model on the full dataset.