MATLAB Answers

Hyperparameter optimization using bayesopt - what does 'Transform' value of 'log' do in optimizableVariable?

13 views (last 30 days)
Louis
Louis on 30 Mar 2020
Commented: Don Mathis on 2 Apr 2020
gives the following example for setting an optimizableVariable with log transform:
Integer variable from 1 to 1000 on a log scale:
var2 = optimizableVariable('ivar',[1 1000],'Type','integer','Transform','log')
var2 =
optimizableVariable with properties:
Name: 'ivar'
Range: [1 1000]
Type: 'integer'
Transform: 'log'
Optimize: 1
Does this search the following values for the 'ivar' hyperparameter? 1, 10, 100, and 1000?
Could someone explain how this works?

  0 Comments

Sign in to comment.

Accepted Answer

Don Mathis
Don Mathis on 2 Apr 2020
The Bayesian Optimization algorithm will model that variable on a log scale. The Gaussian Process model will fit a kernel scale parameter that applies to the log of the variable. All integer values 1..1000 are still available.

  2 Comments

Louis
Louis on 2 Apr 2020
so MATLAB's above 'var2' case will try integer values between log(1)=0 and log(1000)= 6.9078, hence making the searce space for that variable to be [0, 1, 2, 3, 4, 5, 6] ?
Don Mathis
Don Mathis on 2 Apr 2020
No, var2 will try all integer values 1:1000. Those are the values that will be passed to the objective function.
Internally, the bayesopt function will transform the integers 1:1000 into the real range log(1:1000) for the purposes of modeling the objective function with a Gaussian Process (GP) model. The GP model will see var2 in its transformed space, log(1:1000). The main consequence has to do with the kernel scale parameter of the GP. GP learns a separate Kernel Scale for each variable, which is used to measure how "similar" two points are. For example, if KernelScale=1, then the GP will consider the points 2 and 3 to be as similar to each other as the points 5 and 6, because both pairs differ by one KernelScale. Because the GP is operating in the log space, it really means that the GP considers the var2 values exp(2) and exp(3) to be a similar to each other as the points exp(5) and exp(6). Rounded to the nearest integer, it considers 3 and 7 to be as similar as 148 and 403.

Sign in to comment.

More Answers (0)

Products


Release

R2019b

Translated by