MATLAB Answers

0

How does bayesopt() optimise using categorical optimizable variables?

Asked by James Finley on 5 Apr 2019
Latest activity Commented on by James Finley on 5 Apr 2019
I have been using the bayesopt() function to perform Bayesian optimisation for material design. I have one set of optimisable variables that are of categorical data type. I was wondering how Gaussian process regression is performed when categorical input variables are used? Is it simply a case of using one-hot encoding?
Many thanks,
James

  0 Comments

Sign in to comment.

1 Answer

Answer by Don Mathis on 5 Apr 2019
 Accepted Answer

The bayesopt function uses a special technique to handle categorical variables. One-hot coding is not used. Instead, bayesopt encodes the categorical variable as an integer variable, and uses an ARD Gaussian Process kernel with a fixed spatial scale on that dimension that is so small that neighboring integer values have virtually no effect on each other. The value of the objective function model at x=7 has no effect on the model's value at x=8. The result is that the distinct integer values must be probed individually to learn what the objective function is at that value. One-hot coding would probably produce similar behavior but would increase the number of variables and require a constraint between them to make sure only one dimension is probed at a time.

  3 Comments

Dear Don,
Many thanks for your swift and concise response. Do you know of any relevant literature that discusses this encoding method in more detail?
Many thanks,
James
Unfortunately, no. As far as I know this is a novel technique. I know of at least one other B.O. package that encodes categoricals as integers, but then it treats the integer variable like any other, estimating a kernel scale that ties neighboring values together, which to me seems inappropriate.
Okay Don, thanks for your help.
And I agree that this other way of encoding categoricals sounds incorrect.

Sign in to comment.