Best practices for setting penalty values in objective functions for bayesopt in MATLAB

Hi all,

I’m using bayesop, and trying to design a “safe” objective function: whenever my function encounters out-of-domain inputs,or some kind of errors, I return a large penalty value instead of letting the algorithm fail, although I understand that this algorithm never fails but keeps on running even if it encounters an error.

My typical objective values are in the range of -2 to 2. I’m wondering, does the magnitude of the penalty value influence how bayesopt performs?

For example, would assigning a penalty of 20 have a noticeably different impact compared to a penalty of 1e6? Is there an optimal approach or rule of thumb for choosing penalty values so that invalid points are discouraged, yet the optimization routine remains numerically stable and efficient?

I’d appreciate any insights. Thanks!

Answer 1

[Matt J]

You might see slower convergence when the penalty is very large.

[Sauradeep]

Thanks. So, curious to know what's the rule of thumb for choosing penalty values? Given my typical objective values are in the range of -2 to 2, what should I be using?

[Matt J]

There is no rule of thumb. You will have to experiment.