Expected Improvement Bayesian Optimization Plot

Hello all,

I am fitting some Gaussian Process models on some data and I just wanted some help intepreting the plots, along with a change that came about when I started using parallel computering. The two figures with "parallel" in the image title show the two basic types of graphs that come up when you choose to optimize the hyperparameters (I used expected-improvement-plus). I also have an image from the function model before I used parallel computing. After I started using parallel computing, my function model looks alot different and it makes it seem like the model is doing worse at fitting to the observed points. am I understanding the process and why would that be if true?

Answer 1

[Don Mathis]

I think the parallel run just needs to see more points to fit a better model. The point at Sigma=0.4, Y=5.1 looks like a huge outlier, which is making the Gaussian process hypothesize a large noise variance. In your non-parallel run, that wasn't there, so it was able to fit a smaller noise value and a more accurate function. Running more function evaluations should sort things out.