How to use bayesopt function to predict the optimal parameters for the experiment?

I have a list of X values (like experimental parameters) and corresponding Y values (the experimental results), as well as X variable ranges, how can I use the bayesopt() function to predict the optimal X parameters to get the best Y value, since most of the examples on the internet are about find the optimal hyperparameters of a algorithm, which is a litte different from my problem. I don't know how to deal with the object function, since I don't have a object function, my data are experimental parameters and experimental results. I hope to use the existing data to predict the optimal parameters. I find some examples, and write a program, but it works not well as I expected, here is the program,

the examples that I refered,

https://www.mathworks.com/matlabcentral/answers/383808-can-bayesopt-be-run-without-an-objective-function-call

X = unifrnd(0,10,30,3);
Y = unifrnd(0,2000,30,1);
sat1 = optimizableVariable('s1',[0,10],'Type','real');
sat2 = optimizableVariable('s2',[0,10],'Type','real');
sat3 = optimizableVariable('s3',[0,10],'Type','real');
var=[sat1,sat2 sat3];
initialXList = table;
initialXList.s1 = X(:,1);
initialXList.s2 = X(:,2);
initialXList.s3 = X(:,3);
initialObjList = Y;   
dummyFunc = @(Tbl)0;
bayesObject = bayesopt(dummyFunc,var,...
    'InitialX',initialXList,...
    'InitialObjective',initialObjList,...
    'MaxObjectiveEvaluations',50, ...
    'Verbose',1);

[xu supeng]

I have a thought, but maybe it's not easy to realize. Firstly, use a group of X and corresponding Y to train a model with optimal parameters, e.g. fitcknn algorithm (to get the optimal hyperparameters), then, use the model as objective function, and use bayesopt() to select the next 20 X values from the X ranges, and the correspoing Y values, and use the 20 group of [X, Y] as experimental data to train the algorithm again, repeat this process agin and again, finally, get the optimal results, I will do it, if anyone can give me suggestions, I would thanks very much, since I am not familar with algorithm, and which algorithm to choose should have quite influence on the final results~

Note, the [X, Y] are totally experimental results, so there isn't obvious relationship between them~

A simple case, but it seems the bayesopt() function can't use the trained model Md1, why? what's the problem~

rng default
X = unifrnd(0,10,30,3);
Y = unifrnd(0,2000,30,1);

Mdl = fitcknn(X,Y,'OptimizeHyperparameters','auto',...
    'HyperparameterOptimizationOptions',...
    struct('AcquisitionFunctionName','expected-improvement-plus'));

sat1 = optimizableVariable('s1',[0,10],'Type','real');
sat2 = optimizableVariable('s2',[0,10],'Type','real');
sat3 = optimizableVariable('s3',[0,10],'Type','real');
var=[sat1,sat2 sat3];

Fun = @(z)predict(Mdl,[z.s1,z.s2,z.s3]);
bayesObject = bayesopt(Fun,var,...
    'MaxObjectiveEvaluations',20, ...
    'Verbose',1);

Answer 1

[Alan Weiss]

It soiunds to me as if there are two steps to your problem:

1. Fit a parameterized function to some data. So you might have some function like y = A(1)exp(-A(2)*x + A(3)*x^2 and you first need to fit your A values to the given (x,y) data. It really is up to you to decide on a suitable parameterized function.
2. Minimize the resulting function. Once you have a parameterized curve or surface y = f(x) with known function f, use the normal optimization procedures to find the location of the minimum.

You can use bayesopt for either or both steps, or use some other optimizer for either or both steps.

Good luck,

Alan Weiss

MATLAB mathematical toolbox documentation

[xu supeng]

There is a group of

[X, Y] =
       8.1472       2.7603       1.6218     -0.82733       1.4432       4.6309      -35.544
       6.3236         1.19       1.6565    -0.091359      -1.4927     -0.11102      -2659.9
       5.4688       3.4039       6.5408       4.0005      0.50156      -1.0448      -563.86
       9.5751       5.8527       6.8921      -1.3075       1.2248      -1.3256      -2707.1
       9.6489       2.2381       7.4815       -3.888      0.87045       4.8798      -45.208
       9.5717       5.0596       2.2898      -2.5831     -0.29077       4.1329      -3351.8
       4.8538       6.9908       9.1334     -0.96088      -2.6951       2.9618      -3800.9
       9.5949       2.5751       4.4268      -4.4022     -0.64301       2.2123      -1692.7
       6.5574       8.4072       1.0665      -2.6522       -1.889      -3.9324      -3007.7
       8.4913       8.1428     0.046342       3.2119     -0.69793    -0.058261      -2403.5
       9.3399       2.4352       7.7491       -4.846      -3.1518       2.7905      -1049.2
       3.9223       2.5108       3.9978       2.3172      -3.8888      -1.6584      -980.28
       1.7119       4.7329       8.0007     -0.49076      -0.9128      -3.0219      -6379.7
      0.31833       8.3083       9.1065      -2.0368      -2.3779       2.4407      -541.88
      0.46171       5.4972        2.638      -3.1104       2.1122     -0.20078      -5104.9
       8.2346       2.8584       1.3607      -3.1649      -3.8258       1.0987      -5795.5
       9.5022       3.8045       5.4986       2.8023     -0.75833       3.0549      -2164.5
       4.3874      0.75854       8.5303       4.2939      -4.1448      -3.1708      -782.67
        7.952       7.7917       5.1325     -0.64141      -4.7078      -4.7133      -6278.2
       1.8687       9.3401       4.0181     -0.53216       4.2885     -0.10099      -2295.5
       7.5469       3.3712       2.3995       2.9483     -0.41151     -0.28912      -6822.7

six coloums of X, and one coloum corresponding Y, when I use fitrgp() function to fit the data, I can get very beautiful results,

the program is

Md1 = fitrgp(X,Y,'OptimizeHyperparameters','all');

figure;
plot(Y);
hold on;
plot(predict(Md1,X));

then I try to use the trained Md1 as the objective function to search the next 20 group of [X, Y], and the program is

sat1 = optimizableVariable('s1',[0,10],'Type','real');
sat2 = optimizableVariable('s2',[0,10],'Type','real');
sat3 = optimizableVariable('s3',[0,10],'Type','real');
delta1 = optimizableVariable('d1',[-5,5],'Type','real');
delta2 = optimizableVariable('d2',[-5,5],'Type','real');
delta3 = optimizableVariable('d3',[-5,5],'Type','real');

var=[sat1 sat2 sat3 delta1 delta2 delta3];

Fun = @(z)predict(Md1,[z.s1 z.s2 z.s3 z.d1 z.d2 z.d3]);

bayesObject = bayesopt(Fun,var,...
    'MaxObjectiveEvaluations',20, ...
    'Verbose',1);

it has error,

if I add the 'NumCoupledConstraints',1, to the bayesopt() function according to its reminder,

it will give me infeasible results, like

so, why this happen, and how to fix it? (I already seen the explanation about Constraints in Bayesian Optimization, but it's not easy to understand)

When I change to fitrlinear() function to fit the experimental data, and followed by bayesopt optimization, it won't have this problem, but fitrlinear() function can't give me very accurate fitting results, here is the result,

Look forward to your reply！

[Alan Weiss]

You almost had it right. Your mistake was not recognizing that bayesopt passes a table of values, but your constructed model accepts a matrix of values. So you first need to write an objective function that takes a table, converts it to numeric, and then evaluates it, like this:

function f = mdlfun(tbl,mdl)
sat1 = tbl.s1;
sat2 = tbl.s2;
sat3 = tbl.s3;
delta1 = tbl.d1;
delta2 = tbl.d2;
delta3 = tbl.d3;

var = [sat1 sat2 sat3 delta1 delta2 delta3];

f = predict(mdl,var);

end

Then create the optimizable variables and pass them to bayesopt:

sat1 = optimizableVariable('s1',[0,10],'Type','real');
sat2 = optimizableVariable('s2',[0,10],'Type','real');
sat3 = optimizableVariable('s3',[0,10],'Type','real');
delta1 = optimizableVariable('d1',[-5,5],'Type','real');
delta2 = optimizableVariable('d2',[-5,5],'Type','real');
delta3 = optimizableVariable('d3',[-5,5],'Type','real');

var = [sat1 sat2 sat3 delta1 delta2 delta3];

bayesObject = bayesopt(@(tbl)mdlfun(tbl,Md1),var,...
    'MaxObjectiveEvaluations',20, ...
    'Verbose',1);

Alan Weiss

MATLAB mathematical toolbox documentation

[xu supeng]

Yes, it works, thank you so much~

[xu supeng]

Finally, I write a program to optimize the best parameters for experiment, but it seems work not well. Here, I don't ask technical questions, I just wonder why the program doesn't work as I expected (here, I mean generally it cann't give me the minimum Y value or the minimum estimated value is in the minimum experimental data place, and once I changed the experimental data space, the optimal estimated Y value change to the new experimental minimum value place, the optimization cann't give me obvious improvement in Y value, little complex, I try to explain why I think it doesn't work), maybe you can give me some suggestions because you are familar with bayesopt() function.

The process is, firstly, train the fitrgp() model with experimental data, and obtain Md1 with best hyperparameters. Secondly, use the Md1 as the objective function and bayesopt() function to get the next [Xi Yi], then put the [Xi, Yi] into the [X Y] array and use fitrgp() to train to get a new model with best hyperparameters, and put the new model into the bayesopt() function to get the next point, repeat this again and again. This is also what most tutorial said ~

Here is the program, any suggestions, appreciate it so much

clc; 
clear;
tic;
%%%Colom 1 ~ 6 is the X parameters, Colom 7 is the Y values
Data =[5.6207       3.4326      0.99076      -0.5681       2.2269     -0.76352   -2560.6
    9.4475       5.1669       8.8368      -1.5091       4.6748      -1.2324      -738.19
    6.2045       3.1342       5.3344      -1.0461       -4.807      -2.4016      -3044.6
    9.1965       5.4567        2.741       4.9886      -2.2476      -1.2381      -427.06
    5.5178       2.9224      0.47097      -4.7451      -4.3422        2.582      -1363.8
    1.4587         6.82       9.6364     -0.55175      -4.2558       4.4485      -5265.8
    9.003       1.3252      0.91798        3.956       1.3036       2.8979      -788.03
    3.658       4.7395       9.7259       3.8048      -0.8229     0.097753      -2241.8
    3.661       9.6049       9.3967     -0.52114       1.8493      -3.1426      -2594.9
    7.538       6.3236       3.6793     -0.27248      -2.4987       2.9187      -3992.8
    2.8826       2.9379       7.6082      -4.3078       3.5267     -0.76454      -1637.5
    2.5475      0.41993       2.1689       3.0568      -0.5199      -2.6993      -971.66
    9.8318       9.7541       7.1499       -1.069     -0.10125         2.48      -3309.2
    5.0904       0.4727       6.0548      -3.1414      -1.1608       2.6665      -2494.9
    5.3166       9.3089       3.4795       1.1064       1.6673       3.2222      -3272.3
    4.8304      0.35989       1.7593       4.4904      -4.7422       1.9583      -421.03
    5.5926       7.7423       7.4898       1.6932       3.9333       3.2528       -911.6
    8.5082       8.2249       6.6189       4.0599       1.9319     -0.82999        -6993
    7.0123       4.0614       2.0719      -2.5927       -3.013       -1.769      -1800.4
    3.6205       2.0072       4.4781      -1.2097        3.811       2.8131      -1086.1];

X = Data(:,1:6);  
Y = Data(:,7);   

for ii=1:100     %%% 100 groups of Loop, each Loop 1 iterations based on the former X Y data, since the initial [X Y] values are also included in the iteration numbers
    
%     disp([X,Y]);
    disp(ii);
    Len=length(Y);   
%     disp(Len);
       
    %%%  trained with fitrgp to find the optimal hyperparameters for X Y 
    Md1 = fitrgp(X,Y,'OptimizeHyperparameters','all','HyperparameterOptimizationOptions',struct('ShowPlots',false,'Verbose',0));
    
    %     disp([Md1.BasisFunction,'  ',Md1.KernelFunction,'  ',num2str(Md1.Sigma)]);
    
    if mod(ii,10)==0
        figure(ii);
        plot(Y);
        hold on;
        plot(predict(Md1,X));  %% validate the trained model with real data
    end
    
    sat1 = optimizableVariable('s1',[0,10],'Type','real');
    sat2 = optimizableVariable('s2',[0,10],'Type','real');
    sat3 = optimizableVariable('s3',[0,10],'Type','real');
    delta1 = optimizableVariable('d1',[-5,5],'Type','real');
    delta2 = optimizableVariable('d2',[-5,5],'Type','real');
    delta3 = optimizableVariable('d3',[-5,5],'Type','real');   %% set the variable
    var=[sat1 sat2 sat3 delta1 delta2 delta3];
    
    initialXList = table;
    initialXList.s1 = X(:,1);
    initialXList.s2 = X(:,2);
    initialXList.s3 = X(:,3);
    initialXList.d1 = X(:,4);
    initialXList.d2 = X(:,5);
    initialXList.d3 = X(:,6);
    initialObjList = Y;        %%% set te initial X Y values
    
    bayesObject = bayesopt(@(tbl)mdlfun(tbl,Md1),var,...
        'MaxObjectiveEvaluations',Len+1,...         %%% 
        'InitialX',initialXList,...
        'InitialObjective',initialObjList,...
        'PlotFcn',{},...
        'Verbose',0);
    
    disp([bayesObject.XAtMinObjective array2table(bayesObject.MinObjective)]);
    disp([bayesObject.XAtMinEstimatedObjective array2table(bayesObject.MinEstimatedObjective)]);
    
    Results(ii,:) = [bayesObject.XAtMinEstimatedObjective array2table(bayesObject.MinEstimatedObjective)];
    
    X=table2array(bayesObject.XTrace);
    Y=bayesObject.ObjectiveTrace;
end

% disp([bayesObject.XAtMinObjective array2table(bayesObject.MinObjective)]);        %%% optimal observation
% disp([bayesObject.XAtMinEstimatedObjective array2table(bayesObject.MinEstimatedObjective)]);  %%% optimal extimation
figure;
plot(table2array(Results(:,7)),'o');
toc;

function f = mdlfun(tbl,mdl)
sat1 = tbl.s1;
sat2 = tbl.s2;
sat3 = tbl.s3;
delta1 = tbl.d1;
delta2 = tbl.d2;
delta3 = tbl.d3;
var = [sat1 sat2 sat3 delta1 delta2 delta3];
f = predict(mdl,var);
end

[Alan Weiss]

I think that you are not letting bayesopt run long enough. You have a 6-D space to search. That is a lot of volume to cover, and you let it search for very few steps, so it probably does not get close to the minimum.

In fact, I think that it is a mistake to use bayesopt to look for a minimum of the objective function. I'd use fmincon starting from a bunch of initial points, or maybe do that automatically using MultiStart. But if you don't have Global Optimization Toolbox, just use fmincon. Or patternsearch if things are not smooth and you do have Global Optimization Toolbox.

Good luck,

Alan Weiss

MATLAB mathematical toolbox documentation

[xu supeng]

Thanks, I will try these methods~

[xu supeng]

Hi, I have a problem when I try to interpolate Matlab inbuilt function bayesopt with Julia function, in my program, I try to use julia to evaluate my objective function, since it's fast, but I also want to use matlab bayesopt to optimize it, since julia's package is not that complete, it's almost work, but has a problem with variable type, here is an simple example, and the error as follows, thanks in advance and look forward for your reply

using MATLAB
function f(x, y)
    return -x - y
end
function mdlfun(tbl)
    x = tbl.v1
    y = tbl.v2
    return f(x, y)
end

mat"""
xrange = optimizableVariable('v1',[-2,2],'Type','real');
yrange = optimizableVariable('v2',[-5,5],'Type','real');
var=[xrange yrange];

bayesObject =bayesopt(@(tbl)$mdlfun(tbl),var,...
    'MaxObjectiveEvaluations',50)

"""

Unable to use a value of type table as an index.

Error in @(tbl)matlab_jl_2(tbl)

Error in BayesianOptimization/callObjNormally (line 2576)
Objective = this.ObjectiveFcn(conditionalizeX(this, X));

Error in BayesianOptimization/callObjFcn (line 481)
= callObjNormally(this, X);

Error in BayesianOptimization/runSerial (line 1996)
ObjectiveFcnObjectiveEvaluationTime, ObjectiveNargout] = callObjFcn(this, this.XNext);

Error in BayesianOptimization/run (line 1948)
this = runSerial(this);

Error in BayesianOptimization (line 457)
this = run(this);

Error in bayesopt (line 323)
Results = BayesianOptimization(Options);

[Alan Weiss]

I do not understand why you try to use Julia to evaluate your objective function "because it is fast" but then use bayesopt to optimize. The bayesopt function is quite slow, typically adding at least a second of overhead per objective function evaluation. If your objective function takes much less than a second to evaluate in MATLAB, then I don't see the benefit of using something else to evaluate the objective.

As for your current program, I do not understand several lines.

function f(x, y)
    return -x - y
end
% I think that this should be
function z = f(x, y)
    z = -x - y;
end
%------------
function mdlfun(tbl)
    x = tbl.v1
    y = tbl.v2
    return f(x, y)
end
% I think that this should be
function z = mdlfun(tbl)
    x = tbl.v1
    y = tbl.v2
    z = f(x, y);
end

bayesObject =bayesopt(@(tbl)$mdlfun(tbl)...
% I do not understand the existence of $ before mdlfun; it looks like an
% error

Alan Weiss

MATLAB mathematical toolbox documentation

[xu supeng]

Yes, it's my fault, I write this program within Julia, and call matlab bayesopt function from julia, so that's why I write it like this, the definition of function is a little different, and about the objective function, to be honest, with matlab it takes about 200s to run it for one time, but with julia, it takes about 10s, that's why I use Julia, I am sorry, I didn't say it clear, and I think I already try my best to optimimize the program speed, maybe there is still improvement space that I don't know, but matlab is good just not that fast~ thanks anyway

[Alan Weiss]

Thanks for the clarifications. I cannot help you, though; I obviously know nothing about Julia, and the problems you encounter seem to be related to using Julia.

Alan Weiss

MATLAB mathematical toolbox documentation

Answer 2

[xu supeng]

Hi， guys

I already fix this problem last week, the problem is I try to use the fitted objective function to estimated the next Y values, but actually the function is a prediction function, it can tell you the next optimal Xi place where you have the largest opportunity to found the optimal Yi value, and you need to do experiment or simulation with Xi to get the real Yi value and put it into the database and updates the fitted objective function, then it works quite well, here I share the codes and the results ~

clc;
clear;
tic;
%%%Colom 1 ~ 6 is the X parameters, Colom 7 is the Y values
Data =[5.6207       3.4326      0.99076      -0.5681       2.2269     -0.76352   -2560.6
    9.4475       5.1669       8.8368      -1.5091       4.6748      -1.2324      -738.19
    6.2045       3.1342       5.3344      -1.0461       -4.807      -2.4016      -3044.6
    9.1965       5.4567        2.741       4.9886      -2.2476      -1.2381      -427.06
    5.5178       2.9224      0.47097      -4.7451      -4.3422        2.582      -1363.8
    1.4587         6.82       9.6364     -0.55175      -4.2558       4.4485      -5265.8
    9.003       1.3252      0.91798        3.956       1.3036       2.8979      -788.03
    3.658       4.7395       9.7259       3.8048      -0.8229     0.097753      -2241.8
    3.661       9.6049       9.3967     -0.52114       1.8493      -3.1426      -2594.9
    7.538       6.3236       3.6793     -0.27248      -2.4987       2.9187      -3992.8
    2.8826       2.9379       7.6082      -4.3078       3.5267     -0.76454      -1637.5
    2.5475      0.41993       2.1689       3.0568      -0.5199      -2.6993      -971.66
    9.8318       9.7541       7.1499       -1.069     -0.10125         2.48      -3309.2
    5.0904       0.4727       6.0548      -3.1414      -1.1608       2.6665      -2494.9
    5.3166       9.3089       3.4795       1.1064       1.6673       3.2222      -3272.3
    4.8304      0.35989       1.7593       4.4904      -4.7422       1.9583      -421.03
    5.5926       7.7423       7.4898       1.6932       3.9333       3.2528       -911.6
    8.5082       8.2249       6.6189       4.0599       1.9319     -0.82999        -6993
    7.0123       4.0614       2.0719      -2.5927       -3.013       -1.769      -1800.4
    3.6205       2.0072       4.4781      -1.2097        3.811       2.8131      -1086.1];

X = Data(:,1:6);
Y = Data(:,7);

for ii=1:500     %%% 10 groups of Loop, each Loop 20 iterations based on the former X Y data, since the initial [X Y] values are also included in the iteration numbers
    
    %     disp([X,Y]);
    disp(ii);
    Len=length(Y);
    %     disp(Len);
    
    %%%  trained with fitrgp to find the optimal hyperparameters for X Y
    Md1 = fitrgp(X,Y,'OptimizeHyperparameters','all','HyperparameterOptimizationOptions',struct('ShowPlots',false,'Verbose',0));
    
    %     disp([Md1.BasisFunction,'  ',Md1.KernelFunction,'  ',num2str(Md1.Sigma)]);
    
    sat1 = optimizableVariable('s1',[0,10],'Type','real');
    sat2 = optimizableVariable('s2',[0,10],'Type','real');
    sat3 = optimizableVariable('s3',[0,10],'Type','real');
    delta1 = optimizableVariable('d1',[-5,5],'Type','real');
    delta2 = optimizableVariable('d2',[-5,5],'Type','real');
    delta3 = optimizableVariable('d3',[-5,5],'Type','real');   %% set the variable
    var=[sat1 sat2 sat3 delta1 delta2 delta3];
    
    initialXList = table;
    initialXList.s1 = X(:,1);
    initialXList.s2 = X(:,2);
    initialXList.s3 = X(:,3);
    initialXList.d1 = X(:,4);
    initialXList.d2 = X(:,5);
    initialXList.d3 = X(:,6);
    initialObjList = Y;        %%% set te initial X Y values
    
    bayesObject = bayesopt(@(tbl)mdlfun(tbl,Md1),var,...
        'MaxObjectiveEvaluations',Len+1,...         %%%
        'InitialX',initialXList,...
        'InitialObjective',initialObjList,...
        'PlotFcn',{},...
        'Verbose',0);
    
    if mod(ii,5)==0
        disp([bayesObject.XAtMinObjective array2table(bayesObject.MinObjective)]);
        disp([bayesObject.XAtMinEstimatedObjective array2table(bayesObject.MinEstimatedObjective)]);
    end
    
    Results(ii,:) = [table2array(bayesObject.XAtMinObjective) bayesObject.MinObjective table2array(bayesObject.XAtMinEstimatedObjective) bayesObject.MinEstimatedObjective];
    
    X=table2array(bayesObject.XTrace);
    Yplusone=Updates(X(end,:));  %%% Updates the database with the optimal Xi parameters found by the fitted objective function
    if Yplusone>0
        Yplusone=0;
    end
    Y=bayesObject.ObjectiveTrace;
    Y(end)=Yplusone;
end

figure;
plot(Results(:,7),'Linewidth',1.5);
hold on
plot(Results(:,14),'o');

xlabel('Number of iteration','FontSize',20);
ylabel('Slope','FontSize',20);
set(gca,'FontSize',20,'LineWidth',1.5);

toc;

function f = mdlfun(tbl,mdl)
sat1 = tbl.s1;
sat2 = tbl.s2;
sat3 = tbl.s3;
delta1 = tbl.d1;
delta2 = tbl.d2;
delta3 = tbl.d3;
var = [sat1 sat2 sat3 delta1 delta2 delta3];
f = predict(mdl,var);
end

both fitrgp and fitrlinear works well, but fitrlinear is much faster than the fitrgp~~~

[Ceren Tarar]

Hi, what is the Updates function (Yplusone=Updates(X(end,:)); ) here? I can't find it anywhere else. Where is it defined?

[xu supeng]

Sorry, this is my initial program and is not clean or simple, if you are still interested, use the following example:

clc;
clear;
tic;

% define initial X value, but I didn't use initial Y value
X=[
    53.999    34.843    36.974    51.649     2      1      1      2     -4.1921    4.4694    -3.6857    -3.2385
    7.9335     72.09    39.635    53.757     2      1      1      2     -3.7329    4.6742    -2.6008    -2.9756
    32.198    29.476    16.324    59.771     2      1      1      2     -4.1383    5.1433     -3.368    -3.3614
    19.269    91.109    36.935     76.83     2      1      1      2     -4.0158    5.0661    -3.0922    -1.6616
    ];

% set the variable
pow1 = optimizableVariable('p1',[0,100],'Type','real');
pow2 = optimizableVariable('p2',[0,100],'Type','real');
pow3 = optimizableVariable('p3',[0,100],'Type','real');
pow4 = optimizableVariable('p4',[0,100],'Type','real');

pol1 = optimizableVariable('po1',[1,2],'Type','integer');
pol2 = optimizableVariable('po2',[1,2],'Type','integer');
pol3 = optimizableVariable('po3',[1,2],'Type','integer');
pol4 = optimizableVariable('po4',[1,2],'Type','integer');

det1 = optimizableVariable('d1',[-10,10],'Type','real');
det2 = optimizableVariable('d2',[-10,10],'Type','real');
det3 = optimizableVariable('d3',[-10,10],'Type','real');   
det4 = optimizableVariable('d4',[-10,10],'Type','real');

var=[pow1 pow2 pow3 pow4 pol1 pol2 pol3 pol4 det1 det2 det3 det4];

initialXList = table;

initialXList.p1 = X(:,1);
initialXList.p2 = X(:,2);
initialXList.p3 = X(:,3);
initialXList.p4 = X(:,4);

initialXList.po1 = X(:,5);
initialXList.po2 = X(:,6);
initialXList.po3 = X(:,7);
initialXList.po4 = X(:,8);

initialXList.d1 = X(:,9);
initialXList.d2 = X(:,10);
initialXList.d3 = X(:,11);
initialXList.d4 = X(:,12);


bayesObject = bayesopt(@(tbl)mdlfun(tbl),var,...
    'MaxObjectiveEvaluations',1000,...    % optimizaiton numbers
    'IsObjectiveDeterministic',true,...   
    'ExplorationRatio',0.5,...            % exploration degree, default value is 0.5
    'InitialX',initialXList,...           % start from the initial value
    'UseParallel',true);                  % use parallel

figure;
plot(bayesObject.ObjectiveTrace,'o');
hold on;
plot(-bayesObject.ObjectiveMinimumTrace,'Linewidth',2);

xlabel('Iteration times','FontSize',20);
ylabel('Capture velocity (m/s)','FontSize',20);
set(gca,'FontSize',20,'LineWidth',1.5);

toc;


function f = mdlfun(tbl)

pow1 = tbl.p1;
pow2 = tbl.p2;
pow3 = tbl.p3;
pow4 = tbl.p4;

pol1 = tbl.po1;
pol2 = tbl.po2;
pol3 = tbl.po3;
pol4 = tbl.po4;

det1= tbl.d1;
det2= tbl.d2;
det3= tbl.d3;
det4= tbl.d4;

var = [pow1 pow2 pow3 pow4 pol1 pol2 pol3 pol4 det1 det2 det3 det4];

f = black_box_function(var);   % black box function defined by yourself, once input a group of X, it will return a Y value
end

Answer 3

[M Mirrashid]

https://ch.mathworks.com/matlabcentral/fileexchange/112600-transit-search-optimization-algorithm-matlab-codes