Accelerating the pace of engineering and science

# Global Optimization Toolbox

## Simulated Annealing Options

This example shows how to create and manage options for the simulated annealing function SIMULANNEALBND using SAOPTIMSET in the Global Optimization Toolbox.

Optimization Problem Setup

SIMULANNEALBND searches for a minimum of a function using simulated annealing. For this example we use SIMULANNEALBND to minimize the objective function DEJONG5FCN. This function is a real valued function of two variables and has many local minima making it difficult to optimize. There is only one global minimum at x=(-32,-32), where f(x) = 0.998. To define our problem, we must define the objective function, start point, and bounds specified by the range -64 <= x(i) <= 64 for each x(i).

```ObjectiveFunction = @dejong5fcn; startingPoint = [-30 0]; lb = [-64 -64]; ub = [64 64]; ```

The function PLOTOBJECTIVE in the toolbox plots the objective function over the range -64 <= x1 <= 64, -64 <= x2 <= 64.

```plotobjective(ObjectiveFunction,[-64 64; -64 64]); view(-15,150); ```

Now, we can run the SIMULANNEALBND solver to minimize our objective function.

```[x,fval,exitFlag,output] = simulannealbnd(ObjectiveFunction,startingPoint,lb,ub); fprintf('The number of iterations was : %d\n', output.iterations); fprintf('The number of function evaluations was : %d\n', output.funccount); fprintf('The best function value found was : %g\n', fval); ```
```Optimization terminated: change in best function value less than options.TolFun. The number of iterations was : 1095 The number of function evaluations was : 1104 The best function value found was : 2.98211 ```

Note that when you run this example, your results may be different from the results shown above because simulated annealing algorithm uses random numbers to generate points.

SIMULANNEALBND can accept one or more plot functions through an 'options' argument. This feature is useful for visualizing the performance of the solver at run time. Plot functions are selected using SAOPTIMSET. The help for SAOPTIMSET contains a list of plot functions to choose from, or you can provide your own custom plot functions.

To select multiple plot functions, SAOPTIMSET is used to create an options structure. For this example, we select SAPLOTBESTF, which plots the best function value every iteration, SAPLOTTEMPERATURE, which shows the current temperature in each dimension at every iteration, SAPLOTF, which shows the current function value (remember that the current value is not necessarily the best one), and SAPLOTSTOPPING, which plots the percentage of stopping criteria satisfied every ten iterations.

```options = saoptimset('PlotInterval',10, ... 'PlotFcns',{@saplotbestf,@saplottemperature,@saplotf,@saplotstopping}); ```

Run the solver.

```simulannealbnd(ObjectiveFunction,startingPoint,lb,ub,options); ```
```Optimization terminated: change in best function value less than options.TolFun. ```

Specifying Temperature Options

The temperature parameter used in simulated annealing controls the overall search results. The temperature for each dimension is used to limit the extent of search in that dimension. The toolbox lets you specify initial temperature as well as ways to update temperature during the solution process. The two temperature-related options are the `InitialTemperature` and the `TemperatureFcn`.

Specifying initial temperature

The default initial temperature is set to 100 for each dimension. If you want the initial temperature to be different in different dimensions then you must specify a vector of temperatures. This may be necessary in cases when problem is scaled differently in each dimensions. For example,

```options = saoptimset('InitialTemperature',[300 50]); ```

`InitialTemperature` can be set to a vector of length less than the number of variables (dimension); the solver expands the vector to the remaining dimensions by taking the last element of the initial temperature vector. Here we want the initial temperature to be the same in all dimensions so we need only specify the single temperature.

```options = saoptimset('InitialTemperature',100); ```

Specifying a temperature function

The default temperature function used by SIMULANNEALBND is called TEMPERATUREEXP. In the temperatureexp schedule, the temperature at any given step is .95 times the temperature at the previous step. This causes the temperature to go down slowly at first but ultimately get cooler faster than other schemes. If another scheme is desired, e.g. Boltzmann schedule or "Fast" schedule annealing, then TEMPERATUREBOLTZ or TEMPERATUREFAST can be used respectively. To select the fast temperature schedule, we can update our previously created options structure 'options' by passing 'options' to SAOPTIMSET.

```options = saoptimset(options,'TemperatureFcn',@temperaturefast); ```

Specifying reannealing

Reannaling is a part of annealing process. After a certain number of new points are accepted, the temperature is raised to a higher value in hope to restart the search and move out of a local minima. Performing reannealing too soon may not help the solver identify a minimum, so a relatively high interval is a good choice. The interval at which reannealing happens can be set using the `ReannealInterval` option. Here, we reduce the default reannealing interval to 50 because the function seems to be flat in many regions and solver might get stuck rapidly.

```options = saoptimset(options,'ReannealInterval',50); ```

We can also set the `Display` option to control the level of display on the command prompt. Here we set the display option to 'iter' and change the display interval to 400.

```options = saoptimset(options,'Display','iter','DisplayInterval',400); ```

Now that we have setup the new temperature options we run the solver again

```[x,fval,exitFlag,output] = simulannealbnd(ObjectiveFunction,startingPoint,lb,ub,options); ```
``` Best Current Mean Iteration f-count f(x) f(x) temperature 0 1 65.2159 65.2159 100 * 166 169 21.0727 21.0729 17.7907 * 358 363 16.4409 16.7827 12.2086 400 405 16.4409 20.3586 1.88714 * 545 552 16.4409 16.4417 14.1675 * 718 727 16.4409 16.6691 12.3336 800 809 16.4409 17.2669 1.07018 * 920 931 16.4409 16.4409 15.8571 * 1116 1129 16.4409 16.4474 15.5745 1200 1213 16.4409 16.4417 1.10255 * 1291 1306 16.4409 16.4452 16.1383 Optimization terminated: change in best function value less than options.TolFun. ```

Notice that every iteration at which reannealing happens is denoted by a '*' and is always displayed.

```fprintf('The number of iterations was : %d\n', output.iterations); fprintf('The number of function evaluations was : %d\n', output.funccount); fprintf('The best function value found was : %g\n', fval); ```
```The number of iterations was : 1306 The number of function evaluations was : 1321 The best function value found was : 16.4409 ```

Reproducing Results

SIMULANNEALBND is a nondeterministic algorithm. This means that running the solver more than once without changing any settings may give different results. This is because SIMULANNEALBND utilizes MATLAB® random number generators when it generates subsequent points and also when it determines whether or not to accept new points. Every time a random number is generated the state of the random number generators change.

To see this, two runs of SIMULANNEALBND solver yields:

```[x,fval] = simulannealbnd(ObjectiveFunction,startingPoint,lb,ub,options); fprintf('The best function value found was : %g\n', fval); ```
``` Best Current Mean Iteration f-count f(x) f(x) temperature 0 1 65.2159 65.2159 100 * 178 181 10.7632 10.8556 306.84 * 337 342 1.99203 2.23278 43.5661 400 405 1.99203 3.80458 1.50999 * 529 536 1.99203 1.993 120.264 * 711 720 1.99203 1.99252 19.8934 800 809 1.99203 1.99279 1.06345 * 905 916 1.99203 1.99203 5032.9 * 1086 1099 1.99203 1.99221 12.0733 1200 1213 1.99203 2.17076 0.792482 * 1268 1283 1.99203 1.99208 13.7625 Optimization terminated: change in best function value less than options.TolFun. The best function value found was : 1.99203 ```

And,

```[x,fval] = simulannealbnd(ObjectiveFunction,startingPoint,lb,ub,options); fprintf('The best function value found was : %g\n', fval); ```
``` Best Current Mean Iteration f-count f(x) f(x) temperature 0 1 65.2159 65.2159 100 * 194 197 10.7632 10.772 32.3858 * 395 400 10.7632 11.2931 12.9278 400 405 10.7632 11.2931 7.44822 * 578 585 10.7632 10.7644 16.0489 * 768 777 10.7632 12.0341 13.0872 800 809 10.7632 12.0341 2.4326 * 931 942 10.7632 10.7634 16.1797 * 1105 1118 10.7632 11.7832 116.913 Optimization terminated: change in best function value less than options.TolFun. The best function value found was : 10.7632 ```

In the previous two runs SIMULANNEALBND gives different results.

We can reproduce our results if we reset the states of the random number generators between runs of the solver by using information returned by SIMULANNEALBND. SIMULANNEALBND returns the states of the random number generators at the time SIMULANNEALBND is called in the output argument. This information can be used to reset the states. Here we reset the states between runs using this output information so the results of the next two runs are the same.

```[x,fval,exitFlag,output] = simulannealbnd(ObjectiveFunction,startingPoint,lb,ub,options); fprintf('The best function value found was : %g\n', fval); ```
``` Best Current Mean Iteration f-count f(x) f(x) temperature 0 1 65.2159 65.2159 100 * 193 196 20.1535 20.1537 17.8501 * 381 386 20.1535 20.5812 20.9127 400 405 20.1535 20.5812 4.19923 * 574 581 20.1535 20.1543 16.6134 * 751 760 20.1535 20.304 13.2824 800 809 20.1535 20.304 1.72145 * 939 950 20.1535 20.1589 16.3603 * 1117 1130 20.1535 20.155 15.8317 1200 1213 20.1535 20.3304 1.11621 * 1299 1314 20.1535 20.1686 12.9936 * 1487 1504 20.1535 20.1535 14.4676 Optimization terminated: change in best function value less than options.TolFun. The best function value found was : 20.1535 ```

We reset the state of the random number generator.

```RandStream.getGlobalStream.State = output.rngstate.state; ```

Now, let's run SIMULANNEALBND again.

```[x,fval] = simulannealbnd(ObjectiveFunction,startingPoint,lb,ub,options); fprintf('The best function value found was : %g\n', fval); ```
``` Best Current Mean Iteration f-count f(x) f(x) temperature 0 1 65.2159 65.2159 100 * 179 182 11.7187 11.7211 33.0831 * 369 374 11.7187 11.7596 180.265 400 405 11.7187 11.7596 2.97641 * 551 558 11.7187 12.5042 21.3737 * 729 738 11.7187 12.0319 53.538 800 809 6.91013 8.26212 1.34928 * 897 908 6.90334 6.91869 55.3227 * 1099 1112 6.90334 6.90842 13.1488 1200 1213 6.90334 6.92373 0.907765 * 1266 1281 6.90334 6.90334 31.4322 * 1462 1479 6.90334 7.11941 13.1345 1600 1617 6.90334 7.00217 0.678746 * 1636 1655 6.90334 6.90523 21.6782 * 1807 1828 6.90334 7.86653 17.3648 Optimization terminated: change in best function value less than options.TolFun. The best function value found was : 6.90334 ```

Modifying the Stopping Criteria

SIMULANNEALBND uses six different criteria to determine when to stop the solver. SIMULANNEALBND stops when the maximum number of iterations or function evaluation is exceeded; by default the maximum number of iterations is set to Inf and the maximum number of function evaluations is `3000*numberOfVariables`. SIMULANNEALBND keeps track of the average change in the function value for `StallIterLimit` iterations. If the average change is smaller than the function tolerance, `TolFun`, then the algorithm will stop. The solver will also stop when the objective function value reaches `ObjectiveLimit`. Finally the solver will stop after running for `TimeLimit` seconds. Here we set the `TolFun` to 1e-5.

```options = saoptimset(options,'TolFun',1e-5); ```

Run the SIMULANNEALBND solver.

```[x,fval,exitFlag,output] = simulannealbnd(ObjectiveFunction,startingPoint,lb,ub,options); fprintf('The number of iterations was : %d\n', output.iterations); fprintf('The number of function evaluations was : %d\n', output.funccount); fprintf('The best function value found was : %g\n', fval); ```
``` Best Current Mean Iteration f-count f(x) f(x) temperature 0 1 65.2159 65.2159 100 * 208 211 2.98211 3.04982 14.142 * 391 396 2.98211 3.0109 23.1227 400 405 2.98211 3.0109 7.45459 * 576 583 2.98211 3.26265 18.3405 * 746 755 2.98211 3.61504 26.7933 800 809 2.98211 3.61504 1.72351 * 935 946 2.98211 3.1177 15.1217 * 1097 1110 2.98211 2.98211 50.7624 Optimization terminated: change in best function value less than options.TolFun. The number of iterations was : 1146 The number of function evaluations was : 1159 The best function value found was : 2.98211 ```