Question from global optimization webinar

I thought I'd use this forum to post and answer some of the questions I've received from viewers of Global Optimization with MATLAB ( http://www.mathworks.com/company/events/webinars/wbnr43346.html).

0 - Where can I get the code? --> Here: MATLAB Central ( http://www.mathworks.com/matlabcentral/fileexchange/27178)

1 - How many different variables can be keyed into the optimziation tool? Is it only one, x, or can x,y,z be put in as well?

2 - What to use for multivariate global optimization where the data is instrumentally collected and multivariate in nature (with no known function)? In past, we have used genetic algorithms and simulated annealing.

3 - Can we also perform real Genetic optimization rather than binary in MATLAB?

4 - Is it possible to use the L1 norm instead of L2 statistics in the minimization or maximization of a given multivariate (matrix) equation. This is maybe more related to the estimation of coefficients.

Answer 1

[Stuart Kozola]

1 - Optimization tool: To clarify, let's start with what most optimization solvers in MATLAB take in as inputs. They take in a decision vector X. X can be any variable you choose, but it needs to be passed into the solver as a vector.

For example, lets say you have an objective function foo that takes in x,y,z as inputs:

f = foo(x,y,z)

To use this in optimization tool, or the solvers directly at the command line, you need to recast foo to accept X as in input only. this is easy to do using function handles:

objFun = @([x,y,z]) foo(x,y,z)

which can be called as:

X = [x,y,z]; f = objFun(X)

or using a single line of input (such as in Optimization Tool)

f = objFun([x,y,z]);

Answer 2

[Stuart Kozola]

2- Optimization of multivariate data sets. There are two approaches you can take to solve this problem.

A) fit a response surface (multivariate regression or similar) to the data and use this as your functional representation as input to the a solver. This would allow you to use a gradient based solver if the responses are approximately smooth. This is also often referred to as surrogate-based optimization (i.e. your fitting a surrogate model to the date). You can also use a design of experiments coupled with optimization approach to get data, find a minimum, and refine your search further with more data and repeat.

B) Optimize using only the data points. For this, you would need to use a gradient-free method since your domain is discrete (only has values where you have data). Genetic algorithms and simulated annealing are often used, but pattern search is often better here and just as good or better then GA/SA with the right options set. And in pattern search.

Answer 3

[Stuart Kozola]

3 - Is GA only binary?

There is no limitation to binary in the solver. You can create a custom data type (non binary) if you choose to. See this demo for an example:

http://www.mathworks.com/products/global-optimization/demos.html?file=/products/demos/shipping/globaloptim/traveling_salesman_demo.html

Also, your options in the variable type input to GA are:

PopulationType: [ 'bitstring' | 'custom' | {'doubleVector'} ]

You can also use custom functions to control how GA works (such as the population generation functions, crossover function, mutatation, ans so on...)

The particle swarm example shown in the webinar was done using custom functions.

Answer 4

[Stuart Kozola]

4 - Can you use different norms (L1, L2)?

In the webinar, I showed how to fit curves using the lsqcurvefit solver. This uses the standard L2 norm (see norm function in MATLAB). If you want to use a different norm, you can by writing the objective function for the norm you want. You will not use the lsqcurvefit solver, but would use a different minimization solver (see http://www.mathworks.com/help/toolbox/optim/ug/bqnk0r0.html#bqnk0x3-3 for the most appropriate solver for your problem type).

For example, if you wanted to fit data using an L1 norm instead, your objective function would look similar to this:

   function f = myL1norm(coeff,xdata,ydata)
      yhat = myModel(coeff,xdata); % your model that you want to fit
      f = norm(yhat-ydata,1)% the L1 norm

and you could use fminunc if it is unconstrained minimization:

   coeff = fminunc(@(x) myL1norm(x,xdata,ydata), coeffGuess);