fminunc

Find minimum of unconstrained multivariable function

Description

Nonlinear programming solver.

Finds the minimum of a problem specified by

$\underset{x}{\mathrm{min}}f\left(x\right)$

where f(x) is a function that returns a scalar.

x is a vector or a matrix; see Matrix Arguments.

example

x = fminunc(fun,x0) starts at the point x0 and attempts to find a local minimum x of the function described in fun. The point x0 can be a scalar, vector, or matrix.

Note

Passing Extra Parameters explains how to pass extra parameters to the objective function and nonlinear constraint functions, if necessary.

fminunc is for nonlinear problems without constraints. If your problem has constraints, generally use fmincon. See Optimization Decision Table.

example

x = fminunc(fun,x0,options) minimizes fun with the optimization options specified in options. Use optimoptions to set these options.

example

x = fminunc(problem) finds the minimum for problem, a structure described in problem.

example

[x,fval] = fminunc(___), for any syntax, returns the value of the objective function fun at the solution x.

example

[x,fval,exitflag,output] = fminunc(___) additionally returns a value exitflag that describes the exit condition of fminunc, and a structure output with information about the optimization process.

• hessian — Hessian of fun at the solution x. See fminunc Hessian.

Examples

collapse all

Minimize the function $f\left(x\right)=3{x}_{1}^{2}+2{x}_{1}{x}_{2}+{x}_{2}^{2}-4{x}_{1}+5{x}_{2}$.

To do so, write an anonymous function fun that calculates the objective.

fun = @(x)3*x(1)^2 + 2*x(1)*x(2) + x(2)^2 - 4*x(1) + 5*x(2);

Call fminunc to find a minimum of fun near [1,1].

x0 = [1,1];
[x,fval] = fminunc(fun,x0)
Local minimum found.

Optimization completed because the size of the gradient is less than
the value of the optimality tolerance.
x = 1×2

2.2500   -4.7500

fval = -16.3750

fminunc can be faster and more reliable when you provide derivatives.

Write an objective function that returns the gradient as well as the function value. Use the conditionalized form described in Including Gradients and Hessians. The objective function is Rosenbrock's function,

$f\left(x\right)=100{\left({x}_{2}-{x}_{1}^{2}\right)}^{2}+\left(1-{x}_{1}{\right)}^{2},$

$\nabla f\left(x\right)=\left[\begin{array}{c}-400\left({x}_{2}-{x}_{1}^{2}\right){x}_{1}-2\left(1-{x}_{1}\right)\\ 200\left({x}_{2}-{x}_{1}^{2}\right)\end{array}\right]$.

The code for the objective function with gradient appears at the end of this example.

Create options to use the objective function’s gradient. Also, set the algorithm to 'trust-region'.

Set the initial point to [-1,2]. Then call fminunc.

x0 = [-1,2];
x = fminunc(fun,x0,options)
Local minimum found.

Optimization completed because the size of the gradient is less than
the value of the optimality tolerance.
x = 1×2

1.0000    1.0000

The following code creates the rosenbrockwithgrad function, which includes the gradient as the second output.

% Calculate objective f
f = 100*(x(2) - x(1)^2)^2 + (1-x(1))^2;

if nargout > 1 % gradient required
g = [-400*(x(2)-x(1)^2)*x(1) - 2*(1-x(1));
200*(x(2)-x(1)^2)];
end
end

Solve the same problem as in Supply Gradient using a problem structure instead of separate arguments.

Write an objective function that returns the gradient as well as the function value. Use the conditionalized form described in Including Gradients and Hessians. The objective function is Rosenbrock's function,

$f\left(x\right)=100{\left({x}_{2}-{x}_{1}^{2}\right)}^{2}+\left(1-{x}_{1}{\right)}^{2}$,

$\nabla f\left(x\right)=\left[\begin{array}{c}-400\left({x}_{2}-{x}_{1}^{2}\right){x}_{1}-2\left(1-{x}_{1}\right)\\ 200\left({x}_{2}-{x}_{1}^{2}\right)\end{array}\right]$.

The code for the objective function with gradient appears at the end of this example.

Create options to use the objective function’s gradient. Also, set the algorithm to 'trust-region'.

Create a problem structure including the initial point x0 = [-1,2]. For the required fields in this structure, see problem.

problem.options = options;
problem.x0 = [-1,2];
problem.solver = 'fminunc';

Solve the problem.

x = fminunc(problem)
Local minimum found.

Optimization completed because the size of the gradient is less than
the value of the optimality tolerance.
x = 1×2

1.0000    1.0000

The following code creates the rosenbrockwithgrad function, which includes the gradient as the second output.

% Calculate objective f
f = 100*(x(2) - x(1)^2)^2 + (1-x(1))^2;

if nargout > 1 % gradient required
g = [-400*(x(2)-x(1)^2)*x(1)-2*(1-x(1));
200*(x(2)-x(1)^2)];
end
end

Find both the location of the minimum of a nonlinear function and the value of the function at that minimum. The objective function is

$f\left(x\right)=x\left(1\right){e}^{-{‖x‖}_{2}^{2}}+{‖x‖}_{2}^{2}/20$.

fun = @(x)x(1)*exp(-(x(1)^2 + x(2)^2)) + (x(1)^2 + x(2)^2)/20;

Find the location and objective function value of the minimizer starting at x0 = [1,2].

x0 = [1,2];
[x,fval] = fminunc(fun,x0)
Local minimum found.

Optimization completed because the size of the gradient is less than
the value of the optimality tolerance.
x = 1×2

-0.6691    0.0000

fval = -0.4052

Choose fminunc options and outputs to examine the solution process.

Set options to obtain iterative display and use the 'quasi-newton' algorithm.

options = optimoptions(@fminunc,'Display','iter','Algorithm','quasi-newton');

The objective function is

$f\left(x\right)=x\left(1\right){e}^{-{‖x‖}_{2}^{2}}+{‖x‖}_{2}^{2}/20.$

fun = @(x)x(1)*exp(-(x(1)^2 + x(2)^2)) + (x(1)^2 + x(2)^2)/20;

Start the minimization at x0 = [1,2], and obtain outputs that enable you to examine the solution quality and process.

x0 = [1,2];
[x,fval,exitflag,output] = fminunc(fun,x0,options)
First-order
Iteration  Func-count       f(x)        Step-size       optimality
0           3         0.256738                         0.173
1           6         0.222149              1          0.131
2           9          0.15717              1          0.158
3          18        -0.227902       0.438133          0.386
4          21        -0.299271              1           0.46
5          30        -0.404028       0.102071         0.0458
6          33        -0.404868              1         0.0296
7          36        -0.405236              1        0.00119
8          39        -0.405237              1       0.000252
9          42        -0.405237              1       7.97e-07

Local minimum found.

Optimization completed because the size of the gradient is less than
the value of the optimality tolerance.
x = 1×2

-0.6691    0.0000

fval = -0.4052
exitflag = 1
output = struct with fields:
iterations: 9
funcCount: 42
stepsize: 2.9343e-04
lssteplength: 1
firstorderopt: 7.9721e-07
algorithm: 'quasi-newton'
message: '...'

• The exit flag 1 shows that the solution is a local optimum.

• The output structure shows the number of iterations, number of function evaluations, and other information.

• The iterative display also shows the number of iterations and function evaluations.

Input Arguments

collapse all

Function to minimize, specified as a function handle or function name. fun is a function that accepts a vector or array x and returns a real scalar f, the objective function evaluated at x.

fminunc passes x to your objective function in the shape of the x0 argument. For example, if x0 is a 5-by-3 array, then fminunc passes x to fun as a 5-by-3 array.

Specify fun as a function handle for a file:

x = fminunc(@myfun,x0)

where myfun is a MATLAB® function such as

function f = myfun(x)
f = ...            % Compute function value at x

You can also specify fun as a function handle for an anonymous function:

x = fminunc(@(x)norm(x)^2,x0);

If you can compute the gradient of fun and the SpecifyObjectiveGradient option is set to true, as set by

then fun must return the gradient vector g(x) in the second output argument.

If you can also compute the Hessian matrix and the HessianFcn option is set to 'objective' via options = optimoptions('fminunc','HessianFcn','objective') and the Algorithm option is set to 'trust-region', fun must return the Hessian value H(x), a symmetric matrix, in a third output argument. fun can give a sparse Hessian. See Hessian for fminunc trust-region or fmincon trust-region-reflective algorithms for details.

The trust-region algorithm allows you to supply a Hessian multiply function. This function gives the result of a Hessian-times-vector product without computing the Hessian directly. This can save memory. See Hessian Multiply Function.

Example: fun = @(x)sin(x(1))*cos(x(2))

Data Types: char | function_handle | string

Initial point, specified as a real vector or real array. Solvers use the number of elements in x0 and the size of x0 to determine the number and size of variables that fun accepts.

Example: x0 = [1,2,3,4]

Data Types: double

Optimization options, specified as the output of optimoptions or a structure such as optimset returns.

Some options apply to all algorithms, and others are relevant for particular algorithms. See Optimization Options Reference for detailed information.

Some options are absent from the optimoptions display. These options appear in italics in the following table. For details, see View Options.

Problem structure, specified as a structure with the following fields:

Field NameEntry

objective

Objective function

x0

Initial point for x

solver

'fminunc'

options

Options created with optimoptions

Data Types: struct

Output Arguments

collapse all

Solution, returned as a real vector or real array. The size of x is the same as the size of x0. Typically, x is a local solution to the problem when exitflag is positive. For information on the quality of the solution, see When the Solver Succeeds.

Objective function value at the solution, returned as a real number. Generally, fval = fun(x).

Reason fminunc stopped, returned as an integer.

 1 Magnitude of gradient is smaller than the OptimalityTolerance tolerance. 2 Change in x was smaller than the StepTolerance tolerance. 3 Change in the objective function value was less than the FunctionTolerance tolerance. 5 Predicted decrease in the objective function was less than the FunctionTolerance tolerance. 0 Number of iterations exceeded MaxIterations or number of function evaluations exceeded MaxFunctionEvaluations. -1 Algorithm was terminated by the output function. -3 Objective function at current iteration went below ObjectiveLimit.

Information about the optimization process, returned as a structure with fields:

 iterations Number of iterations taken funcCount Number of function evaluations firstorderopt Measure of first-order optimality algorithm Optimization algorithm used cgiterations Total number of PCG iterations ('trust-region' algorithm only) lssteplength Size of line search step relative to search direction ('quasi-newton' algorithm only) stepsize Final displacement in x message Exit message

Gradient at the solution, returned as a real vector. grad gives the gradient of fun at the point x(:).

Approximate Hessian, returned as a real matrix. For the meaning of hessian, see Hessian Output.

Algorithms

collapse all

Quasi-Newton Algorithm

The quasi-newton algorithm uses the BFGS Quasi-Newton method with a cubic line search procedure. This quasi-Newton method uses the BFGS (,,, and ) formula for updating the approximation of the Hessian matrix. You can select the DFP (,, and ) formula, which approximates the inverse Hessian matrix, by setting the HessUpdate option to 'dfp' (and the Algorithm option to 'quasi-newton'). You can select a steepest descent method by setting HessUpdate to 'steepdesc' (and Algorithm to 'quasi-newton'), although this setting is usually inefficient. See fminunc quasi-newton Algorithm.

Trust Region Algorithm

The trust-region algorithm requires that you supply the gradient in fun and set SpecifyObjectiveGradient to true using optimoptions. This algorithm is a subspace trust-region method and is based on the interior-reflective Newton method described in  and . Each iteration involves the approximate solution of a large linear system using the method of preconditioned conjugate gradients (PCG). See fminunc trust-region Algorithm, Trust-Region Methods for Nonlinear Minimization and Preconditioned Conjugate Gradient Method.

Alternative Functionality

App

The Optimize Live Editor task provides a visual interface for fminunc.

 Broyden, C. G. “The Convergence of a Class of Double-Rank Minimization Algorithms.” Journal Inst. Math. Applic., Vol. 6, 1970, pp. 76–90.

 Coleman, T. F. and Y. Li. “An Interior, Trust Region Approach for Nonlinear Minimization Subject to Bounds.” SIAM Journal on Optimization, Vol. 6, 1996, pp. 418–445.

 Coleman, T. F. and Y. Li. “On the Convergence of Reflective Newton Methods for Large-Scale Nonlinear Minimization Subject to Bounds.” Mathematical Programming, Vol. 67, Number 2, 1994, pp. 189–224.

 Davidon, W. C. “Variable Metric Method for Minimization.” A.E.C. Research and Development Report, ANL-5990, 1959.

 Fletcher, R. “A New Approach to Variable Metric Algorithms.” Computer Journal, Vol. 13, 1970, pp. 317–322.

 Fletcher, R. “Practical Methods of Optimization.” Vol. 1, Unconstrained Optimization, John Wiley and Sons, 1980.

 Fletcher, R. and M. J. D. Powell. “A Rapidly Convergent Descent Method for Minimization.” Computer Journal, Vol. 6, 1963, pp. 163–168.

 Goldfarb, D. “A Family of Variable Metric Updates Derived by Variational Means.” Mathematics of Computing, Vol. 24, 1970, pp. 23–26.

 Shanno, D. F. “Conditioning of Quasi-Newton Methods for Function Minimization.” Mathematics of Computing, Vol. 24, 1970, pp. 647–656.