linear least square with nonlinear constraints

I currently have a system of equations y = A*x. y is 18 x 1 vector while matrix A is of size 18x12. x is 12 x 1 parameter vector which is to be determined. Additionally, there are three nonlinear constraints which I want to add. Using fmincon I am not able to formulate the objective function in terms of x(1)..........x(12) for 18 equations. Is there other way to describe the objective function like A*[x(1); x(2);...........x(12)] or how I can solve this problem in matlab?

[Torsten]

The objective function is usually taken as

(y-A*x).' * (y-A*x)

Answer 1

[John D'Errico]

Fmincon requires a single objective to optimize. So if you have a set of 18 equations, in 12 unknowns, then you formulate the objective in terms of the sum of squares of the errors of the equations. For example. Suppose I wanted to solve the simple linear least squares problem

A*x = y

using fmincon, where we have essentially 10 equations and 5 unknowns. (I'm solving a smaller problem here because it will be easier to look at the results.) For now, assume there are no constraints at all. Since I am far too lazy to actually come up with a real problem, here is my test example:

A = randn(10,5);
y = randn(10,1);

So an entirely valid problem. Now we wish to solve the initially unconstrained problem. Just as a test to see the result was successful, we need to know the solution using simple linear least squares.

x0 = A\y

x0 = 5×1

1.0162 0.6095 1.0156 -0.8111 -0.5528

Now, let me try using fmincon.

obj = @(x) norm(A*x-y);
xstart = randn(5,1); % too lazy to think of a better set
[x1,fval,exitflag] = fmincon(obj,xstart)

Local minimum found that satisfies the constraints.

Optimization completed because the objective function is non-decreasing in 
feasible directions, to within the value of the optimality tolerance,
and constraints are satisfied to within the value of the constraint tolerance.

x1 = 5×1

1.0162 0.6095 1.0156 -0.8111 -0.5528

fval = 2.2128

exitflag = 1

Well, I'll be a horned toad! It worked! I got the same result. Of course, it was way more computational effort, the MATLAB equivalent of using a Mack truck to take a pea to Boston. But fmincon gave the same result.

I could have defined the objective function as (A*x-y)'*(A*x-y). But norm does just as well, and it is shorter to write, and since norm by default is the 2-norm, it is effectively the same thing. Well, it would be, if I squared the norm. But since squaring a positive number is a monotonic transformation, the location of the minimum is unchanged.

Now, suppose I wanted to enforce a nonlinear constraint? Trivial. Lets see. I'll decide to enforce that the sum of the squares of the x(i) must be less than 1. I'll throw in an equality constraint too. How about that the sum of the elements must be 0.5? The function nonlcons

[C,CEQ] = nonlcons(x1)

C = 2.3993

CEQ = 0.7774

So the solution generated using the simple linear least squares fails these requirements.

[x2,fval,exitflag] = fmincon(obj,xstart,[],[], [],[], [],[], @nonlcons)

Local minimum found that satisfies the constraints.

Optimization completed because the objective function is non-decreasing in 
feasible directions, to within the value of the optimality tolerance,
and constraints are satisfied to within the value of the constraint tolerance.

x2 = 5×1

0.3520 0.3542 0.6118 -0.5535 -0.2645

fval = 2.4915

exitflag = 1

Does it satisfy the constraints? Of course, at least to within trash on the order of the tolerances employed.

[C,CEQ] = nonlcons(x2)

C = -2.2892e-06

CEQ = -1.1102e-16

function [C,CEQ] = nonlcons(X)
  C = sum(X.^2) - 1; % this must be less than zero.
  CEQ = sum(X) - .5;
end

[Vikas Saroha]

Thanks. It is really useful.