How to solve coupled optimization problem in MATLAB?

I've got a problem of solving coupled optimization problem in MATLAB. For example,

The objective function E(a,b) contains two sets of variables need to be optimized:

E(x, a) = (y - W(a)x)^2;

Assume, W(a) is a transformation matrix, and x is pixel intensity. Essentially, we would like to optimize w.r.t both x and a.

What I have done is using MATLAB medium-scale Quasi-Newton solver by fminunc; however, the optimization doesn't work quite well.

I have proposed some open discussion here is anyone have got any opinions please tell me.

1. If my cost function is really hard to get the analytical gradient and Hessian. How can I use Matlab Large-scale by fminunc?

2. Is fminunc suitable for nonlinear least squares problem, or MATLAB provides other solver?

3. Is MATLAB optimizer good for this kind of coupled optimization?

4. Is there anyway to linearize the cost function?

5. Any ideas where can I find the third-party nonlinear optimizer written in MATLAB?

[Rakesh Kumar]

Have you tried lsqnonlin solver in optimization toolbox?

Answer 1

[Steve Grikschat]

I think that you want to use lsqnonlin or lsqcurvefit to solve your problem. They are built to handle problems of the form:

min norm(f(x))^2

where f(x) is a system of nonlinear functions. lsqnonlin exploits the inherent structure in a least-squares problem and uses it to construct quality estimates for the second derivatives. These estimates have better error bounds than the quasi-Newton matrices as employed in fminunc.

An analytical Jacobian (for lsqnonlin) improves your chances for solving the problem quickly, but is not strictly necessary. For a large problem (which I assume this might be depending on the size of the image), an analytical Jacobian becomes crucial for reasonable computation time.