How to get multi-solution using 'lsqnonlin'?

Hello everyone,

I'm now trying using 'lsqnonlin' to solve 32 equations to get the values of 10 variables. Theoretically speaking, there will be more than one solution on the interval [lb,ub], one of which should be the best. But I only got one solution in the end, which is shown as x_s. Another one should be at around x0, which is also my initial value. The subfunction F was to return all of 32 equations that should be solved, because that was too long, I don't put it here. I'm not expert in matlab, maybe I should use some other solvers instead of lsqnonlin. So do you anyone have better advice? Any help will be appreciated. Thanks in advance!

clear all
x0 = [36e-3,36e-3,36e-3,36e-3,45e-6,760e-6,3e-3,500e-6,45e-6,1e-3];
lb = [ 0,   0,  0,  0,  0,  0,  0,  0,  0,  0];
ub = [0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1];
options = optimoptions('lsqnonlin','Algorithm','levenberg-marquardt','Display','iter','MaxFunEvals',5e7,'MaxIter',5e6,'StepTolerance',1e-9, 'TolFun',1e-18);
L = lsqnonlin(@equ,x0,lb,ub,options);
% x_s = [0.0360,0.0360,0.0359,0.0359,0.0034,0.0048,0.0034,6.2068e-11,3.2405e-4,0.0126];
function F = equ(x)
.
.
.
return
end

Answer 1

[John D'Errico]

lsqnonlin is a numerical optimization tool. It iterations fro mthe start point to a solution. And the solution you get is dependent on where you start.

The analogy I use is to imagine a blind person, set down on the face of the earth, and tasked with finding the point of lowest elevation. All he can do to determine where to walk next is a cane. Please, give him some scuba gear. But it matters a lot if you put him down in different locations. Near the dead sea, for example, would be a bad start point, since he will get stuck in an unfortunate spot. When the person can no longer find a route down from where they are standing, they stop, and annnounce a solution.

There is a concept known as the basin of attraction. So if you picked many start points, all of which were close to the same spot, then they will all end up at essentially the same final point. The basin of attraction for any solution is the set of start points that will converge to that solution.

The point is, you get ONE answer, and one only. If you want multiple solutions, then start at multiple various start points, and hope that one of them is near the lowest point. Intelligently chosen start points are likely better than not.

[Runze Zhang]

Thanks for your answer again, John. That really makes sense. But what did you mean by starting at multiple start point? I think for each iteration I can only choose one start point. Could you please explain it further?

[Alan Weiss]

All he means is that after you get one answer, try again with a different starting point. You might then get a dfferent answer. You can also do this programmatically. Or, if you have Global Optimization Toolbox, get MultiStart to do it for you automatically. For an example, see MultiStart Using lsqcurvefit or lsqnonlin.

Alan Weiss

MATLAB mathematical toolbox documentation

[John D'Errico]

You can just run lsqnonlin in a loop, saving the results after every pass, unless lsqnonlin thinks it failed to converge on that run.. When done, then I like to think of it as a clustering problem. Many of the solutions found will be esssentially the same. Lump them together.

Or better, use multistart, which does much of the work for you.

[Runze Zhang]

Thanks for your answer, Alan and John. Multistart really helps. But for John's advice, firstly do you mean that if run lsqnonlin in a loop, save every result and put it to the initial point of the next iteration? If so, I already had some same solutions as you said, and what do you mean by lump them together and how can I do that? Could you please explain further in details?

[Alan Weiss]

MultiStart does the clumping work for you. But if for some reason you want to do it yourself, I suggest that you examine my post on Loren's blog Selecting the Granularity You Want in GlobalSearch or MultiStart.

Alan Weiss

MATLAB mathematical toolbox documentation

[Runze Zhang]

Hello, Alan. I have gone through your post. And further question is that when I'm using clumpthem, it said unrecognized. And I tried to find the help, but there is nothing. I'm using R2021a, is there anything changed for clumpthem function? The second one is in the results of the local optimum point, they alway contain the boundary value, which obviously don't make any sense, that is also the question that always confused me. How can I solve it?

[Alan Weiss]

Please look in the second paragraph of that post where it says "The other involves the clumpthem function; download it here." Click on the word "here" (you can click on the word in this answer as well).

Alan Weiss

MATLAB mathematical toolbox documentation

[Runze Zhang]

Thanks, Alan. I'm sorry that I overlooked it.