Problem about boundary conditions with lsqnonlin

Dear Sirs,

I would like to ask you something about minimizing a system of non-linear equations. I have 16 equations and 16 unknown paramerters. I wrote a function which give me a vector F (16x1). As the system is composed of non-linear equations and as I need to fix boundary conditions to the solutions I am using the lsqnonlin function.

My problem is regarding to the boundary conditions. I am using random init points and for each one I get a solutions vector in which at least one parameter is almost equal to its boundary condition. For exemple I attached below an example :

1.55001E-06
6.8431E-05
0.185118471
0.000668459
*1.010426164*
*1.00590155*
1.750684827
3.883598517
0.00037502
4.40766E-05
0.384243325
0.38198906
*1.000080588*
4.823156479
1.710537934
1.684077757

lb=[0 0 0 0 1.0 1.0 1.0 1.0 0 0 0 0 1.0 1.0 1.0 1.0]';
ub=[1.0 1.0 1.0 1.0 5 5 5 5 1.0 1.0 1.0 1.0 5 5 2.6 5]';

In bold, the values are equal to the boundary conditions I fixed. I would like to know if there is a possibility to force the algorithm to push away from this bound when approaching ? I just would like to minimize my system and a local minimum inside ]lb;ub[ should exist far away from these boundary values.

If I change this boundary conditions, the same phenomenon still appears.

Thank you very much in advance

[Torsten]

Usually, if you have 16 equations for 16 unknown parameters, your problem should have a unique solution. If you now constrain the variables to lie within some bounds, you will always hit these bounds for some of the variables. What do you get if you don't use any bounds ?

Best wishes

Torsten.

[Matt J]

In addition to what Torsten said, it looks like your solution violates the bounds by a significant amount. Have you been making your own adjustments to the TolCon parameter? We probably need to see your code...

[Gigi57]

Thank you very much for your answer.

I also tried without any bounds but I get some unphysical parameters like these one :

2,33172974808927e-05

0,000553689391623560

0,0751513535517482

5,92012764654519e-06

4,08900460894902

2,96266586669310

2,64491484839856

6,13070815487002

-0,0105996466370521

-0,0538439337929134

-13,4329514089577

3,35096937521886

3,57644907600772

3,33532875456763

-0,0950303011801077

1,36144912185482

with an exitflag of 3.

I let the default TolCon value as shown in my code below :

clear all;

F = @ljtest

format long

lb=[0 0 0 0 1.0 1.0 1.0 1.0 0 0 0 0 1.0 1.0 1.0 1.0]';

ub=[1.0 1.0 1.0 1.0 5 5 5 5 1.0 1.0 1.0 1.0 5 5 2.6 5]';

Epsmin=0.0

SigHmin=2.0

Sigmin=2.0

Epsmax=0.8

EpsHmax=0.8

Sigmax=4.0

n=100

x=zeros(16,n);

Fsumvalue=zeros(18,n);

xmin=[Epsmin Epsmin Epsmin Epsmin SigHmin SigHmin Sigmin Sigmin Epsmin Epsmin Epsmin Epsmin SigHmin SigHmin Sigmin Sigmin]';

xmax=[EpsHmax EpsHmax Epsmax Epsmax Sigmax Sigmax Sigmax Sigmax EpsHmax EpsHmax Epsmax Epsmax Sigmax Sigmax Sigmax Sigmax]';

options = optimoptions('lsqnonlin','Display','off','MaxIter',10000,'MaxFunEvals',10000000);

for k=1:n

    x0(:,k)=xmin+(xmax-xmin).*rand(16,1)
    [x(:,k),resnorm,residual,exitflag,output]=lsqnonlin(F,x0(:,k),lb,ub,options)

    Fsumvalue(1,k)=resnorm

    Fsumvalue(2,k)=exitflag

    Fsumvalue(3:18,k)=x(:,k)

end

format long

xlswrite('Fsumvalue.xlsx',Fsumvalue)

I will try with a smaller value of TolCon.

Thank you again

Answer 1

[John D'Errico]

Standard optimizations that allow bounds treat the bounds as a hard line, "Beyond that line, THERE BE DRAGONS!" But they do not allow you to have more dragons (but smaller ones) just inside the line.

Given a large parameter set, the odds are quite good that a solution will have at least some of the unknowns at or near the boundary. That does not mean another solution does not exist fully inside. Problems with this many unknowns are quite likely to have multiple solutions. This suggests one solution is to simply come up with better starting values. Your answer might be only as good as your starting point. That is not at all uncommon.

It is also true that there exists NO true solution inside your chosen bounds for the parameter space. Yes, stuff happens.

Next, while there is no way to simply tell a tool like lsqnonlin to stay as far away as possible from the bounds, you can do so with a tool like fmincon, if you are careful and you are willing to go down this road.

Formulate an objective that is a sum of squares of residuals, to be called by fmincon. But then we have two options.

1. First is a penalty function approach. Add to the base objective a penalty, based on the distance your parameter set lies from each boundary. You will still have the hard bounds set, but also penalize the objective whenever it gets near a boundary. You will need to decide the strength of that penalty in advance. That reflects how much you hate being near the boundary.

2. The alternative is to view the problem from a Bayesian point of view. (A friend of mine used to refer to himself as a reluctant Bayesian.) Here, you might establish a prior distribution on the parameter set, centered in the middle of your parameter space. Use that prior to tweak the objective function, based on the distance away from the center of the parameter space. Here you will have two disparate sources of information, so a multi-criteria optimization. You choose the relative importance of each piece of information to form a whole.

In either case, you need to understand that your solution will not be a true minimizer of the sum of squares, but a solution adjusted to be one that you "like". Hey, I got an answer, but I don't like it. So I'll just pick something that makes me happy. After all, it is your problem to solve, so as long as you are happy, that is all that matters. How to convince a thesis committee, or the referees on the paper you write about this, your boss, whatever - that is your next problem to solve. :)