I have shifted from lsqnoneg to lsqlin because lsqnonneg does not support additional constraint is it right or wrong choice?

Hi , i am working on laser absorption spectroscopy tomography and i am trying to find my X for AX = b form as b is my data and A is my coefficent matrix and i am using lsqnoeng as my solver , some how when i have to appy the constraint during the reconstruction of X ,then lsqnonneg does not support it, because of it i have shift to lsqlin and setting its lower bound zero or non zero , i am considering it will work like lsqnonneg solver , now i can easily apply constraint into it, but the problem is i am confused whether i did it rigth or wrong as for the better result in lsqlin i have to scale my data b with it maximum value other wise it give very bad results , or i should implement the lsqnoeng and then apply constraint into it , please guide me

[Torsten]

We need the code and the data for a problem in which scaling with "lsqlin" is necessary whereas you get good results with the original data with "lsqnonneg".

[Bruno Luong]

Not sure how you can use lsqnonneg with constraints A*x = b

It is always good for user to scale correctly constraints and other stuffs in optimization, regardless the algorithm.

From my experience, lsqlin is superior (opt toolbox required) to lsqnonneg (standard MATLAB function)

[sanjay]

Yes you are right as i was using default function of lsqnonneg in MATLAB and i was also not sure the how can i use constraints in lsqnonneg so that is why i am using lsqlin default function in MATLAB( dont have idea about the opt toolbox ) as i am not expert user of MATLAB , but i am just saying when i scale the data (dividing the maximum vlaue of data with whole data) then i apply lsqlin, it gives me result otherwise it wont, i dont whether i am doing right or wrong

[Bruno Luong]

If you have linear constrained the you should use lsqlin (assuming the pb is linear), period you cannot use lsqnonneg.

Yes you need to take a minimul care about the scaling. You need scale so that

1. The condition number of A is small, that might require scaling each row of A, and/or each column of A. If you scale row, you must adjust b. If you scale A, you must adjust x.
2. If b and x get very unbalanced due to scaling, you could shift the problem/constraints so that they become balanced. If A has some physical units, I trecommend to scale them soo that they are unitless before put them in the matrix. Don't combine Farad with pico meter in the same equation, otherwise you'lll be in trouble.

[Torsten]

@Bruno Luong

I think that the OP's A is meant to be C in the documentation of "lsqlin" - the regression matrix, not the constraint matrix. From an earlier question, OP wants to set lower and upper bounds on X which is not possible with "lsqnonneg". That's why she/he switched to "lsqlin".

[Bruno Luong]

OK Torsen/ But then I don't understand the question. "Changing the scalling the data b" meaning different least square objective function (its weighting). least square pb with Tichkonov regularization should be well posed and the unique solution should be identical for different solver ( but depends only on the weighting). This does not seem to be the case with this question.

Next question: I do not know if OP changes accordingly the regularisation in which case (lsqnonneg vs lsqlin).?

She/he needs to describe correctly the math rather with those verbose imprecise wording.

[Torsten]

Naivly I thought @sanjay solved

A/max(abs(b)) * X = b/max(abs(b))

instead of

A*X = b

with "lsqlin" and got better results.

[Bruno Luong]

That would be really odd. I cannot believe it.

[Matt J]

That would be really odd. I cannot believe it.

Why? The following example shows that a scale factor can make a big difference:

C=rand(2); d=[1;2];

args={[],[],[1,1],1,[0;0],[], [], optimoptions(@lsqlin,'Display','none')};

x0=lsqlin(C,d, args{:})

x0 = 2×1

1.0000 0.0000

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

x1=lsqlin(C*1e8,d*1e8,args{:})

x1 = 2×1

2.5058 1.2267

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

[Bruno Luong]

@Matt J Your example is with linear equality constraints I though Torsen understood there is not and also OP can solve with lsqnonneg as well. The solution of non negative-only-constraints is linear under positive scalar scalling of b, eg 1/max(b)) (b being data vector to fit, not the constraint RHS).

I'm not going to discuss since everyone seem to understand the OP question differently. It is not clear at all what the question is.

[sanjay]

Okay let me explain you mathematically I am saying my data is b and A is my coefficient matrix , yes my problem is linear Ax = b b = b/max(b) X = lsqlin(A,b) X = max(b)*X Here I'm trying to say I have scaled only b the data not coefficient matrix

[Torsten]

But then you solve a completely different problem:

A = [2 3 ; 4 5];
b = [3;20];
A\b

ans = 2×1

22.5000 -14.0000

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

bscaled = b/max(b);
max(b)*A\bscaled

ans = 2×1

0.0563 -0.0350

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

[Bruno Luong]

Parenthesis in MATLAB expression to impose the order of operaions

Beside that scaling b does not change the solution (without linear constraint case) as your description

A = [2 3 ; 4 5];
b = A*[6;7]; % we want b corresponds to non negative unknown X

A\b

ans = 2×1

6 7

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

bscale = b/max(b);
max(b)*A\bscale % wrong

ans = 2×1

0.0017 0.0020

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

max(b)*(A\bscale)

ans = 2×1

6.0000 7.0000

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

lb = [0;0];
Xscaled = lsqlin(A, bscale, [], [], [], [], lb, []);

Minimum found that satisfies the constraints.

Optimization completed because the objective function is non-decreasing in 
feasible directions, to within the value of the optimality tolerance,
and constraints are satisfied to within the value of the constraint tolerance.

X = max(b)*Xscaled

X = 2×1

6.0000 7.0000

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

Xscaled = lsqnonneg(A, bscale);
X = max(b)*Xscaled

X = 2×1

6.0000 7.0000

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

[Torsten]

But then you solve a completely different problem

Ok, I take it back. :-)

[Bruno Luong]

I didn't see anything (It's so rare that Torsen makes a little mistake. :))

Answer 1

[Matt J]

If your lsqlin can solve your problem, then there is usually never any reason to use lsqnonneg. lsqnonneg is for nonlinear least squares problems, while your problem is linear.

As Torsten said, in the absence of your code and input, we can only make guesses about the other difficulties . However, normalizing your data is always a good thing because the default optimoptions are chosen with the assumption that you don't have crazy large (or small) input data.

[sanjay]

The problem is code is of insights group, can not openly give it to online platform, if you need more details i can give you, secondly results are fine but the lsqnoeng results and the lsqlin results little bit deviate with respect to error but not much , i was concerned much about scaling , i thought lsqnonneg gives is not asking for scaling where lsqlin do so

[Torsten]

@Matt J

lsqnonneg is for nonlinear least squares problems, while your problem is linear.

I think you mean "lsqnonlin" here.

@sanjay

Do problems that work fine with "lsqnonneg" have to be scaled to get the same (good) results using "lsqlin" - yes or no ?

[sanjay]

yes, the results which i am getting in lsqnoneg , if i have to get those results then i am scaling the data in lsqlin then i get the results like lsqnonneg

[sanjay]

no not lsqnononlin , its lsqlin

[sanjay]

@Matt J i am confused let me clear to you i was using lsqnonneg (least square nonegative) to reconstruct X from my A and b, then i wanted to apply some constraints during each iteration which was not possible , then i used lsqlin not lsqnonlin, also clear me should i used lsqlin or lsqnonlin,

[Matt J]

My point still remains. It is a good idea to normalize your matrix data, both the ones used by the cost function and the ones used by the constraints.

[sanjay]

yes i am also noramlizing my constraints with the same max value , i am just asking is it a right approach to do noramalize yoyr data and when you get the X then rescaling it , just want to confrim

[Bruno Luong]

@sanjay Please give a small example but representative MATLAB code to illustrate, still not sure if you have a linear constraint or not in your problem.

[Bruno Luong]

With constraints you have to scale both b and beq by the same scalr scaling factor:

If the scaling factors are different or if you only scale one of those then you solve different problem.

A = [2 3 ; 4 5];
X = [6;7];
b = A*X; % we want b corresponds to non negative unknown X
Aeq = [8 9];
beq = Aeq*X;

A\b

ans = 2×1

6 7

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

bscale = b/max(b);
beqbscale = beq/max(b);
lb = [0;0];
Xscaled = lsqlin(A, bscale, [], [], Aeq, beqbscale, lb, []);

Minimum found that satisfies the constraints.

Optimization completed because the objective function is non-decreasing in 
feasible directions, to within the value of the optimality tolerance,
and constraints are satisfied to within the value of the constraint tolerance.

X = max(b)*Xscaled

X = 2×1

6.0000 7.0000

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>