Solving Over-determined Non-linear equations.

Hi,

I have a set of 8 equations and 6 unknowns to solve for. All the 8 eqns are second order (quadratic) with each eqn having 4 unknowns.

I am listing two of them for eg. 4x^2 + 5y^2 -6x + 7y - 8xy + 6xz - 3yz - 5z + 2w + 9 = 0 ; 4u^2 + 5w^2 -5u + 8w - 9uw + 14uv - 13wv - 4v + 3x + 17 = 0 .. and so on.. with a total of 8 eqns.

The actual coefficients are very small numbers like 8.5e-4 and so on.. i have used integers here in the example to make it more comprehensible. The unknowns are x , y, z, u , v , w . Now how do I solve this system of eqns? I tried using fsolve and lsqnonlin.. Both don't seem to get me even closer to the answers.

I would really appreciate some help in this.

Thanks in advance, Venkat

[the cyclist]

It would be very helpful for you to post the actual code you used to conclude that fsolve or lsqnonlin are not working well.

[bym]

your example equations are linear in the coefficients!??

[Venkatesa]

The coefficients are obviously linear since they are numbers. I have posted one such actual equation below.

[bym]

I meant the system is linear, not non-linear as in the title. Please post & format the equations

Answer 1

[bym]

doc mldivide

[Venkatesa]

Thanks proecsm.. I just saw the doc for this.. I will try using it.

Answer 2

[John Tillinghast]

It would help to see the code. In general, if you are doing complicated operations with small coefficients, you might want to rescale some of the variables so the coefficients are a bit bigger.

Presumably you are trying to minimize some "discrepancy" function (penalty function) for the equations. For example, if you have equations like

a1*x + b1*y = c1 a2*x + b2*y = c2 ...

your penalty function might be something like

sum_i( (a_i * x + b_i * y - c_i)^2 ) [not matlab code, just my notation]

But it's possible that some of the terms are more important than others, so you might really need to minimize

(a1*x + b1*y-c1)^2 + 10 * (a2*x + b2*y-c2)^2 + 100 * (a3*x + b3*y-c3)^2

i.e. a weighted sum instead of the unweighted sum.

But please post the code.

[Venkatesa]

@John - I didn't quite understand your comment. Do you mean to say, I should scale the coefficients from being in the 10^-5 order to 10^1 order? and also I didn't get the last part. Could you please explain it to me?

[Jobayer Rahman]

%fx=3x^3-5x^2+5x-7

%fdx=9x^2-10x+3

x(1);

kk(1)=1;

for k=1:100;

fx=3x(k)^3-5x(k)^2+3(k)-7;

fdx=9x(k)^2-10x(k)+3;

h=-fx/fdx;

x(k+1)=x(k)+h;

kk(k+1)=k+1;

if abx(h)<.0000001

break,end

end

disp([kk1'x'])

is it correct program...can you please check with your matlab program. my matlab getting some problem so that i ask you to just check and run for me. inform me the result...

[Walter Roberson]

fx=3x(k)^3-5x(k)^2+3(k)-7 is not valid MATLAB syntax. MATLAB requires that you use * or .* to indicate multiplication

Answer 3

[Venkatesa]

Hi ,

First of all thanks for your suggestions. I am not using any code to solve the equations. I have a user defined function which has all the equations. The contents of the functions are the following.

function F = myfun(X)
F = [ ((8.856e-6*(X(1,1)^2)) + (8.56e-6*(X(2,1)^2)) + 3.32e-5 - (2.28e-5*X(1,1)) + (3.73e-4*X(2,1)) - (1.75e-5*X(1,1)*X(2,1)) + (1.16e-4*X(3,1)*X(1,1)) - (1.14e-4*X(2,1)*X(3,1)) - (1.42e-4*X(3,1)) - (1.09e-4*X(4,1))); ......];

I didn't copy paste the whole function since it will be cumbersome to read here. I am not sure If I can attach a file here?

This is one such equation and I have 8 such equation defined one after another in this file and then I tried using fsolve and lsqnonlin to solve these.

I use the following set of options

options=optimset('TolFun' , 1e-18,'Display','iter', 'TolX' , 1e-15, 'MaxFunEvals' , 5000, 'MaxIter' , 5000);

[x2,fval] = fsolve(@myfun,A,options)

Using fsolve i get the following.

No solution found.

fsolve stopped because the problem appears to be locally singular.

<stopping criteria details>

But it gives me some values for the x2 are not in the range I expect them to be. The thing is.. I already have the reference set of values for x, y, z , w, u , v and whatever answer I get from fsolve should be atleast close to them if not the same. I hope I am clear.

With regard to lsqnonlin.

[x2 resnorm residual exitflag output ] = lsqnonlin(@myfun , A ,0.45, 0.6,options )

Local minimum possible.

lsqnonlin stopped because the final change in the sum of squares relative to 
its initial value is less than the selected value of the function tolerance.

<stopping criteria details>

If I increase the TolFun or TolX it still calculates the values of x2 but then after one point it doesn't do any minimization at all.

Thanks a lot for all your help,

I would be really grateful to all of you if I can find a way to solve this.

-Venkat

[Venkatesa]

Hi friends...

Anyone any suggestions on this?

Thanks,

Venkat

[bym]

can you post all the equations?

Answer 4

[Venkatesa]

@proecsm -

I am not how readable this would be. Anyways here are the equations.

((8.856e-6*(X(1,1)^2)) + (8.56e-6*(X(2,1)^2)) + 3.32e-5 - (2.28e-5*X(1,1)) + (3.73e-4*X(2,1)) - (1.75e-5*X(1,1)*X(2,1)) + (1.16e-4*X(3,1)*X(1,1)) - (1.14e-4*X(2,1)*X(3,1)) - (1.42e-4*X(3,1)) - (1.09e-4*X(4,1)))

((8.856e-6*(X(1,1)^2)) + (8.56e-6*(X(2,1)^2)) + 3.34e-5 -(5.9e-5*X(1,1)) + (2.30e-4*X(2,1)) -  (1.75e-5*X(1,1)*X(2,1)) + (1.16e-4*X(3,1)*X(1,1)) - (1.14e-4*X(2,1)*X(3,1)) - (1.36e-4*X(3,1)) - (1.09-4*X(4,1)))

((8.856e-6*(X(1,1)^2)) + (8.56e-6*(X(2,1)^2)) - 3.12e-5 - (1.40e-4*X(1,1)) - (1.75e-5*X(1,1)*X(2,1)) + (1.16e-4*X(3,1)*X(1,1)) - (1.14e-4*X(2,1)*X(3,1)) + (1.87e-4*X(2,1)) - (1.47e-4*X(3,1)) - (1.6027e-6*X(5,1)))

((8.856e-6*(X(1,1)^2)) + (8.56e-6*(X(2,1)^2)) - (1.75e-5*X(1,1)*X(2,1)) + (1.16e-4*X(3,1)*X(1,1)) - (1.14e-4*X(2,1)*X(3,1))  - 3.63e-5 - (1.74e-4*X(1,1)) + (2.22e-4*X(2,1)) - (1.46e-4*X(3,1)) - (1.6027e-6*X(5,1)))

((8.856e-6*(X(5,1)^2)) + (8.56e-6*(X(4,1)^2)) + 3.25e-5 - (2.9e-5*X(5,1)) + (1.96e-4*X(4,1)) - (1.75e-5*X(4,1)*X(5,1)) + (1.16e-4*X(6,1)*X(5,1)) - (1.14e-4*X(6,1)*X(4,1)) - (1.09e-4*X(2,1)) - (1.43e-4*X(6,1)))

((8.856e-6*(X(5,1)^2)) + (8.56e-6*(X(4,1)^2))  - (1.75e-5*X(4,1)*X(5,1)) + (1.16e-4*X(6,1)*X(5,1)) - (1.14e-4*X(6,1)*X(4,1))  - (1.09e-4*X(2,1)) - (5.73e-5*X(5,1)) + (2.30e-4*X(4,1)) + 2.75e-5  - (1.390e-4*X(6,1)))

((8.856e-6*(X(5,1)^2)) + (8.56e-6*(X(4,1)^2))  - (1.75e-5*X(4,1)*X(5,1)) + (1.16e-4*X(6,1)*X(5,1)) - (1.14e-4*X(6,1)*X(4,1))  + (1.88e-4*X(4,1)) - (1.41e-4*X(5,1)) - 4.30e-5 - (1.6027e-6*X(1,1)) + (1.82e-5*X(6,1)))

((8.856e-6*(X(5,1)^2)) + (8.56e-6*(X(4,1)^2))  - (1.75e-5*X(4,1)*X(5,1)) + (1.16e-4*X(6,1)*X(5,1)) - (1.14e-4*X(6,1)*X(4,1)) - 4.47e-5 + (2.22e-4*X(4,1)) - (1.75e-4*X(5,1)) - (1.51e-4*X(6,1)) - (1.6027e-6*X(1,1)))

These are the 8 equations I have and X(1,1) to X(6,1) denote the 6 unknowns x , y , z, u , v, w.

Thanks in advance for your help,

Venkat

[Venkatesa]

To add , for all equations the RHS is 0.

[bym]

thanks, I don't have time to work on it tonight, but can tomorrow. I'll see what I can do...unless @Walter Robinson et.al. beats me to it

Answer 5

[bym]

I will refer to your equations as eq1 through eq8. You'll notice that the higher order terms have the same coefficients (this is a clue). For example, subtracting eq3 from eq4 you are left with an expression in just x,y,z.

So, I got a system of 6 equations and unknowns by doing the following :

eq2-eq1 ;eq3-eq2;eq4-eq3;eq8-eq7;eq7-eq6;eq6-eq5

that gave me a coefficient matrix

A =

       6e-006    -0.000143   -3.62e-005            0            0       4.0001
    -1.1e-005    -4.3e-005    -8.1e-005            0 -1.6027e-006           -4
       1e-006     3.5e-005    -3.4e-005            0            0            0
            0            0            0   -0.0001692    -3.4e-005     3.4e-005
            0     0.000109 -1.6027e-006    0.0001572   -8.37e-005    -4.2e-005
            0            0            0       4e-006   -2.83e-005     3.4e-005

and a constant vector

b =

         1.09
      -1.0899
     5.1e-006
     1.7e-006
    7.05e-005
       5e-006

which can be solved using mldivide

c =A\b
c =

      -36.231
      0.81734
     -0.37424
     0.033565
      0.15554
      0.27257

This problem is very sensitive to round off errors, if you have the symbolic tool box you can solve it more accurately

[Venkatesa]

Hi Proecsm,

I tried doing this but this is giving me even weird results. I know my answers should be in the range of 0.45 to 0.65.

Before creating these equations, I had done some curve fitting to find co-efficients to given standard eqns and then used them to arrive at these 8 eqns.

Moreover, when we subtract, wouldn't we have just three terms and the rest 0 in each equation. Like for eg. In the A matrix you have mentioned above, the second column has more than 3 non-zero values. I am confused as to how this can be possible. The equations are such that when we subtract only 3 terms excluding the constant remain.

If there any other way to keep the quadratic and solve it?

Thanks a lot for your valuable help.

-Venkat

[bym]

maybe if I understood what you are trying to accomplish a little better it would help. I gather you are fitting 8 sets of data to prescribed equations in x,y,z,u,v,w.

What is your objective in trying to solve the 8 equations, to find the coefficients that best describe _all_ of the data?

[Venkatesa]

Hi Proecsm,

My objective is to solve the 8 equations to arrive at values to the x,y,z,u,v,w. I have equations of the kind

I = An * exp ((v1 - ( g*x) - (t*v2))/d)

where I, v1, v2 are values that I know and I used them to find the coefficients g,t,d,An. I have a few more equations like this both exp and linear ones. Then I do some measurements (8 to be precise) and then use those values in these equations and combine them to form the final set of 8 equations.

Now I solve the 8 equations to get the values of the variables. My aim is to find coefficients with minimum errors ( which I already did) and then solve the 8 non linear equations. ( here is where I am stuck).

Am I clear now? Please let me know if you didn't understand any of it.

Thanks,

Venkat

[Venkatesa]

Hi Proecsm,,

Did you get a chance to look at this?

Thanks,

Venkat

[bym]

@Venkat - I have not had a chance to revisit this. It sounds as if you are doing some electrical engineering problem. If you can explain what you are trying to from that aspect, maybe others on the forum could help (I am a mechanical engineer by training)

Answer 6

[Walter Roberson]

I followed proecsm's lead but with slightly different arrangements of the equations, using eq1-eq2, eq1-eq3, eq1-eq4, eq5-eq6, eq5-eq7, eq5-eq8

Then substituting the names x, y, z, u, v, w in for X(1,1), X(2,1) and so on, and converting the result to a set, I was able to use Maple:

LinearAlgebra[LeastSquares](
{.50e-5+.283e-4*v-.34e-4*u-.40e-5*w = 0,
1.0899998+.362e-4*x+.143e-3*y-.6e-5*z-4.000109*u = 0,
.644e-4+.1172e-3*x+.186e-3*y+.5e-5*z-.109e-3*u+.16027e-5*v = 0,
.695e-4+.1512e-3*x+.151e-3*y+.4e-5*z-.109e-3*u+.16027e-5*v = 0,
.755e-4+.112e-3*v+.8e-5*u-.109e-3*y-.1612e-3*w+.16027e-5*x = 0,
.772e-4+.146e-3*v-.26e-4*u-.109e-3*y+.8e-5*w+.16027e-5*x = 0},
{x,y,z,u,v,w})

This yielded the result,

{u = .27256965857615293077411639983502007553428451870328,
 v = .15277278781877226253617051867622720938483268112186,
 w = .14025375920513845863417021036636864356272827882778e-1,
 x = -.24280430676962935958328829284572825211877450016313,
 y = .84533097654555229286657861791163208120617978613903,
 z = -32.741930609261728476162053583661883414254625520422}

On the other hand, this is the same as is exactly the same result as you get if you submit that list of 6 equations to simple solve() -- with there only being 6 equations in 6 unknowns, there is no invocation of least-squares solution.

If one substitutes the above results back in to the original 8 equations, the results are not great -- 4 nearly identical residues on the order of +0.01, and 4 nearly identical residues on the order of -0.00001 .

I would thus say that reducing the 8 equations to 6 is not going to work properly for a least-squared solution.

Unfortunately I do not have a recommendation at present; I thought I knew how to deal with x^2 and x*y and so on, but about a week ago, I managed to prove to myself that my technique was fundamentally flawed,, and I have not yet had Time To Go Back To The Old Drawing Board.

[Venkatesa]

Hi Walter,

Thanks for the valuable help. I figured out that the curve fitting I had done earlier has error of 20% and when I use those primary equations to derive these final 8 eqns.. the error shoots up like in the order of 1e2 to 1e3. I need to find a way to fit those curves with as minimal error as possible.

My eqn is

I = K1 ( v1 - (B*v2) - v3) ( v4 -v2) where K1 and B are my coefficients to be found. Here is what I did.

I rearranged the eqn in the following form.

I = K11 ( v1v4 - v1v2 - v3v4 + v2v3 ) + K22 ( v2*v2 - v2v4)

Where K11 - K1 and K22 = K1*B .

Now I assumed these K11 and K22 as my variables. I have simulated values of I from a spice simulator and these values are for various combinations of v2,v3,v4 . v1 is always fixed. Say I got 1000 pts from spice simulations.

Now I used an input matrix A which is a 1000x2 matrix..

where the first column is the coefficient values of K11 and the second is K22. I also have a matrix b which is the I values from the spice simulations.

Now I use y = (A'*A\A'*b) which gives me 2 values for y which correspond to K11 and K22. This is how I found the coeffcients. I am seeing an error of 20% in this fitting which is not acceptable for me. Any suggestions how I can better fit this curve?

Thanks and please let me know if I am not clear anywhere.

Thanks again,

Venkat

[Venkatesa]

Hi,

ANyone any suggestions??

-Venkat