Why I cannot fit a function with more than five variables？

Why I cannot fit a function with more than five variables？It gives the errors below， who can help me？

[David Goodmanson]

Hello zw,

given the heirerarchy of arithmetic operations in Matlab, this expression appears to be reasonable as an algebraic statement, not as the previous comment here indicated. Its applicability as a suitable model is a different question.

[the cyclist]

Posting code is far more useful than posting an image of code, because then we could copy & paste it into a workspace to test it.

It would also be helpful if you uploaded a MAT file with your workspace variables, so we could run your exact code and data.

[huazai2020]

This is the .m and data file, please help me find it ,thank you.

[Image Analyst]

I agree with David. Given your data, I can't see how or why your model even makes sense. Where did you come up with that model??? I don't believe any real world situation would ever need such a crazy model.

[Rik]

Comment posted as flagg by zhihua wan:

But I want to do like this， I see other people do like this， how can I solve it, does there have any other functions that can do this model?

[Alex Sha]

refer to the results below:

Root of Mean Square Error (RMSE): 0.0111861472021732
Sum of Squared Residual: 0.00212720811688768
Correlation Coef. (R): 0.928730260655716
R-Square: 0.862539897057634

Parameter	Best Estimate
----------	-------------
b1	2.17150492153399E-12
b2	1.39716451311507
b3	-2.74854828910142
b4	-3.96812040839357
b5	4.23887915310233
b6	-0.0779426686949187

[huazai2020]

Could you send me your codes for this?Thank you.

[huazai2020]

Hello, I have submitted to the forum, please have a look

[Alex Sha]

Hi, Wan, try a software named "1stOpt", great for curve fitting:

Function y=b1*x1^b2 *x2^b3*x3^b4*x4^b5*x5^b6;
Data;
1000	3	0.00184	0.212	0.01	0.2308
1100	3.1	0.00368	0.42401	0.02	0.28698
1200	3.2	0.00245	0.28267	0.03	0.25763
1300	3.3	0.00123	0.14134	0.04	0.20281
1400	3.4	0.00147	0.1696	0.05	0.21634
1500	3.5	0.00137	0.1575	0.06	0.21959
1600	3.6	0.00153	0.17667	0.07	0.22708
1700	3.7	0.00258	0.29777	0.08	0.23734
1800	3.8	0.0022	0.25327	0.09	0.23406
1900	3.9	0.00273	0.315	0.1	0.28892
2000	4	0.00204	0.23556	0.11	0.25137
2100	4.1	9.11E-04	0.105	0.12	0.1937
2200	4.2	0.00123	0.14133	0.13	0.2096
2300	4.3	0.00517	0.59555	0.14	0.29073
2400	4.4	0.00293	0.3377	0.15	0.2656
2500	4.5	0.00172	0.19852	0.16	0.20503
2600	4.6	0.00176	0.20262	0.17	0.22096

[huazai2020]

Hi,Alex, What is the R-squared after you fit？Could you share your software for me if it is free,how can I learn it，thank you.

[David Goodmanson]

Hi zw,

It's time to look at all of the data. Past time, actually.

The following plot shows the data in log scale. As you can see, y and x4 (meaning x(:,4), similarly for the others) are almost the same. x3 is offset from x4 by an almost-constant amount on log scale, and a quick check shows that those two columns differ by a factor that is very close to being constant. So whatever effect x4 has on the fit, x3 is hardly capable of changing it. This one instance of what Image Analyst points out as highly correlated variables.

Comparing x4 to y, there are still some oscillatory kind of residuals. But since the rest of the columns of x are quite smooth, they will not be able to remove those residuals. Consequently the b exponentials for x1,x2 and x5 are pretty meaningless. Probably, when taken together as blob, they provide a slight level and slope correction to x4. But that's it.

[Image Analyst]

I never see people do that really. Are you sure you're not thinking of a multiple linear regression where the x terms are summed instead of multiplied by each other? Is there some real world physical system you're modeling where 5 variables are mutplied by each other? If so, what is it? What do your x represent? And are you sure you're not thinking of a sum instead of a 5-way product?

[huazai2020]

It is from other people do in their article， I have a question， if I do not know the equation before fitting, how can I fit it?

[David Goodmanson]

Hi h2020

has your nom de plume changed?

Flags are intended to mark messages that in one way or another do not conform to the guidelines. They are not intended for follow up questions, so it woud be better if you deleted the flag and posted a comment.

Here is the code I used, after getting x and y into matrix form with x being 17x5 and y being 17x1.

n = size(x,1);
semilogy(1:n,x,1:n,y,'-o')
legend({'x1','x2','x3','x4','x5','y'})
grid on

[David Goodmanson]

Hi h2020,

While I would not go so far as to say this can't/shouldn't be modeled, I think you should take a close look at the variables involved. Could you comment on what is the physical significance of the five columns of x, and how are they expected to relate to y? If the basic mechanism is known, is there a model that might naturally apply?

As far as x is concerned, x3 and x4 are almost the same, being almost exactly proportional to each other. So they are not really independent variables. y most closely resembles x3 in the sense that there appears to be some correlation in their variations. In terms of their shape and not their size, x1,x2 resemble each other. x5 is similar exept for a downturn on the left that does not appear to be of much help for a potential fit.

On the basis of what is going on physically, one of x1,x2,x5 might be expected to have a closer relationship to y than the others. But no matter what the mathematical form of the model is, fitting all three will probably result in one or two extra parameters that do not mean very much (as AI has mentioned).

[huazai2020]

This variable are just random, which is to verify the ability of the fitting model in matlab.I see there are many models such as nlinfit，fittype and so on in the matlab， what is the difference, which is better for the present more than 4 variables multiply model?

[huazai2020]

Could you tell me how to cancel the flag?

[Walter Roberson]

y=b1*x1^b2 *x2^b3*x3^b4*x4^b5*x5^b6

Take the log of both sides assuming real and non-negative x* :

log(y) = log(b1) + b2*log(x1) + b3*log(x2) + b4*log(x3) + b5*log(x4) + b6*log(x5)

So this is a linear fit in log(x*) and log(y). You can construct

nx = size(x,1);
B = [ones(nx,1), log(x)] \ log(y(:));
B(1) = exp(B(1));    %it is the only log term

B is now the closest fit in log space, suitable for using for initial coordinates for fitting in linear space.

[huazai2020]

what does this （B = [ones(nx,1), log(x)] \ log(y(:));）mean？

[Walter Roberson]

Consider the model

log(y) = log(b1) + b2*log(x1) + b3*log(x2) + b4*log(x3) + b5*log(x4) + b6*log(x5)

but suppose you have several x* and y values. Then you can create a series of equations

log(y(1)) = log(b1)*1 + b2*log(x1(1)) + b3*log(x2(1)) + b4*log(x3(1)) + b5*log(x4(1)) + b6*log(x5(1))
log(y(2)) = log(b1)*1 + b2*log(x1(2)) + b3*log(x2(2)) + b4*log(x3(2)) + b5*log(x4(2)) + b6*log(x5(2))
log(y(3)) = log(b1)*1 + b2*log(x1(3)) + b3*log(x2(3)) + b4*log(x3(3)) + b5*log(x4(3)) + b6*log(x5(3))
...

Now arrange those in matrix form:

log(y(:)) = [1, log(x1(1)), log(x2(1)), log(x3(1)), log(x4(1)), log(x5(1)), log(x6(1));
             1, log(x1(2)), log(x2(2)), log(x3(2)), log(x4(2)), log(x5(2)), log(x6(2));
             1, log(x1(3)), log(x2(3)), log(x3(3)), log(x4(3)), log(x5(3)), log(x6(3));
             ...
             ] * [log(b1); b2; b3; b4; b6; b6]

where * is algebraic matrix multiplication.

This can then be written more compactly as

log(y(:)) = [Column of 1s, log(x1(:)), log(x2(:)), log(x3(:)), log(x4(:)), log(x5(:)), log(x6(:))] ...
             * [log(b1); b2; b3; b4; b6; b6]
         

and since your x1 = x(:,1) and x2 = x(:,2) and so on, the log(x1(:)), log(x2(:)) and so on can be written more compactly as log(x), so

log(y(:)) = [Column of 1's, log(x)] * [log(b1); b2; b3; b4; b6; b6]

This is a system of linear equations. If we say

b = log(y(:))
A = [Column of 1's, log(xdata)]
X = [log(b1); b2; b3; b4; b6; b6]

then we get the familiar A*X = b .

If A were square (that is, you had 7 samples) then mathematically you would multiply the left sides by inv(A), getting

inv(A) * A * X = inv(A) * b

and inv(A) * A would be the identity matrix, and inv(A) * b could be calculated as all of those values are known, so the vector of unknowns X = inv(A) * b

You have more than 7 samples, so you do not have a square system, so you cannot use inv(), but you can do the equivalent of

pinv(A) * A * X = pinv(A) * b

to get X = pinv(A) * b as a solution that attempts to minimize error.

The MATLAB operator A\b roughly calculates pinv(A) * b, but does so using a different method of minimizing error; A\b is a least-squared error method of making that calculation.

ones(nx,1) is the "column of 1s" mentioned earlier.

Now, this all would be a least-squared calculation in log space, and would be the best fit you could get in log space for that model. But you probably want least-squared calculation in linear space, so take the result as initial values to feed into a linear least squared fit routine.

Answer 1

[the cyclist]

I think the primary problem is that some of your variables are highly correlated with each other, and therefore (a) add very little information to the model, and (b) will contribute to over-fitting because of the extra parameters. The following works:

data = xlsread('data_youhua.xlsx','Sheet1');

X = data(:,[1 3]);
y =data(:,6);

modelfun = @(b,x) b(1).*x(:,1).^b(2) .*x(:,2).^b(3);
beta0 = [-1 0.1 0.2];

mdl = fitnlm(X,y,modelfun,beta0);
predicted_y = predict(mdl,X);
figure
hold on
hd = plot(y,'.');
hp = plot(predicted_y,'.');
set([hd hp],'MarkerSize',24)
for ny = 1:numel(y)
    hc = line([ny ny],[y(ny) predicted_y(ny)]);
    set(hc,'Color','black')
end

legend([hd hp],{'data','prediction'})
print('-dpng','-r600','test.png')

and results in the following fit

Note that I did not draw the fit as a continuous line. The reason is that the x-axis here is not a continuous variable. It is just the ordinal count of your data points. There are almost certainly better ways to plot the comparison of the data and the fit, but this is at least not incorrect.

[huazai2020]

Thank you. I have two questions:1.As above Alex said I can use the 1stOpt to fit it ,but you see I cannot , which is right? 2. As you can see the down error, what is the reason?

[the cyclist]

Sorry, use this code instead:

data = xlsread('data_youhua.xlsx','Sheet1');

X = data(:,[1 3]);
y =data(:,6);

modelfun = @(b,x) b(1).*x(:,1).^b(2) .*x(:,2).^b(3);
beta0 = [-1 0.1 0.2];

% mdl = fitnlm(X,y,modelfun,beta0);
% predicted_y = predict(mdl,X);

fit_coefficients = nlinfit(X,y,modelfun,beta0);
predicted_y = modelfun(fit_coefficients,X);

figure
hold on
hd = plot(y,'.');
hp = plot(predicted_y,'.');
set([hd hp],'MarkerSize',24)
for ny = 1:numel(y)
    hc = line([ny ny],[y(ny) predicted_y(ny)]);
    set(hc,'Color','black')
end

legend([hd hp],{'data','prediction'})
print('-dpng','-r600','test.png')

Notice that here I used nlinfit (as you did in your original code), instead of fitnlm.

You must have the Statistics Toolbox, since you are able to use nlinfit, so I assumed you would also have fitnlm. Do you have a very old version of MATLAB? (fitnlm was introduced in R2013b.)

[huazai2020]

My edition is 2012b, can you share your toolbox for me to update it ? I can run it using your second code, but I have a question ,you only fit the data in x(:,1).^b(2) .*x(:,2), which is not what I want, can I use the 1stOpt to fit it, it can fit it, but I do not whether it is right.

[the cyclist]

No, I cannot share a toolbox. That is against the license agreement. You need an up-to-date license of your own.

I can spend some time seeing if nlinfit can work for your model ... but why?? Your model is definitely worse. Although I don't like plotting it this way, take a look at the line plot:

In terms of how well it is fitting the data, it is nearly indistinguishable from the 6-parameter model. But because it uses fewer parameters, it is much less likely to be over-fit. I will be more motivated to think about your more complex model if I understand any reason why it should be used. Otherwise, I feel that I simply giving you bad statistical modeling advice.

[the cyclist]

Well, I got curious, so I'll just give you the bad advice. Maybe an evil professor is requiring you to fit a particular model.

If you use

beta0 = [1 1 1 1 1 1]; 

instead of your starting values, then nlinfit will return coefficients. You will get a warning that it failed to converge, though. This is yet further confirmation that it is a bad model for the data. Be a hero -- do it the right way.

[huazai2020]

This is model is made by myself, I have a question , how can we find the equation type before fitting?

[the cyclist]

The best way to choose the equation type to fit is to have an understanding of the underlying process that generated the data. If there is a theoretical understanding of what that process is, then a particular equation will be appropriate. For an extremely simple example, if you are measuring the force on a spring, a simple one-parameter theoretical equation might just be

F = k * x

and then you could fit for k.

Most real-life examples aren't that easy, but the principle is the same.

[huazai2020]

But it is not simple equation as you said， what can I do? Do there have any black box such as machine learning that can predict it ?

[the cyclist]

I'm not sure what you are trying to achieve here. With the two-parameter equation I used (based on your original equation), you seem to get a pretty good fit. How did you decide on your original 6-parameter equation? Was it pure guesswork and trial & error?

A machine-learning approach is probably not going to be very helpful for a problem with only 17 data points.

Without knowing a lot more about the actual meaning behind your data, it's not really possible to advise. Understanding the data themselves is always the key to building a useful statistical model.

[Image Analyst]

"This is model is made by myself," <== well, take everyone's advice that you thought up a bad model. Sorry, but not every arbitrary model is a good choice for any arbitrary set of data. In fact your data basically looks random, and there are so few points (17 points), that it's probably pointless to even try to apply ANY model to it.

[huazai2020]

@the cyclist, how many data should we have to do the machine learning?

[the cyclist]

It is not possible to give a simple answer to that question. How many data are needed depends on a number of factors, such as the number of dimensions, how correlated the variables are, and so on. Also, I would not get hung up on machine learning vs. other statistical modeling approach. They are often mathematically equivalent. (For example one can build an ordinary linear regression via a "machine learning" algorithm)

I am sorry if the following sounds harsh, but you are clearly very much a beginner in understanding mathematical modeling, and it is simply not possible to teach you everything you need to know, via this forum.

There are entire online courses devoted to how to think about problems like this. One of my favorites is Learning from Data.

[Stephen23]

"how many data should we have to do the machine learning?"

If you are looking for a simple number answer then you will be disappointed.

People have spent millions of human-hours researching and working in this field, if they had discovered a simple answer or formula we would all be using it. Instead you actually have to learn about machine learning (or whatever you choice of data fitting is) and how its limitations relate to the kind of data you are working with:

https://machinelearningmastery.com/much-training-data-required-machine-learning/

Five independent variables for seventeen data points is just noise.

[huazai2020]

What does you mean "Five independent variables for seventeen data points is just noise.", I do not understand

[Rik]

You have way too little data to fit that many variables. Only if your data fits the function perfectly does that make any sense.