Multiple regression by hand

Hi,

I want to compute the b of regression using matrix notation: Meaning this:

b1=regress(a,B) % Matlab function regress

into this

b2=((B'*B).^(-1))*B'*a % matrix notation

But I don't get the same betas: b1 is not equal to b2 when B has more than one column. But both are equal when it is simple regression. Not sure what I may be doing wrong,

Thank you, GM

[Rena Berman]

(Answers Dev) Restored edit

Answer 1

[Image Analyst]

What is your B?

I think the formula should be

b2 = inv(B'*B) * B' * a;

[Tahariet Sharon]

Thank you! I thought the inverse of a matrix could be expressed (B'*B).^(-1), but (B'*B).^(-1) is not equal to inv(B'*B).

[John D'Errico]

NO! NO! NO! NO! NO! NO! NO! NO! NO! NO! NO! NO! NO! NO! NO! NO! NO! NO! (once more for good measure... NO!)

Please do not teach people that the formula SHOULD be as you wrote it. That form is bad numerically for several reasons.

Instead, teach people that the correct formula in MATLAB is simply

b2 = B\a;

using backslash. Backslash has well written code inside it to solve that problem in a stable way. When B is near singular, the backslash variant is far more accurate, far better behaved than the form Image has written. If you would rather, a good alternative is:

b2 = pinv(B)*a;

Because this uses a singular value decomposition, it will be somewhat slower than backslash, although often not by a lot unless the problem is rather large. And the pinv (pseudo-inverse) solution has some virtues to it that some might prefer when the B matrix is rank deficient.

If you wanted to get more deeply into the linear algebra than simply using backslash, you could use a QR factorization of B. Best is to use the column-pivoted QR form, as generated when QR has three output arguments. With that, once can generate in fairly simple form a stable, well written code for the regression problem.

But PLEASE do NOT use the form as Image wrote it!!!!! Do yourself a favor and do not use it. More importantly, please, please, please do NOT teach others to use that form.

[Tahariet Sharon]

Thanks, thanks, thanks, thanks!

[Image Analyst]

John's correct that the more MATLAB-ish backslash method is preferable to the standard "book" formula like I gave, and which is what I thought you wanted. I think I can give myself a pat on the back though for raising John to a level of excitement rarely, if ever, seen before. :-) Now I have a challenge to see what it will take to get 30 NO's.