Regression design matrix is rank deficient to within machine precision. How do I interpret this error?

I tried using Linear Regresssion commant regress on my train and test data and I am getting a warning saing 'X is rank deficient to within machine precision'. I am not able to interpret the error.

Answer 1

[Star Strider]

It means that at least one of the columns in the design matrix is close to being all zeros.

Without knowing more, one way to avoid that could be to re-scale all the variables (independent and dependent) to some larger values. Re-scaling them could mean adding a constant value to all of them. This would need to be done with caution, since it would be possible to end up with useless results.

[Rucha Apte]

Got it ! Thank you!

[Star Strider]

As always, my pleasure!

[Sascha Frölich]

Hey, I get the same error, and no matter what large values I add to my design matrix (to the point that every value is way beyond zero), the error persists. Why could that be?

Nevermind I just figured it out; I had included a constant regressor, while MATLAB includes an intercept term by itself, so my design matrix was redundant. Cheers!

[Star Strider]

@Sascha Frölich —

One possibility is that one or more columns of the design matrix are linearly dependent.

x = randn(5,1);
DM = [x x+eps ones(size(x))];
y = randn(5,1);
B = DM \ y

Warning: Rank deficient, rank = 2, tol = 2.557037e-15.

B = 3×1

-0.8939 0 0.3358

Here, the first and second columns of ‘DM’ are liniearly dependent withiin machine tolerance.

.