performing logistic regression with categorical predictors

Hello, 

I am performing a logistic regression with 3 numerical predictors AND 2 categorical predictors. These are my X's. My Y is 1 or 0. I do not understand the error message that Matlab is giving.

What does this mean:

Warning: Matrix is close to singular or badly scaled. Results may be inaccurate. RCOND =
8.982089e-22. 

And

Error using linsolve
Matrix must be positive definite.

Error in mnrfit (line 252)
            bcov = linsolve(hess,eye(size(hess)),struct('SYM',true,'POSDEF',true));

What am I doing wrong?

[claudia cardarelli]

hello have you found a solution? even i have this problem

Answer 1

[Jeff Miller]

This message likely means that two or more of your predictors are redundant. For example, two of the numerical X predictors might be very highly correlated (positive or negative). Or, the categorical predictors might divide up the cases into almost identical groups (e.g., with eye color and hair color as categories, you might have only blue-eyed blondes and brown-eyed brunettes, so you get the same groups dividing by either attribute). It is also possible for categorical and X predictors to be redundant. Anyway, regression models (logistic or regular) can't handle redundant predictors, essentially because there is no way for the model to estimate separate influences for two identical/equivalent predictors.

The solution is usually to identify the redundant predictors with correlations or crosstabs, and then drop one of them from the predictive model.