MATLAB Answers

0

Does the type of classifier make that much difference in feature selection in sequentialfs function?

Asked by Sara Salimi on 2 Jan 2019
Latest activity Commented on by Don Mathis on 10 Jan 2019
Hi,
I am wondering if does it make significant difference in feature selcetion if we are using difference classifiers during feature selection?
For example, I am using the following naive bayes classifier:
fun = @(Xtrain,Ytrain,Xtest,Ytest)...
sum(Ytest~=predict(fitcnb(Xtrain,Ytrain,'Distribution','kernel'),Xtest));
[inmodel,history] = sequentialfs(fun,myData,classes,'cv',c2,'options',opts,'nfeatures',60);
or if I use fitsvm, instead of fitcnb, will make that much difference on the selceted features?
Thanks

  0 Comments

Sign in to comment.

1 Answer

Answer by Don Mathis on 4 Jan 2019

I think in general it will make a difference which classifier you use, because different classifiers deterct different kinds of patterns. An SVM could discover that a pair of uncorrelated predictors are both relevant because it can learn nonlinear I/O relationships, while a linear classifier might not be able to detect that nonlinear structure, and then "decide" that both predictors are irrelevant.

  2 Comments

Thank you very much for your help,
So, can I come to this conclusion that for finding the relevant features, it would be better to use SVM?
I don't think there's any right answer here. You could even try both and take the union of the features. The features you get will also depend on the hyperparameters you choose for your SVM or NB models.

Sign in to comment.