Shapley based feature selection

Hello everyone, I am trying to perform Shapley based feature selection. I wrote the code below but I did not use the Ytest variable. Xtest does not contain the class labels; they are in the Ytest variable. I am a little confused. Am I doing something wrong? Thanks for the help.

DataSet = load('Seeds.txt');
[~,nFeatures] = size(DataSet);
X = DataSet(:,(1:nFeatures - 1));
Y = DataSet(:,nFeatures); 
c = cvpartition(Y, 'Holdout', 0.20, 'Stratify', true);
Xtrain = X(training(c), :); 
Xtest = X(test(c), :); 
Ytrain = Y(training(c)); 
Ytest = Y(test(c)); 
Mdl = fitcecoc(Xtrain, Ytrain);
LimeRes = shapley(Mdl);
FitRes = fit(LimeRes, Xtest);
plot(FitRes)

[the cyclist]

Can you upload the data? You can use the paper clip icon in the INSERT section of the toolbar.

[MB]

Sorry for the late reply. I’ve just uploaded the data.

[MB]

I found a Python example that can help: Python Example.

Answer 1

[the cyclist]

The Shapley values don't require the class labels (i.e. the actual responses) to determine feature importance.

The Shapley values only indicate, for a given model, how much each feature affects the predicted class label. For example, suppose you are trying to predict whether someone is going to repay their car loan on time. For borrower Alice, the model might predict "NO", because she already has a lot of debt. For borrower Bob, the model might also predict "NO", but because Bob has low income (even if he has low debt).

The Shapley values of debt and income will be different for Alice and Bob. It does not matter whether they actually default or not. The Shapley value is explaining only where the prediction came from.

I hope that helps.

[MB]

Thank you both.

Answer 2

[Walter Roberson]

YtestPred = predict(Mdl, Xtest);
test_accuracy = nnz(Ytest(:) == YtestPred(:)) / numel(Ytest) * 100;
fprintf('test accuracy: %.2f\n', test_accuracy);