How to find training confusion matrix for model exported from classification learner app?

I used the classification learner app to train my SVM models and obtained a classification accuracy. I then exported the model to MATLAB's workspace and used it to predict my test data and obtained my testing confusion matrix.

The issue is that I forgot to export the confusion matrix of the trained model and now require it to find out the precision, recall and F-score for my training data.

Is there any way that I can obtain the training confusion matrix or would I have to train another model?

P.S. The only hesitancy in retraining models is that I would have to train more than 150 models as I had a large set of experiments for my study.

[Abdullah Özkan]

I am not an expert but I believe you can use confusionmat function. Actually, I have faced a similar problem and wrote the following function. I hope it helps.

function [CM, Precision, Recall, F1Score] = MultiClassPerformanceMetrics (ActualLabels, PredictedLabels)

CM = confusionmat(ActualLabels, PredictedLabels);

Diagonal = diag(CM);

SumOfRows = sum(CM,2);

SumOfColumns = sum(CM,1);

EachPrecision = Diagonal ./ SumOfRows;

Precision = mean(EachPrecision);

EachRecall = Diagonal ./SumOfColumns';

Recall = mean(EachRecall);

F1Score = (2*Precision*Recall) / (Precision+Recall);

end

Answer 1

[Drew]

Given:

(1) A model exported from Classification Learner is in the workspace as trainedModel (that is the default name when exporting from Classification Learner)

(2) The corresponding training data is in the workspace variable "x"

(3) The corresponding class labels for the training data are in the workspace variable "y"

Then the confusion matrix on the training data for this model exported from Classification Learner is given by:

>> confusionmat(trainedModel.predictFcn(x),y)

As another option: If the Model was exported from classification learner as a full model, rather than a compact model, then x (the training data) and y (the class labels for the training data) are generally available in the exported data structure, inside the model object. For example, if the exported model is of type ClassificationTree, then the confusion matrix on the training data is given by:

>> confusionmat(trainedModel.predictFcn(trainedModel.ClassificationTree.X),trainedModel.ClassificationTree.Y)

For a simple, similar example at the command line:

t=readtable('fisheriris.csv');
Mdl=fitctree(t,"Species");
confusionmat(Mdl.predict(t),t.Species)

ans = 3×3

50 0 0 0 47 0 0 3 50

Note the low error rate, due to "cheating" by training and testing on the same data. All of the above examples are trained and tested on the entire training set. This is not a good way to estimate the expected accuracy on future unseen test data. To estimate the expected accuracy on future unseen test data, use a separate test set, or use a validation technique like cross-validation (see https://www.mathworks.com/help/stats/classificationsvm.crossval.html).