Visual bag-off-features evalute vs. fitcecoc classification results are very different

5 views (last 30 days)
Preetham Manjunatha
Preetham Manjunatha on 9 Nov 2017
Commented: ahmed shahin on 21 May 2021
Dear All:
I have few questions regarding vision toolbox's visual bags-of-features class. Firstly, I have a feature matrix created using (encode, bagOfFeatures functions) visual-bags-of-words using computer vision toolbox, it is 500 x 14404 (Xtrain = 10793, XVal = 1204 and XTest = 2407 samples). There are 14 classes, target matrix is 14 x 14404 or 14404 X 1 categroical classes. When I use Matlab's default method "evaluate" provided in visual bags-of-words example (I presume it uses multiclass SVM "fitcecoc") to classify the test data I am getting a decent precison/recall values around 75% which is decent for the given dataset. Code snippet is given below:
categoryClassifier = trainImageCategoryClassifier(imdsTrainRandomized, bag_Train_BoFOri);
[confMatTr,knownLabelIdxTr,predictedLabelIdxTr,scoreTr] = evaluate(categoryClassifier,imdsTrainRandomized);
[confMatVl,knownLabelIdxVl,predictedLabelIdxVl,scoreVl] = evaluate(categoryClassifier,imdsValRandomized);
[confMatTs,knownLabelIdxTs,predictedLabelIdxTs,scoreTs] = evaluate(categoryClassifier,imdsTestRandomized);
Whereas, if I use fitcecoc to classify train/test dataset using multiclass SVM. I am getting testing accuracy of only ~9-10%. Precision/recall values less than 8%. Also, I noticed runtime takes less than 3 minutes to train, where I am skeptical about. Below is the multiclass SVM code snippet:
t = templateSVM('KernelFunction','polynomial', 'PolynomialOrder',2);
options = statset('UseParallel',0);
MdlSVM = fitcecoc(Xtrain,Labelstrain, 'Coding','onevsone','Learners',t ,...
isLoss = resubLoss(MdlSVM);
CVSVMModel = crossval(MdlSVM);
FirstModel = CVSVMModel.Trained{1};
yTestSVM = predict(FirstModel,Xtest);
percentErrorsSVM = sum(yTestSVM ~= Labelstest)/numel(Labelstest);
accuracySVM = 1 - percentErrorsSVM
Is there a way we can find the optimal parameters tuned in "evaluate" method? Also, I appreciate if there are any suggestions.

Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by