How to use the 'perfcurve' of Matlab with specific inputs?

Hi Smart guys,

I wrote following codes to get a plot of 'classification accuracy' vs. 'threshold':

(The datasets has the ground truth contains two classes labelled 'Good' or 'Bad')

    LDAClassifierObject                     = ClassificationDiscriminant.fit(featureSelcted, groundTruthGroup, 'DiscrimType', 'linear');
    [LDALabel, LDAScore]                     = resubPredict(LDAClassifierObject);

    [~, AccuracyLDA, Thr]                   = perfcurve(groundTruthNumericalLable(:,1), LDAScore(:,1), 1,'yCrit','accu');
    figure,
    plot(Thr,AccuracyLDA,'r-');
    hold on;
    plot(Thr,AccuracyLDA,'bo');
    xlabel('Threshold for ''good'' Returns');
    ylabel('Classification Accuracy');
    grid on;

    [maxVal, maxInd] = max(AccuracyLDA)

    maxVal =

        0.8696

    maxInd =

        15

    Thr(15)

    ans =

        0.7711

Also, I run the ROC analysis for the same datasets that the ground truth contains two classes labelled 'Good' or 'Bad'

    [FPR, TPR, Thr, AUC, OPTROCPT]       = perfcurve(groundTruthGroup(:,1), LDAScore(:,1), 'Good');

    OPTROCPT =

        0.1250    0.8667

Why Thr(15)=0.7711 is different from OPTROCPT(2)=0.8667 ?

Is the best cut-off point (ie, the best threshold OPTROCPT) obtained by ROC is the one has maximum accuracy of LDA?

Or maybe I am wrong, then what exactly `perfcurve(groundTruthNumericalLable(:,1), LDAScore(:,1), 1,'yCrit','accu')` tell us?

Thanks a lot.

A.

[Ilya]

Why would any value in Thr be equal to any value in OPTROCPT? Could you copy and paste the part in the doc that made you believe these two should be equal?

Answer 1

[Neil Caithness]

OPTROCPT(2) is the TPR value of the optimal cut-point, not the threshold value itself.

In your example, try

a = find(TPR==OPTROCPT(2))

One of these should be your index value 15, then

Thr(a)

should be your optimal threshold value 0.7711