Main Content

anomalyThreshold

Optimal anomaly threshold for set of anomaly scores and corresponding labels

collapse all in page

    Syntax

    t = anomalyThreshold(trueLabels,scores,truePosLabels)
    t = anomalyThreshold(trueLabels,scores,truePosLabels,optimMethod)
    t = anomalyThreshold(trueLabels,scores,truePosLabels,MaxFalsePositiveRate=maxFPR)
    t = anomalyThreshold(trueLabels,scores,truePosLabels,MaxFalseNegativeRate=maxFNR)
    [t,anomalyROC] = anomalyThreshold(___)

    Description

    t = anomalyThreshold(trueLabels,scores,truePosLabels) calculates the optimal anomaly threshold given per-image anomaly scores and corresponding ground truth labels, with positive labels specified by truePosLabels. When performing anomaly detection, images with scores below the calculated threshold are considered normal images and images with scores above the threshold are considered anomalous images.

    Note

    This function requires the Computer Vision Toolbox™ Automated Visual Inspection Library. You can install the Computer Vision Toolbox Automated Visual Inspection Library from Add-On Explorer. For more information about installing add-ons, see Get and Manage Add-Ons. To run this function, you will require the Deep Learning Toolbox™.

    t = anomalyThreshold(trueLabels,scores,truePosLabels,optimMethod) also specifies the optimization method.

    t = anomalyThreshold(trueLabels,scores,truePosLabels,MaxFalsePositiveRate=maxFPR) also specifies the maximum false positive rate.

    t = anomalyThreshold(trueLabels,scores,truePosLabels,MaxFalseNegativeRate=maxFNR) also specifies the maximum false negative rate.

    example

    [t,anomalyROC] = anomalyThreshold(___) also returns the receiver operating characteristic (ROC) curve and performance metrics.

    Examples

    collapse all

    This example uses:

    Open Live Script

    Load calibration images and corresponding labels, then create a datastore that reads the calibration data. The data set consists of grayscale images of handwritten digits 0–9.

    [Xcal,catLabels] = digitTest4DArrayData;
dsCal = arrayDatastore(Xcal,IterationDimension=4);

    Load a pretrained FCDD anomaly detector. This detector has been trained to classify the digit 8 as normal and all other digits as anomalies. Therefore, convert the categorical labels to binary labels by setting the categorical label 8 as 0 (false) and all other labels as 1 (true).

    load("digit8AnomalyDetector.mat");
binaryLabels = catLabels~="8";

    Predict the anomaly score of each calibration image.

    scores = predict(detector,dsCal);

    Calculate the optimal threshold and corresponding ROC metrics from the anomaly scores and ground truth binary labels.

    [T,roc] = anomalyThreshold(binaryLabels,scores,true)
    T = single
    2.9632e-04

    roc = 
  rocmetrics with properties:

    Metrics: [4976x4 table]
        AUC: 0.9114


  Properties, Methods

    Set the Threshold property of the FCDD anomaly detector as the optimal threshold.

    net.Threshold = T;

    Input Arguments

    collapse all

    Ground truth labels for each image, specified as a numeric vector, logical vector, or categorical array. The anomalyThreshold function converts the labels into a logical vector according to the set of positive labels in truePosLabels.

    Predicted anomaly score for each image, specified as a numeric vector of the same length as trueLabels.

    Ground truth positive labels, specified as a vector of the same data type as trueLabels.

    The anomalyThreshold function converts all ground truth labels in trueLabels that belong to the set of positive labels to a logical true, indicating a positive detection. The function converts all other ground truth labels to a logical false, indicating a negative detection.

    Optimization method, specified as one of the values in the table.

    Optimization MethodDescription
    "YoudensIndexROC"

    The optimal threshold is the point on the ROC curve with the maximum difference between the true positive rate (TPR) and false positive rate (FPR). The difference between TPR and FPR is also known as Youden's index or Youden's J statistic.

    "NearestZeroOneROC"

    The optimal threshold is the point on the ROC curve nearest to the coordinate (0, 1), where the FPR is 0 and the TPR is 1. This point minimizes the Euclidean distance between the ROC curve and the (0,1) point.

    "MaxF1Score"

    The optimal threshold is the point on the precision-recall (PR) curve that maximizes the F1 score. This value is the harmonic mean of the precision and recall values.

    Data Types: char | string

    Maximum false positive rate (FPR), specified as a number in the range [0, 1]. The optimal threshold is the point on the ROC curve where the FPR is less or equal to maxFPR.

    Maximum false negative rate (FNR), specified as a number in the range [0, 1]. The optimal threshold is the point on the ROC curve where the FNR is less or equal to maxFNR.

    Output Arguments

    collapse all

    Optimal threshold, returned as a numeric scalar of the same data type as scores.

    ROC curve and performance metrics, returned as an rocmetrics (Deep Learning Toolbox) object.

    Tips

    • You can plot ROC and PR curves returned by anomalyROC using the plot (Deep Learning Toolbox) function.

    Version History

    Introduced in R2022b

    See Also

    | (Deep Learning Toolbox)

    Topics