metricsByArea

Evaluate detection performance across object size ranges

Since R2023b

collapse all in page

Syntax

areaMetrics = metricsByArea(metrics,objectAreaRanges)
areaMetrics = metricsByArea(metrics,objectAreaRanges,ClassNames)

Description

areaMetrics = metricsByArea(metrics,objectAreaRanges) evaluates object detection metrics for the object size range or ranges, specified as bounding box areas, objectAreaRanges. The function evaluates the specified bounding box size range within the existing object detection metrics.

example

areaMetrics = metricsByArea(metrics,objectAreaRanges,ClassNames) also specifies the names of the classes for which to evaluate area-based metrics.

Examples

collapse all

Open Live Script

Load a table containing images and ground truth bounding box labels. The first column contains the images, and the remaining columns contain the labeled bounding boxes.

data = load("vehicleTrainingData.mat");
trainingData = data.vehicleTrainingData;

Set the value of the dataDir variable as the location where the vehicleTrainingData.mat file is located. Load the test data into a local vehicle data folder.

dataDir = fullfile(toolboxdir("vision"),"visiondata");
trainingData.imageFilename = fullfile(dataDir,trainingData.imageFilename);

Create an imageDatastore using the files from the table.

imds = imageDatastore(trainingData.imageFilename);

Create a boxLabelDatastore using the label columns from the table.

blds = boxLabelDatastore(trainingData(:,2:end));

Load a pretrained YOLO v2 object detector trained to detect vehicles into the workspace.

vehicleDetector = load("yolov2VehicleDetector.mat");
detector = vehicleDetector.detector;

Compute Object Detection Performance Metrics

Run the detector on the test images. Set the detection threshold to a low value to detect as many objects as possible. This helps you evaluate the detector precision across the full range of recall values.

results = detect(detector,imds,Threshold=0.01);

Compute metrics for evaluating the performance of an object detector using the evaluateObjectDetection function.

metrics = evaluateObjectDetection(results,blds);

Evaluate Metrics Across Object Size Ranges

Extract the anchor boxes from the detector, calculate their areas, and sort the areas.

areas = prod(detector.AnchorBoxes,2);
sortedAreas = sort(areas);

Define area range limits using the calculated areas. The upper limit for the last range is set to three times the size of the largest area, which is sufficient for the objects in this data set. 

lowerLimit = sortedAreas;
upperLimit = [sortedAreas(2:end);3*sortedAreas(end)];
areaRanges = [lowerLimit upperLimit]
areaRanges = 4×2

          64         960
         960        1536
        1536        3456
        3456       10368

Evaluate detector performance across vehicle size ranges using the metricsByArea object function. Specify the object size ranges using the bounding box area ranges. The number of vehicles falling into each area range is computed in the NumObjects column. 

areaMetrics = metricsByArea(metrics,areaRanges,ClassName="vehicle");
disp(areaMetrics(:,1:3))
      AreaRange      NumObjects    APOverlapAvg
    _____________    __________    ____________

      64      960       119           0.9887   
     960     1536        12          0.99359   
    1536     3456        45          0.99952   
    3456    10368       141          0.98551

Plot the average precision (AP) metric, averaged across all overlap thresholds, as a function of the mean object size per vehicle size range.

figure
bar(categorical(mean(areaRanges,2)),areaMetrics.APOverlapAvg)
title("AP by Vehicle Size")
xlabel("Vehicle Anchor Box Mean Area")
ylabel("AP")
ylim([0.984 1])
grid on

Figure contains an axes object. The axes object with title AP by Vehicle Size, xlabel Vehicle Anchor Box Mean Area, ylabel AP contains an object of type bar.

Input Arguments

collapse all

Object detection metrics, specified as an objectDetectionMetrics object.

Bounding box area ranges (object size ranges), specified as one of the following:

  • An M-by-2 numeric matrix, with each row specifying the upper (exclusive) and lower (inclusive) limits of an object area range in evaluation. For example, if the input is [x1 x2; x3 x4]. The evaluated bounding box area ranges are x1≤ area <x2 and x3≤ area <x4.

  • A vector with M+1 elements, specifying the edge values of M contiguous area ranges. Each range is inclusive in the lower limit, and exclusive in the upper limit. For example, if the input is [x1 x2 x3], the area ranges evaluated are x1≤ area <x2 and x2≤ area <x3.

Names of the classes for which to evaluate area-based metrics. If multiple classes are specified, the function computes averaged metrics across the classes. By default, all classes will be considered and the returned metrics will be the average of all classes.

Output Arguments

collapse all

Metrics by area, returned as a table with M rows. Each row contains metrics using objects within an area range (for a total of M area ranges). The columns are specified by these table variables.

  • AreaRange: The upper and lower limits of each area range, returned as a two-element row vector.

  • NumObjects: The number of objects falling within each area range, returned as a positive integer.

  • mAP: Mean average precision (mAP), or average precision averaged over all classes, computed at all overlap thresholds specified by the OverlapThreshold property, returned as a numThresh-by-1 numeric vector. numThresh is the number of overlap thresholds.

  • mAPOverlapAvg: Mean average precision (mAP) averaged over all thresholds specified by the OverlapThreshold property, returned as a numeric scalar.

Version History

Introduced in R2023b

See Also

Apps

Functions

Topics