Image Labeler

Label images for computer vision applications

Description

The Image Labeler app enables you to label ground truth data in a collection of images. Using the app, you can:

• Define axis-aligned or rotated rectangular regions of interest (ROI) labels, line ROI labels, pixel ROI labels, polygon ROI labels, point ROI labels, projected cuboid ROI labels, and scene labels. Use these labels to interactively label your ground truth data.

• Use built-in detection or tracking algorithms to label your ground truth data.

• Write, import, and use your own custom automation algorithm to automatically label ground truth. See Create Automation Algorithm for Labeling.

• Evaluate the performance of your label automation algorithms using a visual summary. See View Summary of Ground Truth Labels.

• Export the labeled ground truth as a groundTruth object. You can use this object for system verification or for training an object detector or semantic segmentation network. See Training Data for Object Detection and Semantic Segmentation.

The Image Labeler app also enables you to create an individual- or a team labeling project. To launch the Image Labeler app, see Open the Image Labeler App.

After launching the Image Labeler app, select one of these options to create a new labeling project.

• New Individual Project — Create a labeling project for yourself. To get started using an individual labeling project, see Get Started with the Image Labeler.

• New Team Project — Create a labeling project for a team with multiple users. To get started with a team-based project, see Get Started with Team-Based Labeling.

The Image Labeler app supports all image file formats supported by the imread function and additionally supports the Digital Imaging and Communication in Medicine (DICOM) format including the ability to load multiframe data such as an ultrasound video. To read additional file formats supported by the Image Labeler app, you can create an imageDatastore and use the ReadFcn property. To label 2-D or 3-D medical image data stored in the DICOM, Neuroimaging Informatics Technology Initiative (NIfTI), or nearly raw raster data (NRRD) file formats, use the Medical Image Labeler (Medical Imaging Toolbox).

When loading images, if an image has a dimension larger than 8000 pixels or is a multiresolution image, the Image Labeler app offers you the option to convert the image into a blocked image. A blocked image consists of a large image that has been divided into smaller blocks that can fit in memory. Once the Image Labeler converts the large image into a blocked, you can process it in the app as you would any other image. While using blocked images enables you to process images in the app that you might not otherwise be able to, there are some limitations. For more information, see Label Large Images in the Image Labeler.

×

Open the Image Labeler App

• MATLAB^® Toolstrip: On the Apps tab, under Image Processing and Computer Vision, click the app icon.

• MATLAB command prompt: Enter imageLabeler.

Programmatic Use

imageLabeler

imageLabeler opens a new session of the app, enabling you to label ground truth data in images.

imageLabeler(imageFolder)

imageLabeler(imageFolder) opens the app and loads all the images from the folder named imageFolder.

The images in the folder can be unordered and can vary in size. To label a video, or a set of ordered images that resemble a video, use the Video Labeler app instead.

imageLabeler(imageDatastore)

imageLabeler(imageDatastore) opens the app and reads all of the images from an imageDatastore object. The ReadFcn property of the imageDatastore object specifies how to read the data.

For example, to open the app with a collection of stop sign images:

   stopSignsFolder = fullfile(toolboxdir("vision"),"visiondata","stopSignImages");
   imds = imageDatastore(stopSignsFolder)
   imageLabeler(imds)

imageLabeler(sessionFile)

imageLabeler(sessionFile) opens the app and loads a saved Image Labeler session, sessionFile. The sessionFile input contains the path and file name. The MAT-file that sessionFile points to contains the saved session.

imageLabeler(gTruth)

imageLabeler(gTruth) opens the app and loads a groundTruth object. The ground truth object data source must be an image collection or an imageDatastore.

More About

ROI Labels, Sublabels, and Attributes

On the left side of the app, the ROI Labels pane contains the region of interest (ROI) label definitions that you can mark on the frames. You can create label definitions directly from this pane. Alternatively, you can create label definitions programmatically by using a labelDefinitionCreator object and then import these label definitions into an app session.

Selecting ROI Label and Drawing Tool for Your Application

You can use labeled data to train or validate algorithms such as image classifiers, object detectors, and semantic and instance segmentation networks. Consider your application when choosing a labeling drawing tool to create ROI labels. The figure below shows labeling techniques for four different applications.

Application	Label Types	Example
Image Classification	N/A	Boat, Plane
Object Detection	Line Rectangle Rotated Rectangle Projected Cuboid
Semantic Segmentation	Pixel label Polygon
Instance Segmentation	Polygon

ROI Labels

An ROI label is a label that corresponds to a region of interest (ROI) in a signal frame. The table describes the supported label types.

ROI Label	Description	Scene
Rectangle	Draw rectangular ROI labels (bounding boxes) around objects.
Rotated Rectangle	Draw rotated rectangle ROI labels (bounding boxes) around objects
Projected cuboid	Draw cuboidal ROI labels (3-D bounding boxes).
Line	Draw linear ROI labels to represent lines. To draw a line ROI, use two or more points.
Pixel label	Assign labels to pixels for semantic segmentation: Label pixels automatically using Segment Anything Model (SAM). For an example, see Automatically Label Ground Truth Using Segment Anything Model. Label pixels manually using polygons, brushes, or flood fill. For more information, see Label Pixels for Semantic Segmentation
Polygon	Draw a pixel-filled polygon to label ground truth for instance segmentation. You can label distinct instances of the same class. For more information, see Label Objects Using Polygons.
Point	Draw point ROI labels for keypoint detection in objects.

ROI Sublabels

An ROI sublabel is an ROI label that belongs to a parent label. Use ROI sublabels to provide a greater level of detail about the ROIs in your labeled ground truth data. For example, a vehicle label might contain headlight, licensePlate, and wheel sublabels. You can create sublabels for rectangle, polygon, line, and projected cuboid labels. For more details about sublabels, see Use Sublabels and Attributes to Label Ground Truth Data.

ROI Attributes

An ROI attribute specifies additional information about an ROI label or sublabel. For example, in an ocean scene, attributes might include the type or color of a boat. The table describes the supported attribute types.

Attribute Type	Sample Attribute Definition	Sample Default Values
Numeric Value
String
Logical
List

For more details on attributes, see Use Sublabels and Attributes to Label Ground Truth Data.

Algorithms

You can use label automation algorithms to speed up labeling within the app. To create your own label automation algorithm to use within the app, see Create Automation Algorithm for Labeling. You can also use one of the built-in algorithms by following these steps:

1. Import the data you want to label, and create at least one label definition.

2. On the app toolstrip, click Select Algorithm and select one of the built-in automation algorithms.

3. Click Automate, and then follow the automation instructions in the right pane of the automation window.

ACF People Detector

Detect and label people using aggregate channel features (ACF). This algorithm is based on the peopleDetectorACF function. To use this algorithm, you must define at least one rectangle ROI label. You do not need to draw any ROI labels.

To help improve the algorithm results, first click Settings. You can change any of these settings.

• The pretrained people detector model that the algorithm uses — The 'inria-100x41' model was trained using the INRIA person data set. The 'caltech-50x21' model was trained using the Caltech Pedestrian data set.

• The overlap ratio threshold, from 0 to 1, for detecting people — When rectangle ROIs overlap by more than this threshold, the algorithm discards one of the ROIs.

• The classification score threshold for detecting people — Increase the score to increase the prediction confidence of the algorithm. Rectangles with scores below this threshold are discarded.

ACF Vehicle Detector (requires Automated Driving Toolbox)

Detect and label vehicles using aggregate channel features (ACF). This algorithm is based on the vehicleDetectorACF (Automated Driving Toolbox) function. To use this algorithm, you must define at least one rectangle ROI label. You do not need to draw any ROI labels.

To help improve the algorithm results, first click Settings. You can change any of these settings.

• The pretrained vehicle detector model that the algorithm uses — The 'full-view' model was trained using unoccluded images of the front, rear, left, and right sides of vehicles. The 'front-rear-view' model was trained using images of only the front and rear sides of the vehicle.

• The overlap ratio threshold, from 0 to 1, for detecting vehicles — When rectangle ROIs overlap by more than this threshold, the algorithm discards one of the ROIs.

• The classification score threshold for detecting vehicles — Increase the score to increase the prediction confidence of the algorithm. Rectangles with scores below this threshold are discarded.

You can also configure the detector with a calibrated monocular camera by importing a monoCamera (Automated Driving Toolbox) object into the MATLAB workspace. Specify the length and width ranges of the vehicle in world units, such as meters.

Version History

Introduced in R2018a

R2024b: Segment and label ground truth for semantic segmentation using Segment Anything Model (SAM)

Use the Segment Anything pixel labeling tool within the Image Labeler app to automatically segment images using the Segment Anything Model (SAM) and create pixel labels. This functionality requires the Image Processing Toolbox™ Model for Segment Anything Model support package. You can install the support package from Add-On Explorer. For more information about installing add-ons, see Get and Manage Add-Ons. The support package also requires Deep Learning Toolbox™.

Assign pixel labels to objects by segmenting objects in an image with just a few clicks. Alternatively, using the Segment Full Image option of the Segment Anything tool, segment the entire image into regions and click on objects to easily create pixel labels for all the objects in a scene. For an example, see Automatically Label Ground Truth Using Segment Anything Model. To learn more about the SAM, see Get Started with Segment Anything Model for Image Segmentation.