visibleKeypoints

Extract visible keypoints from keypoints detected by HRNet object keypoint detector

Since R2023b

Syntax

visiblePoints = visibleKeypoints(detector,keypoints,scores,valid)

[visiblePoints,visibleScores,visibleLabels] = visibleKeypoints(detector,keypoints,scores,valid)

Description

visiblePoints = visibleKeypoints(detector,keypoints,scores,valid) extracts visible object keypoint locations from the object keypoint locations keypoints detected by the HRNet object keypoint detector detector. The function obtains the locations of visible object keypoints by using the keypoint validity values valid, which are based on the keypoint detection confidence scores scores.

[visiblePoints,visibleScores,visibleLabels] = visibleKeypoints(detector,keypoints,scores,valid) returns the detection confidence scores for the visible keypoints visibleScores, and the labels assigned to the visible keypoints visibleLabels.

Examples

Detect Visible Keypoints in Objects

Read a sample image into the workspace.

I = imread("visionteam.jpg");

Crop and display one person from the image.

personBox = [18.51 42.51 137.21 370.98];
[personImg] = imcrop(I,personBox);
figure
imshow(personImg)

Specify a bounding box for the person in the image. You can use the bounding box region as input to the object keypoint detector to detect the person.

bbox = [3.87 21.845 118.97 345.91];

Create an HRNet object keypoint detector by using a pretrained HRNet deep learning network.

keypointDetector = hrnetObjectKeypointDetector("human-full-body-w32");

Detect the object keypoints of the person in the test image by using the pretrained HRNet object keypoint detector.

[keypoints,keypointScores,valid] = detect(keypointDetector,personImg,bbox);

Detect keypoints with keypoints confidence scores greater than 0.9.

isValid = keypointScores > 0.9;

Extract visible keypoints from keypoints detected by HRNet object keypoint detector.

[visiblePoints,visibleScores,visibleLabels] = visibleKeypoints(keypointDetector,keypoints,keypointScores,isValid);

Insert and display visible keypoints.

detectedKeypoints = insertObjectKeypoints(personImg,visiblePoints{1},KeypointColor="yellow",KeypointSize=4);
figure
imshow(detectedKeypoints);

Display all the visible keypoint levels.

visibleLabels{1}

ans = 10×1 categorical
     Nose 
     Left-eye 
     Right-eye 
     Left-ear 
     Right-ear 
     Left-shoulder 
     Right-shoulder 
     Left-elbow 
     Right-elbow 
     Left-wrist 

Input Arguments

detector — HRNet object keypoint detector

hrnetObjectKeypointDetector object

HRNet object keypoint detector, specified as an hrnetObjectKeypointDetector object.

keypoints — Locations of object keypoints detected in input image

17-by-2-by-M array | T-by-1 cell array

Locations of object keypoints detected in the input image or images, returned as one of these options:

• 17-by-2-by-M array — The keypoints come from a single test image. Each row in the array is of the form [x y] where x and y specify the location of a detected keypoint in an object. M is the number of objects in the image.

• T-by-1 cell array — The keypoints come from an array of test images. T is the number of test images in the array. Each cell in the cell array contains a 17-by-2-by-M array specifying the keypoint detections for the M objects in the image.

Each object has 17 detected keypoints.

Data Types: double

scores — Detection confidence scores for object keypoints

17-by-M matrix | T-by-1 cell array

Detection confidence scores for object keypoints, returned as one of these options:

• 17-by-M matrix — The keypoint scores come from a single test image. M is the number of objects in the image.

• T-by-1 cell array — The keypoint scores come from an array of test images. T is the number of test images in the array. Each cell in the array contains a 17-by-M matrix indicating the detection scores for the keypoints of each of the M objects in an image.

Each object in an image has 17 has keypoint confidence scores. A higher score indicates higher confidence in the detection.

Data Types: double

valid — Validity of detected object keypoints

17-by-M matrix | T-by-1 cell array

Validity of the detected object keypoints, returned as one of these options:

• 17-by-M logical matrix — The keypoint validity values come from a single test image. M is the number of objects in the image.

• T-by-1 cell array — The keypoint validity values come from an array of test images. T is the number of test images in the array. Each cell in the array contains a 17-by-M logical matrix indicating the keypoint validity values for the keypoints of each of the M objects in the corresponding image.

Each object in an image has 17 keypoint validity values. A value of 1 (true) indicates a valid keypoint and 0 (false) indicates an invalid keypoint.

Data Types: logical

Output Arguments

visiblePoints — Locations of visible keypoints

17-by-2-by-M array | T-by-1 cell array

Location of visible keypoints detected in the input image or images, returned as one of these options:

• 17-by-2-by-M array — The visible keypoints come from a single test image. Each row in the array is of the form [x y] where x and y specify the location of a detected visible keypoint in an object. M is the number of objects in the image.

• T-by-1 cell array — The visible keypoints come from an array of test images. T is the number of test images in the array. Each cell in the cell array has an N-by-2 matrix specifying the valid keypoint locations of the form [x, y]. N is the number of visible keypoints for each object in the image, as determined by the scores input argument. N is less than or equal to 17. M is the number of objects in the image.

visibleScores — Detection confidence scores of visible keypoints

17-by-M matrix | T-by-1 cell array

Detection confidence scores of visible keypoints, returned as one of these options:

• 17-by-2-by-M array — The detection scores of visible keypoints come from a single test image. M is the number of objects in the image.

• T-by-1 cell array — The detection scores of visible keypoints come from an array of test images. T is the number of test images in the array. Each cell in the cell array has an N-by-1 matrix specifying the visible keypoint scores. N is the number of visible keypoints for each object in the image, as determined by the scores input argument. N is less than or equal to 17. M is the number of objects in the image.

visibleLabels — Validity labels of visible keypoints

17-by-M matrix | T-by-1 cell array

Validity labels of visible keypoints, returned as one of these options:

• 17-by-1-by-M array — The validity labels of visible keypoints come from a single test image. M is the number of objects in the image.

• T-by-1 cell array — The validity labels of visible keypoints come from an array of test images. T is the number of test images in the array. Each cell in the cell array has an N-by-1 matrix specifying the valid keypoint labels. N is the number of visible keypoints for each object in the image, as determined by the scores input argument. N is less than or equal to 17. M is the number of objects in the image.

Version History

Introduced in R2023b