updateSequence

Update video sequence for classification

Syntax

classifierUpdated = udpateSequence(classifier,videoFrame)

classifierUpdated = udpateSequence(classifier,videoFrame,ExecutionEnvironment=env)

Description

classifierUpdated = udpateSequence(classifier,videoFrame) updates the video sequence with the video frame videoFrame, and outputs an updated classifier. The output contains the updated classifier object.

To classify the updated sequence, use the classifySequence object function.

The video classifier object maintains the sequence as a first-in first-out (FIFO) queue. The InputSize property of the classifier object specifies the number of frames in the queue.

classifierUpdated = udpateSequence(classifier,videoFrame,ExecutionEnvironment=env) specifies the hardware resources for running the classifier in addition to any combination of arguments from previous syntaxes, as one of these options:

• "auto" — Sets the execution environment to the GPU, if available. Otherwise the function sets it to the CPU.

• "gpu" — Sets the execution environment to the GPU. Usage of the GPU requires Parallel Computing Toolbox™ and a CUDA^® enabled NVIDIA^® GPU. For information about the supported compute capabilities, see GPU Support by Release (Parallel Computing Toolbox).

• "cpu" — Sets the execution environment to the CPU.

Examples

Classify Video Sequences in a Video File

This example shows how to classify video sequences in a video file using a SlowFast Video Classifier pretrained on the Kinetics-400 video activity recognition dataset. To learn more about how to train a video classifier network for your dataset, see Gesture Recognition using Videos and Deep Learning.

Load SlowFast Video Classifier

sf = slowFastVideoClassifier();

Setup Video Player and Video Reader

Specify the video file name to stream video frames.

videoFilename = "pushup.mp4";

Create a VideoReader to read video.

reader = VideoReader(videoFilename);

Setup a video player.

player = vision.VideoPlayer;

Classify Video Sequences

Specify how frequently the classifier should be applied to incoming video frames.

classifyInterval = 10;

A value of 10 balances runtime performance against classification performance. Increase this value to improve runtime performance at the cost of missing actions from the video file.

Obtain the sequence length of the SlowFast Video Classifier. Classify only after capturing at least sequenceLength number of frames from the video file.

sequenceLength = sf.InputSize(4);

Read video frames using the hasFrame and readFrame functions of the VideoReader. Using the updateSequence function update the video classifier's sequence. Using the classifySequence function classify the updated sequence.

numFrames = 0;
text = "";

while hasFrame(reader)
    frame = readFrame(reader);
    numFrames = numFrames + 1;

    % Update the sequence with the next video frame.
    sf = updateSequence(sf,frame);

    % Classify the sequence only at every classifyInterval number of frames.
    if mod(numFrames, classifyInterval) == 0 && numFrames >= sequenceLength
        [label,score] = classifySequence(sf);
        text = string(label) + "; " + num2str(score, "%0.2f");
    end
    frame = insertText(frame,[30,30],text,'FontSize',24);
    step(player,frame);
end

Input Arguments

classifier — Classifier object

r2plus1dVideoClassifier object | slowFastVideoClassifier object

Classifier object, specified as r2plus1dVideoClassifier or slowFastVideoClassifier object.

videoFrame — Video frame

numeric array

Video frame for updating the classifier, specified as an H-by-W-by-C numeric array. H, W, and C represent the height, width, and number of channels, respectively. The number of channels must match the number of channels set by the InputSize property of the classifier object.

Output Arguments

classifierUpdated — Updated video classifier

r2plus1dVideoClassifier object | slowFastVideoClassifier object

Updated video classifier, returned as a r2plus1dVideoClassifier or slowFastVideoClassifier object.