Video applications present common but difficult challenges that require flexible analysis and processing functionality. Using MATLAB® and Simulink® products, you can develop solutions to common video processing challenges such as video stabilization, video mosaicking, target detection, and tracking.
Object tracking is an essential part of many applications including pedestrian avoidance, security and surveillance, and augmented reality. In this example, a small ball is tracked using Kalman filtering.
Object Detection and Counting
Video processing can be used to detect and count objects that move in video sequences. In this case study, scientists in Australia are using video footage to estimate the wildlife population of waterbirds.
Video Processing in 4 Easy Steps
Video processing in MATLAB involves the following steps:
- Reading the video
- Displaying the video
- Processing the video
- Writing the video
Step 1. Reading the Video
You can read video from files or directly from cameras.
A single MATLAB command lets you read in videos from a file:
>> vid = VideoReader('filename.avi')
MATLAB lets you read video files using a variety of codecs including OS-specific codecs for Microsoft® Windows®, Mac, and Linux®.
Step 3. Processing the Video
A video is a sequence of individual video frames, or images. This means an algorithm designed to perform edge detection on an image can be quickly converted to perform edge detection on a video.
Read single image
|Read image frame from video|
Video processing can be very simple, as in the example using edge detection, or significantly more complex, such as tracking algorithms that must account for an object’s location in previous frames.
For more information on advanced video processing, see examples for:
Step 4. Writing the Video
After processing, you can write each frame of a video back to a file. You can create a video file with the function:
>> vid_w = VideoWriter('newfile.avi'); >> open(vid_w)
vid_w can accumulate new frames to create a video.
A Complete MATLAB Example
Putting all the components together, let’s run through a complete example to show the steps of reading, displaying, processing, and writing video:
%% Read and process a video into MATLAB % Setup: create Video Reader and Writer videoFileReader = VideoReader('tilted_face.avi'); myVideo = VideoWriter('myFile.avi'); % Setup: create deployable video player and face detector depVideoPlayer = vision.DeployableVideoPlayer; faceDetector = vision.CascadeObjectDetector(); open(myVideo); %% Detect faces in each frame while hasFrame(videoFileReader) % read video frame videoFrame = readFrame(videoFileReader); % process frame bbox = faceDetector(videoFrame); videoFrame = insertShape(videoFrame, 'Rectangle', bbox); % Display video frame to screen depVideoPlayer(videoFrame); % Write frame to final video file writeVideo(myVideo, videoFrame); pause(1/videoFileReader.FrameRate); end close(myVideo)
You can download this code on MATLAB Central.
Video Processing Algorithms for Computer Vision
MATLAB algorithms that use temporal correlation for video processing are based on the concept of “state,” the idea that the algorithm is working on a current video frame but also uses previous frames to determine its output. This is crucial for object tracking algorithms, which rely on prior information to inform future action. A common example of tracking is the KLT algorithm, which tracks individual points in an object to keep track of an object’s location.
Developers of video processing algorithms can also use the vision-specific algorithms in Computer Vision System Toolbox™. The algorithms let you read and view high-resolution videos in a rapid and memory-efficient way. The toolbox also includes algorithms for 3D point cloud processing, stereo vision, object detection, tracking and recognition, and other applications.