Track Objects in a Parking Lot Using TI mmWave Radar

Since R2023b

This example shows you how to use data captured using the Texas Instruments (TI) mmWave radar for tracking objects in a parking lot. You learn how to use a processing chain to process radar point clouds and estimate the number of people in the scene as well as their states such as position and velocity.

Scenario and Data

In this example, you use a pre-recorded data set captured by a TI mmWave radar AWR6843 mounted on a stationary host vehicle. At each instant, the radar data includes a point cloud and a timestamp in seconds. The data is recorded using the mmWaveRadar class. The point cloud is described using a cell array of objectDetectionDelay objects, where each objectDetection represents a point. The Measurement property of the objectDetection object in this example represents the azimuth (deg), range (m), and range-rate (m/s) of the detected point. The MeasurementNoise property describes the uncertainty (noise statistics) in the measurement. By default, the mmWaveRadar class assumes that standard deviation of the noise are same as the radar resolutions in azimuth, range, and range-rate dimensions. In addition, each objectDetection also contains the mounting information of the radar. In this example, the radar is assumed to be at the origin with its orientation aligned with the X-axis. For more information on how to represent mounting information using objectDetection, refer to the Convert Detections to objectDetection Format example.

The radar data is recorded using the configuration file HelperParkingLotConfiguration.cfg, attached with this example. It can be downloaded from the following location.

https://ssd.mathworks.com/supportfiles/timmwaveradar/data/ParkingLotTIRadarData.zip

You can visualize the scenario used to record the data by playing the video file, ParkingLotTIRadarVideo.mp4, attached with this example. To record new data using the same configuration file, you can use the following code.

% Configuration file
cfgFile = fullfile(pwd,'\HelperParkingLotConfiguration.cfg');

% Create sensor 
rdr = mmWaveRadar("TI AWR6843ISK",...
    ConfigFile=cfgFile,...
    ReadMode="oldest",...
    DetectionCoordinates="sensor spherical",...
    EnableDopplerGroups=false,...
    EnableRangeGroups=false,...
    RemoveStaticClutter=false);

% Allocate memory
detectionLog = cell(1,0); 
timeLog = zeros(1,0); 

% Duration of recording in secondns
stopTime = 50; 

% Record until stop time
while isempty(timeLog) || timeLog(end) < stopTime
    [detectionLog{end+1},timeLog(end+1)] = rdr();
end

Download the data files into the current working directory. If you want to place the files in a different folder, change the directory name.

% Load data if unavailable. 
if ~exist('ParkingLotTIRadarData','file')
    dataURL = "https://ssd.mathworks.com/supportfiles/timmwaveradar/data/ParkingLotTIRadarData.zip";
    datasetFolder = pwd;
    unzip(dataURL, datasetFolder);
end

% Load radar data
load(fullfile(pwd,'ParkingLotTIRadarData','ParkingLotTIRadarData.mat'),'detectionLog','timeLog');

% Load video data
vidReader = VideoReader(fullfile(pwd,'ParkingLotTIRadarData','ParkingLotRadarReferenceVideo.mp4'));

To visualize the radar data, the estimates tracks, and the intermediate steps of processing chain used in this example, you use the HelperTIRadarTrackingDisplay class attached with this example. The image below shows a sample of the radar point cloud and the camera reference image at a time instant.

display = HelperTIRadarTrackingDisplay;
refImage = read(vidReader, 350);
display(detectionLog{350}, {}, objectTrack.empty(0,1), refImage);

Radar Processing Chain

You use the following processing chain to process raw radar point clouds into tracks. The DBSCAN algorithm groups the point cloud from the same object into a cluster. The centroid estimation algorithm converts the point cloud from the same object into a single point representation. The centroid is then tracked by using a joint integrated probabilistic data association (JIPDA) tracker. The tracks represent the objects as points in the environment and provide position and velocity estimates.

To describe the noise statistics of the measurements for a point object model, you assume that the azimuth accuracy is about 6 degrees and the range accuracy is about 0.6 meters. These accuracies intend to describe the error between true object measurement and observed measurements. You update the recorded data log to provide the accuracy information using the MeasurementNoise property of the objectDetection. The range-rate accuracy is assumed to be the same as resolution of the radar.

for k = 1:numel(detectionLog)
    detections = detectionLog{k};
    for j = 1:numel(detections)
        detections{j}.MeasurementNoise(1,1) = 36; % 6^2 deg^2
        detections{j}.MeasurementNoise(2,2) = 0.36; % 0.6^2 m^2
    end
    detectionLog{k} = detections;
end

Next, you specify the parameters of the DBSCAN algorithm. You use the Mahalanobis distance to cluster measurements into groups using the DBSCAN algorithm. You choose the epsilon threshold for DBSCAN as 2 and the minimum number of points per cluster as 1. Later, you use the partitionDetections function to partition the detections at each time instant into groups.

epsilon = 2;
minNumPts = 1;

You will obtain the centroid of each group using the mergeDetections function, which merges all the measurements and their noise statistics in a group using a standard Gaussian mixture merging algorithm.

Next, you create a JIPDA tracker using the trackerJPDA System object™. You set the TrackLogic property to enable the 'Integrated' logic. For the filter in the tracker, you use an extended Kalman filter (trackingEKF) with a 2-D constant velocity motion model as defined in the initParkingLotFilter supporting function. The initParkingLotFilter function defines a low-uncertainty in cross range-rate when initializing the filter and a low process noise to model objects that do not maneuver rapidly.

tracker = trackerJPDA(TrackLogic="Integrated");
tracker.FilterInitializationFcn = @initParkingLotFilter;

Further, you specify clutter density, birth rate, and detection probability of the radar in the tracker. In this example, you use prior knowledge to calculate these numbers. The volume of the sensor in measurement space can be calculated using the measurement limits in each dimension. You assume that on average there 5 false alarms per step and 1 new target appearing in the field of view every 10 steps.

% Volume of measurement space
azSpan = 60;
rSpan = 25;
dopplerSpan = 5;
V = azSpan*rSpan*dopplerSpan;

% Number of false alarms per step
nFalse = 5;

% Number of new targets per step
nNew = 0.1;

% Probability of detecting the object
Pd = 0.9;

tracker.ClutterDensity = nFalse/V;
tracker.NewTargetDensity = nNew/V;
tracker.DetectionProbability = Pd;

Lastly, you configure the track management properties of the tracker to allow the tracker to discriminate between false and true objects and to create and delete tracks when they enter and exit the field of view, respectively.

% Confirm a track when with more than 90 percent
% probability of existence
tracker.ConfirmationThreshold = 0.9; 

% Delete a track with less than 0.0001
% probability of existence
tracker.DeletionThreshold = 1e-4;

Now, you loop over the recorded data and run the processing chain to get estimates about the number of objects and their states. You visualize the point cloud, the clustered centroid measurements, as well as the position and velocity of each track.

tracks = objectTrack.empty(0,1);

for k = 1:numel(detectionLog)
    % Timestamp
    time = timeLog(k);

    % Radar at current time stamp
    detections = detectionLog{k};
    
    % Camera image
    refImage = read(vidReader, k);
    
    % Cluster detections
    if isempty(detections)
        clusters = zeros(0,1,'uint32');
    else
        clusters = partitionDetections(detections,epsilon,minNumPts,'Algorithm','DBSCAN');
    end

    % Centroid estimation
    clusteredDets = mergeDetections(detections, clusters);
    
    % Track centroid returns
    if isLocked(tracker) || ~isempty(clusteredDets)
        tracks = tracker(clusteredDets, time);
    end

    % Update display
    display(detections, clusteredDets, tracks, refImage);
end

Results

In the display, you can visualize the raw data as well as results produced by the processing chain. Two dotted black reference lines are plotted on the camera image and the radar coverage to illustrate the projection from radar coordinates to camera pixels. Notice that both static and dynamic obstacles around the radar are tracked, their identities are preserved across time, and their estimated positions are close to the point clouds returned by the radar. The tracker did lose the track of the parked vehicle when it was occluded by the crossing vehicle for a long time. This is because the radar did not observe any returns from the vehicle and the tracker expected the radar to detect the vehicle with a 90% probability. However, as soon as the radar sees the vehicle again, a new track is established on the vehicle by the tracker.

Summary

In this example, you learned how to use a processing chain which uses returns from a radar in parking lot situation to track objects around the host vehicle.

Supporting Functions

function filter = initParkingLotFilter(detection)
% Create 3-D filter first
filter3D = initcvekf(detection);

% Create 2-D filter from the 3-D
state = filter3D.State(1:4);
stateCov = filter3D.StateCovariance(1:4,1:4);

% Reduce uncertainty in cross range-rate to 5 m/s
velCov = stateCov([2 4],[2 4]);
[v, d] = eig(velCov);
D = diag(d);
D(2) = 1;
stateCov([2 4],[2 4]) = v*diag(D)*v';

% Process noise in a slowly changing environment
Q = 0.25*eye(2);

filter = trackingEKF(State = state,...
    StateCovariance = stateCov,...
    StateTransitionFcn = @constvel,...
    StateTransitionJacobianFcn = @constveljac,...
    HasAdditiveProcessNoise = false,...
    MeasurementFcn = @cvmeas,...
    MeasurementJacobianFcn = @cvmeasjac,...
    ProcessNoise = Q,...
    MeasurementNoise = detection.MeasurementNoise);

end