Create Multi-Sensor System from Pairwise Calibrations Using MUN-FRL Dataset

R2026b

Step 4 of 5 in Calibrate a Multi-Sensor System Using MUN-FRL Dataset

• 3
• 4
• 5

This example shows how to chain pairwise extrinsic calibration results into a unified multi-sensor coordinate system using the MUN-FRL dataset [1]. The result is a multiSensorParameters object that represents the spatial configuration of all the sensors in a common reference frame.

This example is part of the Calibrate a Multi-Sensor System Using MUN-FRL Dataset series. It uses the calibration results estimated in the previous examples:

Pairwise Calibration Results	Example
Camera intrinsics and IMU parameters	Calibrate Multi-Sensor Intrinsics Using MUN-FRL Dataset
Lidar-to-camera extrinsics	Calibrate Lidar-Camera Extrinsics Using MUN-FRL Dataset
Camera-to-IMU extrinsics	Calibrate Camera-IMU Extrinsics Using MUN-FRL Dataset

Load Pairwise Calibration Results

Load the calibrated intrinsics and pairwise extrinsic results.

% Load calibrated intrinsics
intrinsicsFront = load("intrinsicsFront.mat").intrinsicsFront;
intrinsicsDown = load("intrinsicsDown.mat").intrinsicsDown;
imuParams = load("imuIntrinsics.mat").imuParams;

% Load calibrated extrinsics
% Camera-to-IMU transforms estimated by estimateCameraIMUTransform.
camFrontToIMU = load("IMUtoCameraFront.mat").tformFront;
camDownToIMU  = load("IMUtoCameraDown.mat").tformDown;

% Lidar-to-camera transform estimated by Lidar Camera Calibrator.
lidarToCamDown = load("lidarToDownCameraResult.mat").tform;

Build Multi-Sensor System

Construct the multiSensorParameters object with the IMU as the reference frame. Add each sensor by specifying the transform from the sensor frame to its parent frame.

% Construct the multiSensorParameters object. The addSensor function
% expects the transform from the new sensor's frame to its parent's frame.
msp = multiSensorParameters(ReferenceFrame="IMU");
msp = addSensor(msp, "IMU", "IMU", rigidtform3d(), Intrinsics=imuParams);
msp = addSensor(msp, "CameraFront", "camera", camFrontToIMU, "IMU", Intrinsics=intrinsicsFront);
msp = addSensor(msp, "CameraDown", "camera", camDownToIMU, "IMU", Intrinsics=intrinsicsDown);
msp = addSensor(msp, "Lidar", "lidar", lidarToCamDown, "CameraDown");

The lidar is added with CameraDown as its parent, not the IMU directly. The multiSensorParameters object automatically composes the lidar-to-camera and camera-to-IMU transforms to express the lidar in the IMU reference frame.

Visualize Calibrated System

Plot the calibrated multi-sensor system to verify the spatial arrangement of all sensors. The plot shows the coordinate frame of each sensor positioned and oriented according to its estimated extrinsic transform relative to the IMU.

% Plot the calibrated multi-sensor system
ax = plot(msp, SensorSize=0.05, FrameSize=0.1, ShowFrameAxisLabels=false);

% Reverse Z-axis to match the IMU body frame convention where Z points down
set(ax, ZDir="reverse");

To further validate the calibration results by fusing different sensor data, see the Validate Calibration by Building a Colorized 3-D Map Using MUN-FRL Dataset example.

References

[1] Thalagala, Ravindu G., Oscar De Silva, Awantha Jayasiri, Arthur Gubbels, George KI Mann, and Raymond G. Gosine. "MUN-FRL: A visual-inertial-LiDAR dataset for aerial autonomous navigation and mapping." The International Journal of Robotics Research 43, no. 12 (2024): 1853-1866.