Main Content

Calibration and Sensor Fusion

Interactively perform lidar-camera calibration, estimate transformation matrix, and fuse data from multiple sensors

Most modern autonomous systems in applications such as manufacturing, transportation, and construction, employ multiple sensors. Sensor Fusion is the process of bringing together data from multiple sensors, such as radar sensors, lidar sensors, and cameras. The fused data enables greater accuracy because it leverages the strengths of each sensor to overcome the limitations of the others.

To understand and correlate the data from individual sensors, you must develop a geometric correspondence between them. Calibration is the process of developing this correspondence. Use Lidar Toolbox™ functions to perform lidar-camera calibration. To get started, see What Is Lidar-Camera Calibration?

You can also interactively calibrate the sensors by using the Lidar Camera Calibrator app. For more information, see Get Started with Lidar Camera Calibrator.

Lidar Toolbox also supports downstream workflows such as projecting lidar points on images, fusing color information in lidar point clouds, and transferring bounding boxes from camera data to lidar data.

Multi-sensor system


Lidar Camera CalibratorInteractively estimate rigid transformation between lidar sensor and camera


expand all

estimateCheckerboardCorners3dEstimate world frame coordinates of checkerboard corner points in image
detectRectangularPlanePointsDetect rectangular plane of specified dimensions in point cloud
estimateLidarCameraTransformEstimate rigid transformation from lidar sensor to camera
projectLidarPointsOnImageProject lidar point cloud data onto image coordinate frame
fuseCameraToLidarFuse image information to lidar point cloud
bboxCameraToLidarEstimate 3-D bounding boxes in point cloud from 2-D bounding boxes in image
bboxLidarToCameraEstimate 2-D bounding box in camera frame using 3-D bounding box in lidar frame