Unexpected image resolution loss after rectifying verged / angled cameras for stereo depth

I have a stereo rig setup on a rigid crossbar. Each camera has a resolution of 540(H) x 720(W).

After rectifying the images rectifyStereoImages() I lose some resolution, which is to be expected.

However, as I verge the cameras inwards (even just a few degrees) the resolution of the image width is severly reduced.

• With cameras pointing in the same direction (camera2 rotation vector of [0, 0, 0]) the rectified image resolution = 508 x 700 (stereoParams.mat attached)
• With cameras verged inwards at only 8 degrees (camera2 rotation vector of [0, 0.14, 0]) the rectified image resolution = 508 x 469 (stereoParamsVerged.mat attached)

My understanding is that rectification aligns the camera images along the same horizontal plane. If true, then why would verging the camera angle inwards cause any change in the rectified image resolution? Afterall the images still lie on the same horizontal place before/after verging.

Would anyone be able to shed some light on why this obeserved rectified image resolution (width) loss is happening after verging and whether this is expected behaviour?

Also, any advice on how to alevite this would be hugely appreciated.

I have attached example left / right images if useful.

Answer 1

[Aishwarya]

Hello,

Based on my understanding of the question, you have angled the stereo cameras inward by 8 degrees and then rectified the images using the "rectifyStereoImages" function. You've noticed that the image width has significantly decreased after rectification and are seeking to understand the reason for this change.

I would like to provide some insights that might help with the query:

• The resolution of the rectified images appears reduced because the "rectifyStereoImages" function, by default, crops the output to the size of the largest common rectangle that contains valid pixels.
• To obtain rectified images that retain the full pixel data from both stereo images, you can adjust the "OutputView" parameter of the "rectifyStereoImages" function to "full".
• I have prepared code to analyze the data you provided. In this code, the rectified stereo images are displayed in red-cyan anaglyph using the “stereoAnaglyph” function :

        I1 = imread("Verged Left.png");
        I2 = rgb2gray(imread("Verged Right.png"));
        load stereoParamsVerged.mat
        [J1_full,J2_full] = rectifyStereoImages(I1,I2,stereo_cameras_params,OutputView='full');
        figure; 
        imshow(stereoAnaglyph(J1_full,J2_full));

The anaglyph image demonstrates that the common area between the left and right images indeed has a reduced width, which corresponds to the output you observed.

• One possible reason behind the reduced width in rectified images could be because verging the cameras introduces rotational distortion to the already unrectified images. For example, if the initial setup was that images were captured from two parallel cameras, then the rectified images can be obtained by only resolving the translation misalignment. But when you verge both the cameras by certain degrees, the image planes of both the cameras are rotated hence introducing more distortion leading to less area of common region when they are rectified.

You can refer to the below documentation for more information about the functions used:

• “rectifyStereoImages” function: https://www.mathworks.com/help/vision/ref/rectifystereoimages.html
• “stereoAnaglyph” function: https://www.mathworks.com/help/vision/ref/stereoanaglyph.html

I hope this clarifies your query.