Problem with Rectify stereo images

I want to do 3D reconstrucion using the app of Stereo Camera Calibrator, I'm using the same model for the cameras and the set up of them is pretty good I can't bring them closer. I have 20 pairs of them my error seems fine, but when I use the code to rectify, doesn't look good, this is my code.(I'm using Matlab R2023a)

clear all; close all; clc 
load stereoParametersD.mat
% Leer las imágenes
L = imread("image19_cam1.png");
R = imread("image19_cam2.png");
% Mostrar las imágenes sin rectificar
figure 
imshowpair(L, R)
title("Before rectification")
% Rectificar las imágenes
[OutL, OutR] = rectifyStereoImages(L, R, stereoParams);
% Mostrar las imágenes rectificadas
figure
imshowpair(OutL, OutR)
title("Rectified")
% Convertir a escala de grises
OutR_gray = im2gray(OutR); 
OutL_gray = im2gray(OutL);
% Calcular la disparidad
dispar = disparity(OutL_gray, OutR_gray);
% Mostrar la disparidad
figure 
imshow(dispar)
title("Disparity")
% Mostrar la disparidad con contraste modificado
figure 
imshow(dispar, [0, 64])
title("Disparidad entre las imágenes modificando el contraste")
colormap jet
colorbar 
% Crear y mostrar el anaglifo estereoscópico
J1_valid = OutL_gray;  % Asegurarse de que las imágenes estén definidas
J2_valid = OutR_gray;
figure
imshow(stereoAnaglyph(J1_valid, J2_valid))

Answer 1

[Umar]

Hi @Kenny Manjarrez Gracia ,

Your code appears to follow the correct sequence for rectifying stereo images and calculating disparity. However, there are several aspects you might want to consider for improving the quality of your rectified images and the resulting disparity map.

Calibration Parameters

Ensure that the stereoParams variable loaded from stereoParametersD.mat is accurate. It should contain intrinsic and extrinsic parameters that reflect the actual physical arrangement of your cameras. You can visualize these parameters by using showReprojectionErrors(stereoParams) to ensure that they align well with your calibration patterns.

Image Quality

Make sure that the images you are loading (e.g., image19_cam1.png and image19_cam2.png) are of high quality with sufficient texture for disparity calculation. Low-contrast or blurry images can lead to poor rectification and disparity maps.Consider preprocessing your images (e.g., applying histogram equalization) to enhance their features before rectification.

Rectification Process

The rectifyStereoImages function should be used correctly, but double-check that stereoParams indeed matches your camera setup. If your images are not aligned properly after rectification, it may indicate an issue with the calibration or the rectification parameters.

Disparity Calculation

The disparity function can be sensitive to image quality and calibration accuracy. If you are getting unexpected results, experiment with different disparity algorithms available in MATLAB, such as disparityBM (Block Matching) or disparitySGM (Semi-Global Matching). Example modification:

     disparityMap = disparityBM(OutL_gray, OutR_gray);

For more information on disparityBM function, please refer to

https://www.mathworks.com/help/vision/ref/disparitybm.html

Visualization

• When visualizing the disparity map, ensure that you are interpreting the values correctly. The display range [0, 64] may not be suitable for all datasets. Consider adjusting this range based on the actual range of disparity values calculated.

• Use imshow(dispar, []) to auto-scale the display based on minimum and maximum values.

Here are some additional insights I would like to share with you.

Multiple Image Pairs: Since you have 20 pairs of images, consider averaging or refining your disparity maps across multiple pairs to improve robustness.

Error Analysis: If possible, analyze any discrepancies in results across different image pairs to identify systematic issues in calibration or environmental factors affecting image capture (lighting conditions, occlusions).

Testing with Known Patterns: If feasible, test your setup with known geometric patterns or structures to verify that your calibration parameters yield expected results.

Hope this helps.

If issues persist after these adjustments, consider revisiting each step methodically or consulting additional resources on stereo vision and image processing techniques.