Stereo vision 3d scene reconstruction is flat

Hi, i have a little problem here, i calibrated my stereo cameras, with rep. error 0.8, i picked 2 images, did disparity map, 3d scene reconstruction but the result is for some reason flat, here is my code, and the result.

clc
clear all
close all
load('C:\Users\PC\Documents\KUKA-3D-PROJEKT\P\stereoParams.mat');
showExtrinsics(stereoParams);
I1 = imread('C:\Users\PC\Documents\KUKA-3D-PROJEKT\picC3\Frame01.png');
I2 = imread('C:\Users\PC\Documents\KUKA-3D-PROJEKT\picC3\Frame101.png');
A = stereoAnaglyph(I1,I2);
figure
imshow(A)
J1 = rgb2gray(I1);
J2 = rgb2gray(I2);
figure 
[J1, J2] = rectifyStereoImages(I1,I2,stereoParams);
disparityRange = [0 128];
disparityMap = disparityBM(rgb2gray(J1),rgb2gray(J2),'DisparityRange',disparityRange,'UniquenessThreshold',2)
figure
imshow(disparityMap,disparityRange)
title('Disparity Map')
colormap jet
colorbar
xyzPoints = reconstructScene(disparityMap,stereoParams);
xyzPoints = xyzPoints ./1000;
ptCloud = pointCloud(xyzPoints, 'Color', J2);

ptCloud = pointCloud(xyzPoints, 'Color', J1);
player3D = pcplayer([-4, 4], [-4, 4], [0, 10], 'VerticalAxis', 'x', ...
    'VerticalAxisDir', 'down');

view(player3D, ptCloud);
xyzPoints = xyzPoints ./1000;
ptClourd = pointCloud(xyzPoints, 'Color', J2);
player3d = pcplayer([-4, 4], [-4, 4], [0, 10], 'VerticalAxis', 'y', ...
    'VerticalAxisDir', 'up');

As u can see the board and the box is flat, but irl the box is 22 mm.

What am i doing wrong here ?

[Prabhan Purwar]

Plss attach stereoParams.mat, Frame01.png and Frame101.png files to recreate the 3D map.

Answer 1

[Prabhan Purwar]

Hey,

Attached code seems good, although with a minor problem like ptCloud is overwritten. It seems like input images or stereoParams.mat files are not able to meet the code requirements.

Please have a look over the following code, it makes use of MATLAB default images and stereoParams as inputs and gives 3D Reconstructed scene as output.

load('webcamsSceneReconstruction.mat'); 
  
%Read in the stereo pair of images. 
I1 = imread('sceneReconstructionLeft.jpg'); 
I2 = imread('sceneReconstructionRight.jpg'); 
  
%Rectify the images. 
[J1, J2] = rectifyStereoImages(I1,I2,stereoParams); 
  
%Display the images after rectification. 
figure  
imshow(cat(3,J1(:,:,1),J2(:,:,2:3)),'InitialMagnification',50); 
  
%Compute the disparity. 
disparityMap = disparitySGM(rgb2gray(J1),rgb2gray(J2)); 
figure  
imshow(disparityMap,[0,64],'InitialMagnification',50); 
  
%Reconstruct the 3-D world coordinates of points corresponding to each pixel from the disparity map. 
xyzPoints = reconstructScene(disparityMap,stereoParams); 
xyzPoints = xyzPoints ./1000; 
  
%J1 = cat(3,J1,J1,J1); 
ptCloud = pointCloud(xyzPoints,'Color', J1); 
  
player3D = pcplayer([-4, 4], [-4, 4], [0, 10], 'VerticalAxis', 'x', ... 
    'VerticalAxisDir', 'down'); 
%pcshow(ptCloud) 
view(player3D, ptCloud); 

Outputs:

Image after rectification.

Disparity map

3D Reconstructed map

Please go through the following links for more deeper understanding of the code.

Links:

• https://www.mathworks.com/help/vision/ref/stereoparameters.html (stereoParameters)
• https://www.mathworks.com/help/vision/ref/reconstructscene.html (reconstructScene)
• https://www.mathworks.com/help/vision/examples/depth-estimation-from-stereo-video.html (Depth Estimation from Stereo Video)

[Michal Vasat]

I'm using disparityBM not SGM, should i try the sgm one ?

like, the picture in point cloud is almost perfect, the only problem is that its flat and i need to gain height of the box irl in data.

even if i tried the attach. code u gave me, it is flat, something is wrong, could it be the calibration ? ive been doing the calibration on the same table as the one with the box, so the height is perfectly right, should i try something else ?

[Prabhan Purwar]

Hi,

I have already checked my code with SGM, it was working fine. Most probably, there is some mistake while taking stereo images or calibration.

[Michal Vasat]

Im not sure, should my cameras look straight down or with some angle?.

[Michal Vasat]

i've tried this calibration session rn, still the same problem.

[Maryam]

hi

where i can find the MATLAB default images and stereoParams that you used?