Create a projective homography matrix with pitch/roll/yaw

Hi,

I am working with the imwarp() function where I can put in a picture and a 3x3 projective homography matrix. I got this matrix from the GeometricTransformEstimator but now I want to create such a matrix myself by using a pitch/roll/yaw angle.

I tried the matlab function angle2dcm(yaw, pitch, roll) which gives me a rotation matrix but it looks like it is not the same as a homography matrix. Is it even possible to do what I want?

Best Regards

[Matt J]

Every 3x3 non-singular matrix (including pure rotation matrices) defines a projective homography. What don't you like about the result you got?

[Patrick]

Hi,

the normal rotation, clockwise/counterclockwise, which seems to be yaw here works fine. Also scale and translation don't seem to be a problem. But I am more interested in the other 2 rotations. If I put in something like 0.001 for pitch and roll it seems to be doing what I want. At least a part of it.

Attached is a picture of the coordinates of the edges of the picture (triangles) and of the transformed picture(circles) with pitch=0.001. Values >0.1 just produce crap. I expected the left dots to come closer together in the y direction because it is further away and the right dots are further away from each other because it is closer to the camera. And I expected the left dots to move in the +x direction because the picture is not that wide anymore. But the right dots should have moved in the -x direction and not +x!?

But maybe I am just missing something simple here?

[Matt J]

I still don't think you've defined what you're trying to compute. You mean you have 2 cameras separated by a known 3D rotation about the world origin? And you're trying to compute the homography relating the images of points from those cameras? You would need the camera calibration matrix for that.

[Patrick]

Yes you are right. I think you could describe it like that. I have the camera calibration matrix of my camera but I am not sure where to put it. Can you tell me the name of the thing I am trying to do? Maybe then I can find out how it works.

Answer 1

[David Young]

[ Edited: example changed and transpose of matrix in call to projective2d() inserted. See comments.]

It might be worth experimenting with the code below, which uses various of my functions. These are attached.

Possible stumbling blocks are connected with the direction of view of the camera (along the z-axis in the code below), and the default behaviour of imwarp(), which crops the output to the new corner positions of the image. This code compares the result of doing that with the result of keeping the transformed image in the same coordinates as the original image (using imwarp_same()), which allows the effects of the rotation to be seen more clearly.

By fiddling with the values of yaw, pitch, roll and the camera position vector you should be able to get a fix on what is going on and get behaviour that you expect.

example image

img = imread('peppers.png');

% Set angles here. Start with two of them set to zero and third set quite
% small to see the effect.
yaw = 0;  
pitch = 0; 
roll = -0.2;

% Set camera position here. To understand the effect of angles, move it
% along the z-axis and y-axis so it is above the centre of the top edge of
% the picture. Note that as cammoves is true in call to homography_matrix,
% the z-axis movement needs to be negative.
% The camera is looking along the z-axis.
campos = [size(img,2)/2; 0; -200];

% Construct a rotation matrix. My sign convention is opposite to
% angle2dcm, but I don't have the aerospace toolbox. Computed rmat agrees
% with example in angle2dcm documentation.
yawmat = rotation_matrix([0 0 1], -yaw);       % about z
pitchmat = rotation_matrix([0 1 0], -pitch);   % about y
rollmat = rotation_matrix([1 0 0], -roll);     % about x

rmat = rollmat * pitchmat * yawmat;   % zyx order - default for angle2dcm

% construct homography matrix
F = 100;    % focal length
hmat = homography_matrix([], rmat, campos, F, true);

% construct projective2d object
tform = projective2d(hmat.');

% display original image and results
subplot(1,3,1);
imshow(img);

% show image warped and cropped
imw_crop = imwarp(img, tform);
subplot(1,3,2);
imshow(imw_crop);

% show image warped in same coordinate system as original
imw_same = imwarp_same(img, tform);
subplot(1,3,3);
imshow(imw_same); shg;

[Patrick]

Hi,

thanks for your files but I had no luck in accomplishing what I wanted to do. I tried something like this:

yaw = 0.1; 
pitch = 0.1;
roll = 0.1;
rotMatFromAngles = angle2dcm( yaw, pitch, roll );

F = 100;

hom = homography_matrix([], rotMatFromAngles, [1; 1; 1] , F, 1);

tformerr = projective2d(hom);
B = imwarp(I1, tformerr);
figure; 
imshow(B); title('Recovered image');

But I never saw an output that looked like a picture.

[David Young]

It is very easy to get lost amongst the coordinate systems and rotations. I have updated my answer with demonstration code that shows rotations being applied to an image. You may be able to take it forward from there.

[Patrick]

Hi,

I am sorry it took me so long for my answer and I still didn't manage to produce something useful. I tried your new code but all it does seems to be scaling the picture. Changing the angles seems to produce the right scale but this perspective effect is missing completely.

For example in the screenshot:

yaw = 0;  
pitch = 0; 
roll = 0.8;

;

[David Young]

In this example, there's more than just scaling - the roll has changed the aspect ratio because you are looking at the scene from an oblique angle. I'm not sure what you feel is still missing.

[Patrick]

Yes, I just meant it looks like scaling in y direction. But I expected something like this:

[David Young]

Sorry about the delay - I missed your comment, and Answers doesn't have a notification mechanism. You are quite right, there was an error in my example. I'd forgotten that projective2d requires the matrix to be transposed. That is, it adopts the convention

[x y 1] = [u v 1] * T

where T is the transform matrix, but I always expect to use

[x y 1]' = T * [u v 1]'

When that is fixed, you do indeed get a result that matches what you expect. I'm editing the code to put in the transpose, and also to make the example demonstrate the perspective effect.

[Patrick]

Thanks a lot, this is exactly what I was looking for

Answer 2

[Image Analyst]

Would the camera calibration capabilities of the Computer Vision System Toolbox help? http://www.mathworks.com/products/computer-vision/features.html#camera-calibration

[Matt J]

Probably. The estimateCameraParameters command can be used to obtain a cameraParameters object. It should then be possible to compute the homography relating 2 camera views.

[Patrick]

Hi,

thanks, I had a look at this but I have no idea how I can feed angles to this thing.

[Image Analyst]

Call them and ask for help. I have not used the camera calibration functionality of that toolbox so I'm not a good resource for that.