Can we impose geometry constraints on fisheye lens calibration and rectification?

Assuming that the distorted lines generated by fisheye projection should be straight after rectification, can we use deep neural network to impose explicit geometry constraints onto processes of the fisheye lens calibration and the distorted image rectification?

[cr]

Can it not be done with lens calibration?

[xingxingcui]

In cases where calibration is not easy/unknown, such as when there is only one distorted image and the a priori linear geometric relationship in the image is known and can be corrected without the need to obtain an internal camera intrinsic matrix.

For example, how do you obtain a corrected image based on this a priori information alone when the wall edge and parking space lines away from the car perimeter should be straight?

[xingxingcui]

It can use chatGPT to get this answer, follow is work?

----------------------

In this blog post, I will show you a simple and effective method to remove distortion from any image without any prior knowledge. The method is based on a novel algorithm that can compute a mapping table for distortion removal from any aberrated image. The algorithm does not require any camera internal reference or aberration coefficients. It only needs the input image and some user-defined parameters.

The algorithm works as follows:

1. The input image is divided into small patches of equal size.

2. For each patch, the algorithm computes a local homography that maps the patch to a rectangular grid.

3. The local homographies are refined using a global optimization method that minimizes the distortion error and preserves the continuity and smoothness of the mapping.

4. The final mapping table is obtained by interpolating the local homographies over the whole image domain.

The algorithm can handle various types of distortion, such as radial, tangential, pincushion, barrel, and perspective distortion. It can also handle images with multiple distortions or complex distortions that cannot be modeled by simple equations.

The algorithm has several advantages over existing methods:

- It does not require any prior knowledge about the camera or the lens that captured the image.

- It does not require any reference object or pattern in the image.

- It can handle any type of distortion or combination of distortions.

- It is fast and robust to noise and occlusion.

[Bjorn Gustavsson]

Sure, find the report or article where this method was presented, and implement it from there.

Answer 1

[xingxingcui]

Last year I temporarily did not find the use of deep learning to correct an irregularly distorted image, but used a custom set of laws to achieve a de-distorted image, and now open-source it( following link) in the hope that it can help people with similar needs!

https://ww2.mathworks.cn/matlabcentral/fileexchange/171764-single-image-undistort