opencv - The functions distortPoints and undistortPoints are not inverse of each other

Question

I am trying to understand the camera calibration/3D reconstruction and facing a strange behavior of cv::fisheye::distort/undistortPoints functions. I would expect that the fisheye model moves a point along the ray connecting it to the principal point (cx, cy), however, it doesn't. Moreover, the functions cv::fisheye::distortPoints and cv::fisheye::undistortPoints are not inverse of each other (as one would expect).

The following code creates a camera matrix with distortion coefficients and undistorts and then distorts back an arbitrary point. The values for the camera intrinsics and distortion coefficients were taken from a public dataset.

cv::Mat camera_matrix = cv::Mat::zeros(3,3,CV_64F);
camera_matrix.at<double>(0,0) = 190.9784;
camera_matrix.at<double>(1,1) = 190.9733;
camera_matrix.at<double>(0,2) = 254.9317;
camera_matrix.at<double>(1,2) = 256.8974;
camera_matrix.at<double>(2,2) = 1;

std::cout << "Camera matrix: 
" << camera_matrix << "
" <<std::endl;

cv::Mat distortion_coefficients(4,1,CV_64F);
distortion_coefficients.at<double>(0) = 0.003482;
distortion_coefficients.at<double>(1) = 0.000715;
distortion_coefficients.at<double>(2) = -0.0020532;
distortion_coefficients.at<double>(3) = 0.000203;

std::cout << "Distortion coefficients
"<< distortion_coefficients<< "
" << std::endl;

cv::Mat original_point(1,1,CV_64FC2);
original_point.at<cv::Point2d>(0).x= 7.7;
original_point.at<cv::Point2d>(0).y= 9.9;
cv::Mat undistorted, distorted;
cv::fisheye::undistortPoints(original_point, undistorted, camera_matrix, 
            distortion_coefficients, cv::Mat(), camera_matrix);
cv::fisheye::distortPoints(undistorted, distorted, camera_matrix, distortion_coefficients);

std:: cout << "Original point: " << original_point.at<cv::Point2d>(0).x << " " << original_point.at<cv::Point2d>(0).y << std::endl;
std:: cout << "Undistorted point: " << undistorted.at<cv::Point2d>(0).x << " " << undistorted.at<cv::Point2d>(0).y<< std::endl;
std:: cout << "Distorted point: " << distorted.at<cv::Point2d>(0).x << " " << distorted.at<cv::Point2d>(0).y;

The result of this is

Camera matrix: 
[190.9784, 0, 254.9317;
 0, 190.9733, 256.8974;
 0, 0, 1]

Distortion coefficients
[0.003482;
 0.000715;
 -0.0020532;
 0.000203]

Original point: 7.7 9.9
Undistorted point: 8905.69 8899.45
Distorted point: 464.919 466.732

The point that is near the top left corner is moved far bottom right.

Is this a bug or I do not understand something?

cv::fisheye::undistortImage is working on the dataset images - the curves are turned back into lines.

What am I missing?

question from:https://stackoverflow.com/questions/66059554/the-functions-distortpoints-and-undistortpoints-are-not-inverse-of-each-other

Answer 1

Waitting for answers