Camera calibration usually adopts the aperture imaging model, and the ideal aperture model is a linear model. However, due to the lens distortion and other reasons, the linear model usually needs to add some internal parameters to become a nonlinear model. This paper deals with the principles and solutions of this phenomenon.

1. Lens distortion phenomenon introduction

The process of camera imaging is essentially the transformation of coordinate system. Firstly, the points in the space are converted from the “world coordinate system” to the “camera coordinate system”, and then they are projected to the imaging plane (image physical coordinate system), and finally the data on the imaging plane are converted to the image pixel coordinate system. However, due to the deviation of lens manufacturing accuracy and assembly process, distortion will be introduced, resulting in the distortion of the original image. Lens distortion can be divided into radial distortion and tangential distortion.

(1). Radial distortion

Radial distortion is the distortion distributed along the radius of the lens. The reason is that the light is more curved at the center of the lens than near the center. This distortion is more obvious in ordinary cheap lenses.

The distortion in the center of the imager’s optical axis is 0, which moves toward the edge along the lens radius, and the distortion becomes more and more serious. Mathematical model can be used to the distortion of the main points (principle point) of the Taylor series expansion around the first few items, usually use the first two, namely, k1 and k2, for lens distortion is very big, such as fish eye lens, can increase the use of the third k3 to describe, imager on some point according to its position in the distribution of the radial direction, The adjustment formula is:

(x0, y0) in the formula is distortion in the original location on the imager, (x, y) is a serious distortion after the new location, and below the point of the different distance is the distance light heart after radial lens distortion point offset sketch, the farther the distance light heart, the greater the radial displacement, said the distortion is, the greater the near the heart of light, almost no deviation.

(2). Tangential distortion

Tangential distortion is caused when the lens itself is not parallel to the camera sensor plane (imaging plane) or image plane, which is mostly caused by the mounting deviation of the lens when it is pasted to the lens module. The distortion model can be described by two additional parameters, P1 and P2:

In general, the distortion displacement is symmetric with the bottom left-top right line, indicating that the lens has an Angle of rotation perpendicular to that direction.

There are five distortion parameters in the radial distortion model and tangential distortion model. In Opencv, they are arranged into a 5*1 matrix, including K1, K2, P1, P2 and K3 in turn, which is often defined as the form of Mat matrix, for example, Mat distCoeffs=Mat (1,5, CV_32FC1, Scalar::all (0)); These five parameters are the five camera distortion coefficients that need to be determined in camera calibration. After obtaining these 5 parameters, the image distortion caused by lens distortion can be corrected.