用OpenCV+Python对广角(包括鱼眼)镜头进行实时畸变校正

用普通摄像头的标定方式(cv2.calibrateCamera()或cv2.calibrateCameraExtended())去获取摄像头内参(包括畸变参数)已不适用于广角或鱼眼摄像头了。
此文主要针对于大广角摄像头且边缘处畸变较大的摄像头进行标定与畸变校正。
开发语言:Python
OpenCV中关于鱼眼镜头的Python版标定的API源码如下所示:

def calibrate(objectPoints, imagePoints, image_size, K, D, rvecs=None, tvecs=None, flags=None, criteria=None): # real signature unknown; restored from __doc__"""calibrate(objectPoints, imagePoints, image_size, K, D[, rvecs[, tvecs[, flags[, criteria]]]]) -> retval, K, D, rvecs, tvecs.   @brief Performs camera calibaration.   .   @param objectPoints vector of vectors of calibration pattern points in the calibration pattern.   coordinate space..   @param imagePoints vector of vectors of the projections of calibration pattern points..   imagePoints.size() and objectPoints.size() and imagePoints[i].size() must be equal to.   objectPoints[i].size() for each i..   @param image_size Size of the image used only to initialize the intrinsic camera matrix..   @param K Output 3x3 floating-point camera matrix.   \f$A = \vecthreethree{f_x}{0}{c_x}{0}{f_y}{c_y}{0}{0}{1}\f$ . If.   fisheye::CALIB_USE_INTRINSIC_GUESS/ is specified, some or all of fx, fy, cx, cy must be.   initialized before calling the function..   @param D Output vector of distortion coefficients \f$(k_1, k_2, k_3, k_4)\f$..   @param rvecs Output vector of rotation vectors (see Rodrigues ) estimated for each pattern view..   That is, each k-th rotation vector together with the corresponding k-th translation vector (see.   the next output parameter description) brings the calibration pattern from the model coordinate.   space (in which object points are specified) to the world coordinate space, that is, a real.   position of the calibration pattern in the k-th pattern view (k=0.. *M* -1)..   @param tvecs Output vector of translation vectors estimated for each pattern view..   @param flags Different flags that may be zero or a combination of the following values:.   -   **fisheye::CALIB_USE_INTRINSIC_GUESS** cameraMatrix contains valid initial values of.   fx, fy, cx, cy that are optimized further. Otherwise, (cx, cy) is initially set to the image.   center ( imageSize is used), and focal distances are computed in a least-squares fashion..   -   **fisheye::CALIB_RECOMPUTE_EXTRINSIC** Extrinsic will be recomputed after each iteration.   of intrinsic optimization..   -   **fisheye::CALIB_CHECK_COND** The functions will check validity of condition number..   -   **fisheye::CALIB_FIX_SKEW** Skew coefficient (alpha) is set to zero and stay zero..   -   **fisheye::CALIB_FIX_K1..fisheye::CALIB_FIX_K4** Selected distortion coefficients.   are set to zeros and stay zero..   -   **fisheye::CALIB_FIX_PRINCIPAL_POINT** The principal point is not changed during the global.   optimization. It stays at the center or at a different location specified when CALIB_USE_INTRINSIC_GUESS is set too..   @param criteria Termination criteria for the iterative optimization algorithm."""pass

畸变校正的API源码如下所示:

def estimateNewCameraMatrixForUndistortRectify(K, D, image_size, R, P=None, balance=None, new_size=None, fov_scale=None): # real signature unknown; restored from __doc__"""estimateNewCameraMatrixForUndistortRectify(K, D, image_size, R[, P[, balance[, new_size[, fov_scale]]]]) -> P.   @brief Estimates new camera matrix for undistortion or rectification..   .   @param K Camera matrix \f$K = \vecthreethree{f_x}{0}{c_x}{0}{f_y}{c_y}{0}{0}{_1}\f$..   @param image_size.   @param D Input vector of distortion coefficients \f$(k_1, k_2, k_3, k_4)\f$..   @param R Rectification transformation in the object space: 3x3 1-channel, or vector: 3x1/1x3.   1-channel or 1x1 3-channel.   @param P New camera matrix (3x3) or new projection matrix (3x4).   @param balance Sets the new focal length in range between the min focal length and the max focal.   length. Balance is in range of [0, 1]..   @param new_size.   @param fov_scale Divisor for new focal length."""passdef initUndistortRectifyMap(K, D, R, P, size, m1type, map1=None, map2=None): # real signature unknown; restored from __doc__"""initUndistortRectifyMap(K, D, R, P, size, m1type[, map1[, map2]]) -> map1, map2.   @brief Computes undistortion and rectification maps for image transform by cv::remap(). If D is empty zero.   distortion is used, if R or P is empty identity matrixes are used..   .   @param K Camera matrix \f$K = \vecthreethree{f_x}{0}{c_x}{0}{f_y}{c_y}{0}{0}{_1}\f$..   @param D Input vector of distortion coefficients \f$(k_1, k_2, k_3, k_4)\f$..   @param R Rectification transformation in the object space: 3x3 1-channel, or vector: 3x1/1x3.   1-channel or 1x1 3-channel.   @param P New camera matrix (3x3) or new projection matrix (3x4).   @param size Undistorted image size..   @param m1type Type of the first output map that can be CV_32FC1 or CV_16SC2 . See convertMaps().   for details..   @param map1 The first output map..   @param map2 The second output map."""pass

直接上干货:
标定板准备:(此处为9*6的棋盘格)

标定主程序:

程序运行后按空格键拍摄左右摄像头的图片(此处30张)

标定代码:

def calibrate_single(imgNums, CheckerboardSize, Nx_cor, Ny_cor, saveFile=False, saveImages=False):'''单目(普通+广角/鱼眼)摄像头标定:param imgNums: 标定所需样本数,一般在20~40之间.按键盘空格键实时拍摄:param CheckerboardSize: 标定的棋盘格尺寸,必须为整数.(单位:mm或0.1mm):param Nx_cor: 棋盘格横向内角数:param Ny_cor: 棋盘格纵向内角数:param saveFile: 是否保存标定结果,默认不保存.:param saveImages: 是否保存图片,默认不保存.:return mtx: 内参数矩阵.{f_x}{0}{c_x}{0}{f_y}{c_y}{0}{0}{1}:return dist: 畸变系数.(k_1,k_2,p_1,p_2,k_3)'''# 找棋盘格角点(角点精准化迭代过程的终止条件)criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, CheckerboardSize, 1e-6)  # (3,27,1e-6)flags = cv2.CALIB_CB_ADAPTIVE_THRESH + cv2.CALIB_CB_FAST_CHECK + cv2.CALIB_CB_NORMALIZE_IMAGE  # 11flags_fisheye = cv2.fisheye.CALIB_RECOMPUTE_EXTRINSIC + cv2.fisheye.CALIB_CHECK_COND + cv2.fisheye.CALIB_FIX_SKEW  # 14# 世界坐标系中的棋盘格点,例如(0,0,0), (1,0,0), (2,0,0) ....,(8,5,0)objp = np.zeros((1, Nx_cor * Ny_cor, 3), np.float32)objp[0, :, :2] = np.mgrid[0:Nx_cor, 0:Ny_cor].T.reshape(-1, 2)# 储存棋盘格角点的世界坐标和图像坐标对objpoints = []  # 在世界坐标系中的三维点imgpoints = []  # 在图像平面的二维点count = 0  # 用来标志成功检测到的棋盘格画面数量while (True):ret, frame = cap.read()frame = cv2.resize(frame, imageSize)cv2.imshow('frame', frame)if cv2.waitKey(1) & 0xFF == ord(' '):gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)# 寻找棋盘格模板的角点ok, corners = cv2.findChessboardCorners(gray, (Nx_cor, Ny_cor), flags)if count >= imgNums:breakif ok:  # 如果找到，添加目标点，图像点objpoints.append(objp)cv2.cornerSubPix(gray, corners, (5, 5), (-1, -1), criteria)  # 获取更精确的角点位置imgpoints.append(corners)# 将角点在图像上显示cv2.drawChessboardCorners(frame, (Nx_cor, Ny_cor), corners, ok)count += 1if saveImages:cv2.imwrite('../imgs/' + str(count) + '.jpg', frame)print('NO.' + str(count))if cv2.waitKey(1) & 0xFF == ord('q'):breakglobal mtx, dist# 标定. rvec和tvec是在获取了相机内参mtx,dist之后通过内部调用solvePnPRansac()函数获得的# ret为标定结果，mtx为内参数矩阵，dist为畸变系数，rvecs为旋转矩阵，tvecs为平移向量ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, gray.shape[:2][::-1], None, criteria)# 摄像头内参mtx = [[f_x,0,c_x][0,f_y,c_y][0,0,1]]print('mtx=np.array( ' + str(mtx.tolist()) + " )")# 畸变系数dist = (k1,k2,p1,p2,k3)print('dist=np.array( ' + str(dist.tolist()) + " )")# 鱼眼/大广角镜头的单目标定K = np.zeros((3, 3))D = np.zeros((4, 1))RR = [np.zeros((1, 1, 3), dtype=np.float64) for i in range(len(objpoints))]TT = [np.zeros((1, 1, 3), dtype=np.float64) for i in range(len(objpoints))]rms, _, _, _, _ = cv2.fisheye.calibrate(objpoints, imgpoints, gray.shape[:2][::-1], K, D, RR, TT, flags_fisheye, criteria)# 摄像头内参,此结果与mtx相比更为稳定和精确print("K=np.array( " + str(K.tolist()) + " )")# 畸变系数D = (k1,k2,k3,k4)print("D=np.array( " + str(D.tolist()) + " )")# 计算反投影误差mean_error = 0for i in range(len(objpoints)):imgpoints2, _ = cv2.projectPoints(objpoints[i], rvecs[i], tvecs[i], mtx, dist)error = cv2.norm(imgpoints[i], imgpoints2, cv2.NORM_L2) / len(imgpoints2)mean_error += errorprint("total error: ", mean_error / len(objpoints))if saveFile:np.savez("./calibrate.npz", mtx=mtx, dist=dist, K=K, D=D)cv2.destroyAllWindows()return mtx, dist, K, D

结果:

校正前: