Camera Calibration 相机标定：Opencv应用方法

来源：互联网发布：网络图标声音图标打叉编辑：程序博客网时间：2024/05/28 17:07

本系列文章由 @YhL_Leo 出品，转载请注明出处。
文章链接： http://blog.csdn.net/yhl_leo/article/details/49427383

Opencv中Camera Calibration and 3D Reconstruction中使用的是Z. Zhang(PAMI, 2000). A Flexible New Technique for Camera Calibration的方法。原理见原理简介（五）本文将对其进行介绍。

1 标定步骤

简单来说，Opencv中基于二维标定平面的标定方法主要步骤有：

1 读取相关设置信息，包括采用的pattern 信息（类型，尺寸），输入标定数据的信息（图像列表文件，视频采样方法），输出文件设置等，这些信息可以存为XML或YAML文件的形式或者在代码里直接显示设置。这里给出Opencv中提供的configuration file：

<?xml version="1.0"?><opencv_storage><Settings><!-- Number of inner corners per a item row and column. (square, circle) --><BoardSize_Width>9</BoardSize_Width><BoardSize_Height>6</BoardSize_Height><!-- The size of a square in some user defined metric system (pixel, millimeter)--><Square_Size>50</Square_Size><!-- The type of input used for camera calibration. One of: CHESSBOARD CIRCLES_GRID ASYMMETRIC_CIRCLES_GRID --><Calibrate_Pattern>"CHESSBOARD"</Calibrate_Pattern><!-- The input to use for calibration.         To use an input camera -> give the ID of the camera, like "1"        To use an input video  -> give the path of the input video, like "/tmp/x.avi"        To use an image list   -> give the path to the XML or YAML file containing the list of the images, like "/tmp/circles_list.xml"--><Input>"images/CameraCalibraation/VID5/VID5.xml"</Input><!--  If true (non-zero) we flip the input images around the horizontal axis.--><Input_FlipAroundHorizontalAxis>0</Input_FlipAroundHorizontalAxis><!--  Time delay between frames in case of camera.  --><Input_Delay>100</Input_Delay><!--  How many frames to use, for calibration.  --><Calibrate_NrOfFrameToUse>25</Calibrate_NrOfFrameToUse><!-- Consider only fy as a free parameter, the ratio fx/fy stays the same as in the input cameraMatrix.        Use or not setting. 0 - False Non-Zero - True--><Calibrate_FixAspectRatio>1</Calibrate_FixAspectRatio><!-- If true (non-zero) tangential distortion coefficients  are set to zeros and stay zero.--><Calibrate_AssumeZeroTangentialDistortion>1</Calibrate_AssumeZeroTangentialDistortion><!-- If true (non-zero) the principal point is not changed during the global optimization.--><Calibrate_FixPrincipalPointAtTheCenter>1</Calibrate_FixPrincipalPointAtTheCenter><!--  The name of the output log file.  --><Write_outputFileName>"out_camera_data.xml"</Write_outputFileName><!-- If true (non-zero) we write to the output file the feature points.--><Write_DetectedFeaturePoints>1</Write_DetectedFeaturePoints><!-- If true (non-zero) we write to the output file the extrinsic camera parameters.--><Write_extrinsicParameters>1</Write_extrinsicParameters><!-- If true (non-zero) we show after calibration the undistorted images.--><Show_UndistortedImage>1</Show_UndistortedImage></Settings></opencv_storage>

其中，图像文件列表images/CameraCalibraation/VID5/VID5.xmlOpencv中采用列举法：

<?xml version="1.0"?><opencv_storage><images>images/CameraCalibraation/VID5/xx1.jpg images/CameraCalibraation/VID5/xx2.jpg images/CameraCalibraation/VID5/xx3.jpg images/CameraCalibraation/VID5/xx4.jpg images/CameraCalibraation/VID5/xx5.jpg images/CameraCalibraation/VID5/xx6.jpg images/CameraCalibraation/VID5/xx7.jpg images/CameraCalibraation/VID5/xx8.jpg</images></opencv_storage>

文件中参数的含义比较清晰明了，此处就不累述。

2 依次从图像中检测pattern信息，如果检测成功，角点信息将会存储记录，用于标定解算。

cv::Mat viewGray;if ( view.channels() == 3 )    cv::cvtColor( view, viewGray, CV_BGR2GRAY );else    view.copyTo( viewGray );std::vector<cv::Point2f> imagePoints;   bool success = cv::findChessboardCorners( viewGray , boardSize, imagePoints);

3 优化角点检测精度，将上述检测成功的角点，通过精确角点定位方法，提高精度，下图为Opencv提供的检测结果。

cv::cornerSubPix( viewGray,               imagePoints,               cv::Size(11,11),              cv::TermCriteria( CV_TERMCRIT_EPS+CV_TERMCRIT_ITER, 30, 0.1 ));

Opencv

4 标定解算，每幅图像都进行上述的角点检测后，一般给像点对应的物方角点虚拟坐标的方式赋予对应的坐标，即可进行相机标定解算，包括相机内参，相机畸变系数，以及相机在虚拟坐标所在坐标系中相对于每幅图像的相对位置姿态（旋转向量和平移向量）。

double reprojectionError= cv::calibrateCamera(    objectPoints,   // calibration pattern points in the calibration pattern coordinate space    imagePoints,    // projections of calibration pattern points    imageSize,      // Size of the image used only to initialize the intrinsic camera matrix    cameraMatrix,   // camera matrix A    distCoeffs,     // distortion coefficients (k1,k2,p1,p2[,k3[,k4,k5,k6]])    rvecs,          // rotation vectors    tvecs,          // translation vectors    flag,           // different calibration model    criteria);      // Termination criteria for iterative optimization algorithm

5 标定精度评估，为了评价标定后的结果，可以按照标定得到的相机成像模型，由像点反算出物方空间坐标，进而得到一系列点云，通过对比解算点云与虚拟点云之间的差异性，就可以知道获得模型的好坏（严格来讲，如果误差较小，两者基本应该是一致的）。
6 图像畸变校正，在opencv示例中，作为标定的最后一个步骤，但是个人认为，这个应该可以作为一个相机标定后的副产品，对于处理的图像产品精度要求较高时，可以先进行畸变校正，再投入生产。下图为Opencv提供的畸变校正结果。

Opencv2

2 代码及结果

下面是个人的代码程序，有些部分并没完全按照Opencv的做法：

/*   Calibrate camera with chess board pattern.   - Editor: Menghan Xia, Yahui Liu.   - Data:   2015-07-28   - Email:  yahui.cvrs@gmail.com   - Address: Computer Vision and Remote Sensing(CVRS) Lab, Wuhan University.**/#include<iostream>#include <vector>#include <string>#include "cv.h"#include "highgui.h"#include "toolFunction.h"#define DEBUG_OUTPUT_INFOusing namespace std;using namespace cv;void main(){       char* folderPath = "E:/Images/New";           // image folder    std::vector<std::string> graphPaths;    std::vector<std::string> graphSuccess;    CalibrationAssist calAssist;    graphPaths = calAssist.get_filelist(folderPath); // collect image list#ifdef DEBUG_OUTPUT_INFO    std::cout << "loaded " << graphPaths.size() << " images"<< std::endl;#endif    if ( !graphPaths.empty() )    {#ifdef DEBUG_OUTPUT_INFO        std::cout << "Start corner detection ..." << std::endl;#endif        cv::Mat curGraph;  // current image        cv::Mat gray;      // gray image of current image        int imageCount = graphPaths.size();        int imageCountSuccess = 0;        cv::Size image_size;         cv::Size boardSize  = cv::Size(19, 19);     // chess board pattern size        cv::Size squareSize = cv::Size(15, 15);     // grid physical size, as a scale factor        std::vector<cv::Point2f> corners;                  // one image corner list        std::vector<std::vector<cv::Point2f> > seqCorners; // n images corner list        if ( graphPaths.size() < 3 )        {#ifdef DEBUG_OUTPUT_INFO            std::cout << "Calibrate failed, with less than three images!" << std::endl;#endif            return ;        }        for ( int i=0; i<graphPaths.size(); i++ )        {               string graphpath = folderPath;            graphpath += "/" + graphPaths[i];            curGraph = cv::imread(graphpath);            if ( curGraph.channels() == 3 )                cv::cvtColor( curGraph, gray, CV_BGR2GRAY );            else                curGraph.copyTo( gray );            // for every image, empty the corner list            std::vector<cv::Point2f>().swap( corners );              // corners detection            bool success = cv::findChessboardCorners( curGraph, boardSize, corners );             if ( success ) // succeed            {#ifdef DEBUG_OUTPUT_INFO                std::cout << i << " " << graphPaths[i] << " succeed"<< std::endl;#endif                int row = curGraph.rows;                int col = curGraph.cols;                graphSuccess.push_back( graphpath );                imageCountSuccess ++;                image_size = cv::Size( col, row );                // find sub-pixels                cv::cornerSubPix(                     gray,                     corners,                     cv::Size( 11, 11 ),                     cv::Size( -1, -1 ),                    cv::TermCriteria( CV_TERMCRIT_EPS + CV_TERMCRIT_ITER, 30, 0.1 ) );                seqCorners.push_back( corners );#if 1                // draw corners and show them in current image                cv::Mat imageDrawCorners;                if ( curGraph.channels() == 3 )                    curGraph.copyTo( imageDrawCorners );                else                    cv::cvtColor( curGraph, imageDrawCorners, CV_GRAY2RGB );                for ( int j = 0; j < corners.size(); j ++)                {                    cv::Point2f dotPoint = corners[j];                    cv::circle( imageDrawCorners, dotPoint, 3.0, cv::Scalar( 0, 255, 0 ), -1 );                    cv::Point2f pt_m = dotPoint + cv::Point2f(4,4);                    char text[100];                    sprintf( text, "%d", j+1 );  // corner indexes which start from 1                    cv::putText( imageDrawCorners, text, pt_m, 1, 0.5, cv::Scalar( 255, 0, 255 ) );                }                std::string pathTemp;                pathTemp = folderPath;                pathTemp += "/#" + graphPaths[i];                // save image drawn with corners and labeled with indexes                cv::imwrite( pathTemp, imageDrawCorners ); #endif            }#ifdef DEBUG_OUTPUT_INFO            else // failed            {                std::cout << graphPaths[i] << " corner detect failed!" << std::endl;            }#endif        }#ifdef DEBUG_OUTPUT_INFO        std::cout << "Corner detect done!" << std::endl             << imageCountSuccess << " succeed! " << std::endl;#endif        if ( imageCountSuccess < 3 )        {#ifdef DEBUG_OUTPUT_INFO            std::cout << "Calibrated success " << imageCountSuccess                 << " images, less than 3 images." << std::endl;#endif            return ;        }        else        {#ifdef DEBUG_OUTPUT_INFO            std::cout << "Start calibration ..." << std::endl;#endif            cv::Point3f point3D;            std::vector<cv::Point3f> objectPoints;            std::vector<double> distCoeffs;            std::vector<double> rotation;            std::vector<double> translation;            std::vector<std::vector<cv::Point3f>> seqObjectPoints;            std::vector<std::vector<double>> seqRotation;            std::vector<std::vector<double>> seqTranslation;            cv::Mat_<double> cameraMatrix;            // calibration pattern points in the calibration pattern coordinate space            for ( int t=0; t<imageCountSuccess; t++ )            {                objectPoints.clear();                for ( int i=0; i<boardSize.height; i++ )                {                    for ( int j=0; j<boardSize.width; j++ )                    {                        point3D.x = i * squareSize.width;                        point3D.y = j * squareSize.height;                        point3D.z = 0;                        objectPoints.push_back(point3D);                    }                }                seqObjectPoints.push_back(objectPoints);            }            double reprojectionError = calibrateCamera(                seqObjectPoints,                 seqCorners,                 image_size,                 cameraMatrix,                 distCoeffs,                 seqRotation,                 seqTranslation,                CV_CALIB_FIX_ASPECT_RATIO|CV_CALIB_FIX_PRINCIPAL_POINT );#ifdef DEBUG_OUTPUT_INFO            std::cout << "Calibration done!" << std::endl;#endif            // calculate the calibration pattern points with the camera model            std::vector<cv::Mat_<double>> projectMats;            for ( int i=0; i<imageCountSuccess; i++ )            {                cv::Mat_<double> R, T;                // translate rotation vector to rotation matrix via Rodrigues transformation                cv::Rodrigues( seqRotation[i], R );                 T = cv::Mat( cv::Matx31d(                     seqTranslation[i][0],                     seqTranslation[i][1],                    seqTranslation[i][2]) );                cv::Mat_<double> P = cameraMatrix * cv::Mat( cv::Matx34d(                     R(0,0), R(0,1), R(0,2), T(0),                      R(1,0), R(1,1), R(1,2), T(1),                      R(2,0), R(2,1), R(2,2), T(2) ) );                 projectMats.push_back(P);            }            std::vector<cv::Point2d> PointSet;            int pointNum = boardSize.width*boardSize.height;            std::vector<cv::Point3d> objectClouds;            for ( int i=0; i<pointNum; i++ )            {                PointSet.clear();                for ( int j=0; j<imageCountSuccess; j++ )                {                    cv::Point2d tempPoint = seqCorners[j][i];                    PointSet.push_back(tempPoint);                }                // calculate calibration pattern points                cv::Point3d objectPoint = calAssist.triangulate(projectMats,PointSet);                objectClouds.push_back(objectPoint);            }            std::string pathTemp_point;            pathTemp_point = folderPath;            pathTemp_point += "/point.txt";            calAssist.save3dPoint(pathTemp_point,objectClouds);            std::string pathTemp_calib;            pathTemp_calib = folderPath;            pathTemp_calib += "/calibration.txt";            FILE* fp = fopen( pathTemp_calib.c_str(), "w" );            fprintf( fp, "The average of re-projection error : %lf\n", reprojectionError );            for ( int i=0; i<imageCountSuccess; i++ )            {                std::vector<cv::Point2f> errorList;                cv::projectPoints(                     seqObjectPoints[i],                     seqRotation[i],                     seqTranslation[i],                     cameraMatrix,                     distCoeffs,                     errorList );                corners.clear();                corners = seqCorners[i];                double meanError(0.0);                for ( int j=0; j<corners.size(); j++ )                {                       meanError += std::sqrt((errorList[j].x - corners[j].x)*(errorList[j].x - corners[j].x) +                         (errorList[j].y - corners[j].y)*(errorList[j].y - corners[j].y));                }                rotation.clear();                translation.clear();                rotation = seqRotation[i];                translation = seqTranslation[i];                fprintf( fp, "Re-projection of image %d：%lf\n", i+1, meanError/corners.size() );                fprintf( fp, "Rotation vector :\n" );                fprintf( fp, "%lf %lf %lf\n", rotation[0], rotation[1], rotation[2] );                fprintf( fp, "Translation vector :\n" );                fprintf( fp, "%lf %lf %lf\n\n", translation[0], translation[1], translation[2] );            }            fprintf( fp, "Camera internal matrix :\n" );            fprintf( fp, "%lf %lf %lf\n%lf %lf %lf\n%lf %lf %lf\n",                 cameraMatrix(0,0), cameraMatrix(0,1), cameraMatrix(0,2),                cameraMatrix(1,0), cameraMatrix(1,1), cameraMatrix(1,2),                cameraMatrix(2,0), cameraMatrix(2,1), cameraMatrix(2,2));            fprintf( fp,"Distortion coefficient :\n" );            for ( int k=0; k<distCoeffs.size(); k++)                fprintf( fp, "%lf ", distCoeffs[k] );#ifdef DEBUG_OUTPUT_INFO            std::cout << "Results are saved!" << std::endl;#endif          }    }}

// toolFunction.h#ifndef TOOL_FUNCTION_H#pragma once#define TOOL_FUNCTION_H#include<iostream>#include <Windows.h>#include <math.h>#include <fstream>#include <vector>#include <string>#include "cv.h"#include "highgui.h"using namespace cv;using namespace std;class CalibrationAssist{public:    CalibrationAssist() {}    ~CalibrationAssist() {}public:    std::vector<std::string>get_filelist( std::string foldname );    cv::Point3d triangulate( std::vector<cv::Mat_<double>> &ProjectMats,         std::vector<cv::Point2d> &imagePoints );    void save3dPoint( std::string path_, std::vector<cv::Point3d> &Point3dLists );};#endif // TOOL_FUNCTION_H

// toolFunction.cpp#include "toolFunction.h"std::vector<std::string> CalibrationAssist::get_filelist( std::string foldname ){    foldname += "/*.*";    const char * mystr=foldname.c_str();    std::vector<std::string> flist;    std::string lineStr;    std::vector<std::string> extendName;    extendName.push_back("jpg");    extendName.push_back("JPG");    extendName.push_back("bmp");    extendName.push_back("png");    extendName.push_back("gif");    HANDLE file;    WIN32_FIND_DATA fileData;    char line[1024];    wchar_t fn[1000];    mbstowcs( fn, mystr, 999 );    file = FindFirstFile( fn, &fileData );    FindNextFile( file, &fileData );    while(FindNextFile( file, &fileData ))    {        wcstombs( line, (const wchar_t*)fileData.cFileName, 259);        lineStr = line;        // remove the files which are not images        for (int i = 0; i < 4; i ++)        {            if (lineStr.find(extendName[i]) < 999)            {                flist.push_back(lineStr);                break;            }        }       }    return flist;}cv::Point3d CalibrationAssist::triangulate(    std::vector<cv::Mat_<double>> &ProjectMats,     std::vector<cv::Point2d> &imagePoints){    int i,j;    std::vector<cv::Point2d> pointSet;    int frameSum = ProjectMats.size();    cv::Mat A(2*frameSum,3,CV_32FC1);    cv::Mat B(2*frameSum,1,CV_32FC1);    cv::Point2d u,u1;    cv::Mat_<double> P;    cv::Mat_<double> rowA1,rowA2,rowB1,rowB2;    int k = 0;    for ( i = 0; i < frameSum; i++ )     //get the coefficient matrix A and B    {        u = imagePoints[i];        P = ProjectMats[i];        cv::Mat( cv::Matx13d(             u.x*P(2,0)-P(0,0),            u.x*P(2,1)-P(0,1),            u.x*P(2,2)-P(0,2) ) ).copyTo( A.row(k) );        cv::Mat( cv::Matx13d(             u.y*P(2,0)-P(1,0),            u.y*P(2,1)-P(1,1),            u.y*P(2,2)-P(1,2) ) ).copyTo( A.row(k+1) );        cv::Mat rowB1( 1, 1, CV_32FC1, cv::Scalar( -(u.x*P(2,3)-P(0,3)) ) );        cv::Mat rowB2( 1, 1, CV_32FC1, cv::Scalar(-(u.y*P(2,3)-P(1,3)) ) );        rowB1.copyTo( B.row(k) );        rowB2.copyTo( B.row(k+1) );        k += 2;    }    cv::Mat X;      cv::solve( A, B, X, DECOMP_SVD );      return Point3d(X); }void CalibrationAssist::save3dPoint( std::string path_, std::vector<cv::Point3d> &Point3dLists){    const char * path = path_.c_str();    FILE* fp = fopen( path, "w" );    for ( int i = 0; i < Point3dLists.size(); i ++)    {        //      fprintf(fp,"%d ",i);        fprintf( fp, "%lf %lf %lf\n",             Point3dLists[i].x, Point3dLists[i].y, Point3dLists[i].z);    }    fclose(fp);#if 1    std::cout << "clouds of points are saved!" << std::endl;#endif}

使用数据为9张1200×800的图像：

Opencv4

程序运行结果：

1 运行控制台输出结果
2 角点检测图

Opencv5

3 反投影点云（CloudCompare显示）

Opencv6

对于上述结果的生成文件，此处用了C语言写成txt的方式，读者完全可以考虑使用XML或YAML格式文件保存，至于畸变纠正的问题，也很简单，直接利用标定得到的相机内参和畸变系数，查询remap函数的使用方法即可。此外，处理较大图像时，Opencv提供的方法速度可能会较慢，遇到这种情况，可以考虑把图像缩小或重写角点检测算法。

3 0