人脸和性别识别（基于OpenCV）

描述

人脸识别包括四个步骤

1. 人脸检测：定位人脸区域，只关心是不是脸；
2. 人脸预处理：对人脸检测出来的图片进行调整优化；
3. 收集和学习人脸：收集要识别的人的预处理过的人脸，然后通过一些算法去学习如何识别；
4. 人脸识别：识别当前人脸与数据库里的哪个人脸最相似。

人脸检测

OpenCV集成了基于PCA LDA 和LBP的人脸检测器，源文件自带很多各种训练好的检测器。下表是常用的XML文件

上面的XML文件可以检测正面人脸、眼睛或鼻子。检测人脸我采用的是第一个或第二个Harr人脸检测器。识别率比较好。

第一步：加载Harr人脸检测XML文件

try{        faceCascade.load(faceCascadeFilename);    }catch(cv::Exception& e){}        if ( faceCascade.empty() ) {        cerr << "ERROR: Could not load Face Detection cascade classifier [" << faceCascadeFilename << "]!" << endl;        cerr << "Copy the file from your OpenCV data folder (eg: 'C:\\OpenCV\\data\\haarcascade_frontalface_alt2') into this WebcamFaceRec folder." << endl;        exit(1);    }    cout << "Loaded the Face Detection cascade classifier [" << faceCascadeFilename << "]." << endl;

第二步：加载摄像头，从视频获取图像帧。

try{        videoCapture.open(CameraID);    }catch(cv::Exception& e){}    if(!videoCapture.isOpened()){        cerr << "ERROR: could not open Camera!" << endl;        exit(1);    }videoCapture >> cameraFrame;

第三步：一帧图像预处理

1、 灰度转换：使用cvtColor()函数，将彩色图像转换为灰度图像。台式机是3通道的BGR，移动设备则是4通道的BGRA格式

if(srcimg.channels() ==3 ){         cvtColor(srcimg,gray_img,CV_BGR2GRAY);     }    else if(srcimg.channels() ==4 ){         cvtColor(srcimg,gray_img,CV_BGRA2GRAY);     }    else {         gray_img = srcimg;     }

2、直方图均衡化，在OpenCV函数中利用equalizeHist()函数执行直方图均衡化，提升对比度和亮度。

equalizeHist(gray_img,equalized_Img);

第四步：检测人脸

上面已经创建了级联分类器并加载好XML文件，接着使用函数Classifier::detecMultiScale()函数来检测人脸。这个函数的参数说明：
a、minFeatureSize: 该参数决定最小的人脸大小，通常可以设为20*20或30*30像素。如果使用摄像机或移动设备检测，则人脸一般很接近摄像机，可把参数调大，80*80；
b、searchScaleFactor: 该参数决定有多少不同大小的人脸要搜索，通常设为1.1
c、minNeighbors： 该参数决定检测器如何确定人脸已经被检测到。通常设为3
d、flags: 该参数设定是否要查找所有的人脸或最大的人脸
（CASCADE_FIND_BIGGEST_OBJECT）

int flags = CASCADE_FIND_BIGGEST_OBJECT;    //smallest object Size    Size minFeatureSize = Size(20,20);    // How detailed should the search be. Must be larger than 1.0.    float searchScaleFactor = 1.1f;    // How much the detections should be filtered out. This should depend on how bad false detections are to your system.    // minNeighbors=2 means lots of good+bad detections, and minNeighbors=6 means only good detections are given but some are missed.    int minNeighbors = 6;    vector<Rect> faces;    faceCascade.detectMultiScale(dectImg,faces,searchScaleFactor,                                    minNeighbors,flags,minFeatureSize);    //faceCascade.detectMultiScale(equalized_Img, faces);    int i = 0;    for(i = 0; i < faces.size(); i++){        Rect face_id = faces[i];        rectangle(orginalimg,face_id,Scalar(0,255,0),1);    }

人脸识别

为了识别人脸，需要收集足够多的要识别的人的人脸图像。收集好之后，选择适合人脸识别的机器学习算法，通过算法来学习收集的数据，从而训练出一个模型并保存，下次进来一帧图像，通过算法对模型里的参数进行匹配识别。人脸识别机器学习算法有很多，如SVM（支持向量机），ANN（人工神经网络）还有最常用的是基于特征脸的算法。OpenCV提供了CV::Algorithm类，类中有基于特征脸的（PCA 主成分分析）、Fisher脸（LDA 线性判别分析）和LPBH（局部二值模式直方图）

使用里面的算法，第一步必须通过cv::Algorithm::creat< FaceRecognizer>创建一个FaceRecognizer对象。创建了FaceRecognizer对象之后，将收集的人脸数据和标签传递给FaceRecognizer::train() 函数即可进行训练模型。

string facerecAlgorithm = "FaceRecognizer.Fisherfaces";Ptr<FaceRecognizer> model;// Use OpenCV's new FaceRecognizer in the "contrib" module:model = Algorithm::create<FaceRecognizer>(facerecAlgorithm);if (model.empty()) {cerr << "ERROR: The FaceRecognizer [" << facerecAlgorithm; cerr << "] is not available in your version of OpenCV. "; cerr << "Please update to OpenCV v2.4.1 or newer." << endl; exit(1);}model->train(preprocessedFaces, faceLabels);

训练好模型之后，一般是把模型保存下来，以免下次重复训练。直接加载模型即可。下一步就是人脸识别。同样，opencv把识别算法集成在FaceRecognizer类中，简单地调用FaceRecognizer::predict() 就可以识别。

int identity = model->predict(preprocessedFace);

测试程序

/* * Copyright (c) 2011. Philipp Wagner <bytefish[at]gmx[dot]de>. * Released to public domain under terms of the BSD Simplified license. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are met: *   * Redistributions of source code must retain the above copyright *     notice, this list of conditions and the following disclaimer. *   * Redistributions in binary form must reproduce the above copyright *     notice, this list of conditions and the following disclaimer in the *     documentation and/or other materials provided with the distribution. *   * Neither the name of the organization nor the names of its contributors *     may be used to endorse or promote products derived from this software *     without specific prior written permission. * *   See <http://www.opensource.org/licenses/bsd-license> */#include "opencv2/core/core.hpp"#include "opencv2/contrib/contrib.hpp"#include "opencv2/highgui/highgui.hpp"#include <iostream>#include <fstream>#include <sstream>#include <direct.h>using namespace cv;using namespace std;//const char *faceCascadeFilename = "C:\\opencv\\sources\\data\\lbpcascades\\lbpcascade_frontalface.xml"; const char *faceCascadeFilename = "C:\\opencv\\sources\\data\\haarcascades\\haarcascade_frontalface_alt2.xml"; const char *eyeCascadeFilename1 = "C:\\opencv\\sources\\data\\haarcascades\\haarcascade_eye.xml";               // Basic eye detector for open eyes only.const char *eyeCascadeFilename2 = "C:\\opencv\\sources\\data\\haarcascades\\haarcascade_eye_tree_eyeglasses.xml";const char *face_lib            = "face_train_img//";const int DESIRED_CAMERA_WIDTH = 640;const int DESIRED_CAMERA_HEIGHT = 480;const int Width = 92;const int Height = 112;int gender_width;int gender_height;int im_width;int im_height;string g_listname_t[]= {    "Jack",    "William",    "huang",    "Barton"};static Mat norm_0_255(InputArray _src) {    Mat src = _src.getMat();    // Create and return normalized image:    Mat dst;    switch(src.channels()) {    case 1:        cv::normalize(_src, dst, 0, 255, NORM_MINMAX, CV_8UC1);        break;    case 3:        cv::normalize(_src, dst, 0, 255, NORM_MINMAX, CV_8UC3);        break;    default:        src.copyTo(dst);        break;    }    return dst;}static void read_csv(const string& filename, vector<Mat>& images, vector<int>& labels, char separator = ';') {    std::ifstream file(filename.c_str(), ifstream::in);    if (!file) {        string error_message = "No valid input file was given, please check the given filename.";        CV_Error(CV_StsBadArg, error_message);    }    string line, path, classlabel;    while (getline(file, line)) {        stringstream liness(line);        getline(liness, path, separator);        getline(liness, classlabel);        if(!path.empty() && !classlabel.empty()) {            images.push_back(imread(path, 0));            labels.push_back(atoi(classlabel.c_str()));        }    }}static void InitVideoCapture(VideoCapture &videoCapture, int CameraID){    try{        videoCapture.open(CameraID);    }catch(cv::Exception& e){}    if(!videoCapture.isOpened()){        cerr << "ERROR: could not open Camera!" << endl;        exit(1);    }    videoCapture.set(CV_CAP_PROP_FRAME_WIDTH, DESIRED_CAMERA_WIDTH);    videoCapture.set(CV_CAP_PROP_FRAME_HEIGHT, DESIRED_CAMERA_HEIGHT);    cout << "CameraID is :" << CameraID << endl;}static void InitDetectors(CascadeClassifier &faceCascade, CascadeClassifier &eyeCascade1, CascadeClassifier &eyeCascade2){    try{        faceCascade.load(faceCascadeFilename);    }catch(cv::Exception& e){}        if ( faceCascade.empty() ) {        cerr << "ERROR: Could not load Face Detection cascade classifier [" << faceCascadeFilename << "]!" << endl;        cerr << "Copy the file from your OpenCV data folder (eg: 'C:\\OpenCV\\data\\haarcascade_frontalface_alt2') into this WebcamFaceRec folder." << endl;        exit(1);    }    cout << "Loaded the Face Detection cascade classifier [" << faceCascadeFilename << "]." << endl;    // Load the Eye Detection cascade classifier xml file.    try {   // Surround the OpenCV call by a try/catch block so we can give a useful error message!        eyeCascade1.load(eyeCascadeFilename1);    } catch (cv::Exception& e) {}    if ( eyeCascade1.empty() ) {        cerr << "ERROR: Could not load 1st Eye Detection cascade classifier [" << eyeCascadeFilename1 << "]!" << endl;        cerr << "Copy the file from your OpenCV data folder (eg: 'C:\\OpenCV\\data\\haarcascades') into this WebcamFaceRec folder." << endl;        exit(1);    }    cout << "Loaded the 1st Eye Detection cascade classifier [" << eyeCascadeFilename1 << "]." << endl;    // Load the Eye Detection cascade classifier xml file.    try {   // Surround the OpenCV call by a try/catch block so we can give a useful error message!        eyeCascade2.load(eyeCascadeFilename2);    } catch (cv::Exception& e) {}    if ( eyeCascade2.empty() ) {        cerr << "Could not load 2nd Eye Detection cascade classifier [" << eyeCascadeFilename2 << "]." << endl;        // Dont exit if the 2nd eye detector did not load, because we have the 1st eye detector at least.        //exit(1);    }    else        cout << "Loaded the 2nd Eye Detection cascade classifier [" << eyeCascadeFilename2 << "]." << endl;}void readDataTraining(Ptr<FaceRecognizer> &model,vector<Mat> &images,vector<int> &labels,string &filePath ){    // These vectors hold the images and corresponding labels.    // Read in the data. This can fail if no valid    // input filename is given.    try {        read_csv(filePath, images, labels);    } catch (cv::Exception& e) {        cerr << "Error opening file \"" << filePath << "\". Reason: " << e.msg << endl;        // nothing more we can do        exit(1);    }    // Quit if there are not enough images for this demo.    if(images.size() <= 1) {        string error_message = "This demo needs at least 2 images to work. Please add more images to your data set!";        CV_Error(CV_StsError, error_message);    }/*    Mat testSample = images[images.size() - 1];    int testLabel = labels[labels.size() - 1];    images.pop_back();    labels.pop_back();*/    model->train(images, labels);    //int predictedLabel = model->predict(testSample);    //    // To get the confidence of a prediction call the model with:    //    //      int predictedLabel = -1;    //      double confidence = 0.0;    //      model->predict(testSample, predictedLabel, confidence);    //    /*string result_message = format("Predicted class = %d / Actual class = %d.", predictedLabel, testLabel);    cout << result_message << endl;*/}void preprocessing(Mat &srcimg, Mat &dstimg){    Mat gray_img;    if(srcimg.channels() ==3 ){         cvtColor(srcimg,gray_img,CV_BGR2GRAY);     }    else if(srcimg.channels() ==4 ){         cvtColor(srcimg,gray_img,CV_BGRA2GRAY);     }    else {         gray_img = srcimg;     }     /*Mat equalized_Img;     equalizeHist(gray_img,equalized_Img);*/    dstimg = gray_img;}void faceDectRecog(CascadeClassifier &faceCascade,Ptr<FaceRecognizer> &model,Ptr<FaceRecognizer> &gender_model,Ptr<FaceRecognizer> &fishermodel,Mat &orginalimg,Mat &dectImg){    static int num = 0;    int predictedLabel = 0;    int gender_predict = 0;    // Only search for just 1 object (the biggest in the image).    int flags = CASCADE_FIND_BIGGEST_OBJECT;    //smallest object Size    Size minFeatureSize = Size(20,20);    // How detailed should the search be. Must be larger than 1.0.    float searchScaleFactor = 1.1f;    // How much the detections should be filtered out. This should depend on how bad false detections are to your system.    // minNeighbors=2 means lots of good+bad detections, and minNeighbors=6 means only good detections are given but some are missed.    int minNeighbors = 8;    vector<Rect> faces;    faceCascade.detectMultiScale(dectImg,faces,searchScaleFactor,                                    minNeighbors,flags,minFeatureSize);    //faceCascade.detectMultiScale(equalized_Img, faces);/*  pca + Lda    Mat eigenvalues = gender_model->getMat("eigenvalues");//提取model中的特征值，该特征值默认由大到小排列      Mat W = gender_model->getMat("eigenvectors");//提取model中的特征向量，特征向量的排列方式与特征值排列顺序一一对应      int xth = 121;//打算保留前121个特征向量，代码中没有体现原因，但选择121是经过斟酌的，首先，在我的实验中，"前121个特征值之和/所有特征值总和>0.97"；其次，121=11^2，可以将结果表示成一个11*11的2维图像方阵，交给fisherface去计算。      //vector<Mat> reduceDemensionimages;//降维后的图像矩阵      Mat evs = Mat(W, Range::all(), Range(0, xth));//选择前xth个特征向量，其余舍弃    Mat mean = gender_model->getMat("mean"); */    int i = 0;    for(i = 0; i < faces.size(); i++){        Rect face_id = faces[i];        Mat face = dectImg(face_id);        Mat face_resized;        Mat gender_resized;        cv::resize(face, face_resized, Size(im_width, im_height), 1.0, 1.0, INTER_CUBIC);        cv::resize(face, gender_resized, Size(gender_width, gender_height), 1.0, 1.0, INTER_CUBIC);        rectangle(orginalimg,face_id,Scalar(0,255,0),1);        predictedLabel = model->predict(face_resized);        string result_message;        /*result_message = format("Predicted  = %d ", predictedLabel);        cout << result_message << endl;*//*          PCA +LDA        Mat projection = subspaceProject(evs, mean, gender_resized.reshape(1,1));//做子空间投影          //reduceDemensionimages.push_back(projection.reshape(1,sqrt(xth*1.0)));        Mat reduceDemensionimages = projection.reshape(1,sqrt(xth*1.0));        gender_predict = fishermodel->predict(reduceDemensionimages);*/        string box_text;        box_text = format( "Prediction = " );        if ( predictedLabel >= 0 && predictedLabel <=3 )        {            box_text.append( g_listname_t[predictedLabel] );        }        else box_text.append( "Unknown" );        gender_predict = gender_model->predict(face_resized);        if(gender_predict == 0)        {            result_message = format("Predicted: female");            box_text.append( "   female" );        }        else if (gender_predict == 1)        {            result_message = format("Predicted: male");            box_text.append( "   male" );        }        else result_message = format("Predicted: Unknow");        cout << result_message << endl;        // Calculate the position for annotated text (make sure we don't        // put illegal values in there):        int pos_x = std::max(face_id.tl().x - 10, 0);        int pos_y = std::max(face_id.tl().y - 10, 0);        // And now put it into the image:         putText(orginalimg, box_text, Point(pos_x, pos_y), FONT_HERSHEY_PLAIN, 1.0, CV_RGB(0,255,0), 2.0);    }}void faceDect(CascadeClassifier &faceCascade,Mat &orginalimg,Mat &dectImg){    // Only search for just 1 object (the biggest in the image).    int flags = CASCADE_FIND_BIGGEST_OBJECT;    //smallest object Size    Size minFeatureSize = Size(20,20);    // How detailed should the search be. Must be larger than 1.0.    float searchScaleFactor = 1.1f;    // How much the detections should be filtered out. This should depend on how bad false detections are to your system.    // minNeighbors=2 means lots of good+bad detections, and minNeighbors=6 means only good detections are given but some are missed.    int minNeighbors = 6;    vector<Rect> faces;    faceCascade.detectMultiScale(dectImg,faces,searchScaleFactor,                                    minNeighbors,flags,minFeatureSize);    //faceCascade.detectMultiScale(equalized_Img, faces);    int i = 0;    for(i = 0; i < faces.size(); i++){        Rect face_id = faces[i];        rectangle(orginalimg,face_id,Scalar(0,255,0),1);    }}void CaptureFace(CascadeClassifier &faceCascade,Ptr<FaceRecognizer> &model,Mat &orginalimg,Mat &dectImg){    static int num = 0;    // Only search for just 1 object (the biggest in the image).    int flags = CASCADE_FIND_BIGGEST_OBJECT;    //smallest object Size    Size minFeatureSize = Size(20,20);    // How detailed should the search be. Must be larger than 1.0.    float searchScaleFactor = 1.1f;    // How much the detections should be filtered out. This should depend on how bad false detections are to your system.    // minNeighbors=2 means lots of good+bad detections, and minNeighbors=6 means only good detections are given but some are missed.    int minNeighbors = 6;    vector<Rect> faces;    faceCascade.detectMultiScale(dectImg,faces,searchScaleFactor,                                    minNeighbors,flags,minFeatureSize);    //faceCascade.detectMultiScale(equalized_Img, faces);    int i = 0;    for(i = 0; i < faces.size(); i++){        Rect face_id = faces[i];        Mat face = dectImg(face_id);        Mat face_resized;        cv::resize(face, face_resized, Size(im_width, im_height), 1.0, 1.0, INTER_CUBIC);        char c[4];        itoa(num,c,10);        string s = face_lib + (string)c + ".png";        imwrite(s,face_resized);        cout << "Capture the" << num << "face" << endl;        cout << s << ";" << face_id << endl;        num ++;        rectangle(orginalimg,face_id,Scalar(0,255,0),1);    }}int main(int argc, const char *argv[]) {    int mode;    int i;    // Get the path to your CSV.    string fn_csv = string("at.txt");    string gender_csv = string("gender.txt");    string temp_csv = string("test.txt");    CascadeClassifier faceCascade;    CascadeClassifier eyeCascade1;    CascadeClassifier eyeCascade2;    VideoCapture videoCapture;    Ptr<FaceRecognizer> model;    Ptr<FaceRecognizer> temp_model;    Ptr<FaceRecognizer> gender_model;    int CameraID = 0;    vector<Mat> images;    vector<int> labels;    vector<Mat> temp_images;    vector<int> temp_labels;    vector<Mat> gender_images;    vector<int> gender_labels;    cout << "Compiled with OpenCV version " << CV_VERSION << endl << endl;    InitDetectors(faceCascade,eyeCascade1,eyeCascade2);    InitVideoCapture(videoCapture,CameraID);    printf("\n");    printf("FaceDec and Recognition V0.1\n");    printf("Usage: mode 0 : FaceDect; 1: train your own face; 2: Recognition \n");    printf("please input mode\n");    scanf("%d",&mode);    //model = createEigenFaceRecognizer();    temp_model = createEigenFaceRecognizer();    model =createEigenFaceRecognizer();    gender_model =createEigenFaceRecognizer();    Ptr<FaceRecognizer> fishermodel = createFisherFaceRecognizer();      //gender_model = createEigenFaceRecognizer();    if(mode == 3)    {        readDataTraining(model,images,labels,fn_csv);        readDataTraining(gender_model,gender_images,gender_labels,gender_csv);        gender_width = gender_images[0].cols;        gender_height = gender_images[0].rows;        im_width = images[0].cols;        im_height = images[0].rows;        model->save("Face_recog.yml");        gender_model->save("gender_recog.yml");    }    //readDataTraining(temp_model,temp_images,gender_labels,temp_csv);    model->load("Face_recog.yml");    gender_width = Width;    gender_height = Height;    im_width = Width;    im_height = Height;    gender_model->load("eigenface_gender.yml");//保存训练结果，供检测时使用      fishermodel->load("fisher.yml");    printf("gender_width :%d gender_height :%d im_width: %d im_height:%d\n",gender_width,gender_height,im_width,im_height);    int num = 0;    Mat cameraFrame;    if(mode == 4)    {        read_csv(temp_csv, temp_images, temp_labels);        for(i = 0; i < temp_images.size(); i ++)        {            Mat temp_img;            preprocessing(temp_images[i],temp_img);            CaptureFace(faceCascade,temp_model,temp_images[i],temp_img);        }    }    for(;;){        videoCapture >> cameraFrame;        if( cameraFrame.empty()){            cerr << "Error : could not grap next frame " << endl;        }        Mat processFrame = cameraFrame.clone();        Mat preprocess_img;        preprocessing(processFrame,preprocess_img);        switch(mode){        case 0:            faceDect(faceCascade,processFrame,preprocess_img);            break;        case 1:            CaptureFace(faceCascade,model,processFrame,preprocess_img);        case 2:            faceDectRecog(faceCascade,model,gender_model,fishermodel,processFrame,preprocess_img);        default:            break;        }        imshow("face_recognizer",processFrame);        char key = (char) waitKey(300);        if(key == 27)            break;    }    return 0;}