iOS开发之opencv学习笔记二：使用CascadeClassifier进行对特定物体的跟踪

上一篇介绍了opencv的下载和安装

如果成功编译出opencv3.2.0的iOS framework，那么就可以使用opencv的功能了。

1.CascadeClassifier是什么？

opencv有一个很强大的功能：识别跟踪。你可以利用CascadeClassifier对特定的物体进行跟踪，来实现VR/AR功能。

2.怎么做人脸识别跟踪？

首先，我们需要一个级联特征库，这是一个xml文件，可以认为它是opencv的记忆载体。

我找到了一个haarcascade_frontalface_alt2.xml，这记录的是如何辨别人脸的特征库。那么，我们开始做人脸识别吧。

创建一个iOS工程，把haarcascade_frontalface_alt2.xml拷到工程，导入头文件

#ifdef __cplusplus#import <opencv2/opencv.hpp>#import <opencv2/core.hpp>#import <opencv2/objdetect/objdetect.hpp>#import <opencv2/highgui/highgui.hpp>#import <opencv2/calib3d/calib3d.hpp>#import <opencv2/imgproc/imgproc_c.h>#import <opencv2/features2d/features2d.hpp>#import <opencv2/highgui/highgui.hpp>#endif

建议做一个PCH文件，把这块放到PCH文件里。
做一个摄像头取景的ViewController，大家都是iOS高手，我就不赘述过程了。不过我建议使用AVCaptureVideoPreviewLayer来渲染帧数据，而不是用UIImageView。
初始化并加载一个CascadeClassifier

NSString *path = [[NSBundle mainBundle] pathForResource:@"haarcascade_frontalface_alt2" ofType:@"xml"];cv::CascadeClassifier clsfr;        clsfr.load([path cStringUsingEncoding:NSASCIIStringEncoding]);

在摄像头帧回调里面将CMSampleBufferRef转换成UIIMage

+ (UIImage *) imageFromSampleBuffer:(CMSampleBufferRef) sampleBuffer{    // Get a CMSampleBuffer's Core Video image buffer for the media data    CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);    // Lock the base address of the pixel buffer    CVPixelBufferLockBaseAddress(imageBuffer, 0);    // Get the number of bytes per row for the plane pixel buffer    void *baseAddress = CVPixelBufferGetBaseAddress(imageBuffer);    // Get the number of bytes per row for the plane pixel buffer    size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);    // Get the pixel buffer width and height    size_t width = CVPixelBufferGetWidth(imageBuffer);    size_t height = CVPixelBufferGetHeight(imageBuffer);    // Create a device-dependent gray color space    CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();    // Create a bitmap graphics context with the sample buffer data    CGContextRef context = CGBitmapContextCreate(baseAddress, width, height, 8,                                                 bytesPerRow, colorSpace, kCGBitmapByteOrder32Little | kCGImageAlphaNoneSkipFirst);    // Create a Quartz image from the pixel data in the bitmap graphics context    CGImageRef quartzImage = CGBitmapContextCreateImage(context);    // Unlock the pixel buffer    CVPixelBufferUnlockBaseAddress(imageBuffer,0);    // Free up the context and color space    CGContextRelease(context);    CGColorSpaceRelease(colorSpace);    // Create an image object from the Quartz image    UIImage *image = [UIImage imageWithCGImage:quartzImage scale:1 orientation:UIImageOrientationRight];    // Release the Quartz image    CGImageRelease(quartzImage);    //    _frameImage = image;    return (image);}

这时候就得到一张可能90度倒转的图片，这时候就要对这张图片进行修正

- (UIImage *)fixOrientation{    if (self.imageOrientation == UIImageOrientationUp) return self;        // We need to calculate the proper transformation to make the image upright.    // We do it in 2 steps: Rotate if Left/Right/Down, and then flip if Mirrored.    CGAffineTransform transform = CGAffineTransformIdentity;        switch (self.imageOrientation)    {        case UIImageOrientationDown:        case UIImageOrientationDownMirrored:            transform = CGAffineTransformTranslate(transform, self.size.width, self.size.height);            transform = CGAffineTransformRotate(transform, M_PI);            break;                    case UIImageOrientationLeft:        case UIImageOrientationLeftMirrored:            transform = CGAffineTransformTranslate(transform, self.size.width, 0);            transform = CGAffineTransformRotate(transform, M_PI_2);            break;                    case UIImageOrientationRight:        case UIImageOrientationRightMirrored:            transform = CGAffineTransformTranslate(transform, 0, self.size.height);            transform = CGAffineTransformRotate(transform, -M_PI_2);            break;        case UIImageOrientationUp:        case UIImageOrientationUpMirrored:            break;    }        switch (self.imageOrientation)    {        case UIImageOrientationUpMirrored:        case UIImageOrientationDownMirrored:            transform = CGAffineTransformTranslate(transform, self.size.width, 0);            transform = CGAffineTransformScale(transform, -1, 1);            break;                    case UIImageOrientationLeftMirrored:        case UIImageOrientationRightMirrored:            transform = CGAffineTransformTranslate(transform, self.size.height, 0);            transform = CGAffineTransformScale(transform, -1, 1);            break;        case UIImageOrientationUp:        case UIImageOrientationDown:        case UIImageOrientationLeft:        case UIImageOrientationRight:            break;    }        // Now we draw the underlying CGImage into a new context, applying the transform    // calculated above.    CGContextRef ctx = CGBitmapContextCreate(NULL, self.size.width, self.size.height,                                             CGImageGetBitsPerComponent(self.CGImage), 0,                                             CGImageGetColorSpace(self.CGImage),                                             CGImageGetBitmapInfo(self.CGImage));    CGContextConcatCTM(ctx, transform);        switch (self.imageOrientation)    {        case UIImageOrientationLeft:        case UIImageOrientationLeftMirrored:        case UIImageOrientationRight:        case UIImageOrientationRightMirrored:            CGContextDrawImage(ctx, CGRectMake(0,0,self.size.height,self.size.width), self.CGImage);            break;                    default:            CGContextDrawImage(ctx, CGRectMake(0,0,self.size.width,self.size.height), self.CGImage);            break;    }        CGImageRef cgimg = CGBitmapContextCreateImage(ctx);    UIImage *img = [UIImage imageWithCGImage:cgimg];    CGContextRelease(ctx);    CGImageRelease(cgimg);        return img;}

然后将这张图片转换成cv::Mat

+ (void) UIImageToMat:(UIImage *)image mat:(cv::Mat&)m alphaExist:(bool)alphaExist{        CGColorSpaceRef colorSpace = CGImageGetColorSpace(image.CGImage);    CGFloat cols = image.size.width, rows = image.size.height;    CGContextRef contextRef;    CGBitmapInfo bitmapInfo = kCGImageAlphaPremultipliedLast;    if (CGColorSpaceGetModel(colorSpace) == 0)    {        m.create(rows, cols, CV_8UC1); // 8 bits per component, 1 channel        bitmapInfo = kCGImageAlphaNone;        if (!alphaExist)            bitmapInfo = kCGImageAlphaNone;        contextRef = CGBitmapContextCreate(m.data, m.cols, m.rows, 8,                                           m.step[0], colorSpace,                                           bitmapInfo);    }    else    {        m.create(rows, cols, CV_8UC4); // 8 bits per component, 4 channels        if (!alphaExist)            bitmapInfo = kCGImageAlphaNoneSkipLast |            kCGBitmapByteOrderDefault;        contextRef = CGBitmapContextCreate(m.data, m.cols, m.rows, 8,                                           m.step[0], colorSpace,                                           bitmapInfo);    }    CGContextDrawImage(contextRef, CGRectMake(0, 0, cols, rows),                       image.CGImage);    CGContextRelease(contextRef);}

接下来就可以这样了

vector<cv::Rect> faces;double scalingFactor = 1.1;int minNeighbors = 2;int flags = 0;clsfr.detectMultiScale(mat,                                faces,                                scalingFactor,                                minNeighbors,                                flags,                                cv::Size(30, 30));if (faces.size() > 0) {                        //创建画布将人脸部分标记出            CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();            CGContextRef contextRef = CGBitmapContextCreate(NULL, img.size.width, img.size.height,8, img.size.width * 4,colorSpace, kCGImageAlphaPremultipliedLast|kCGBitmapByteOrderDefault);                        CGContextSetLineWidth(contextRef, 4);            CGContextSetRGBStrokeColor(contextRef, 1.0, 0.0, 0.0, 1);                        for(int i = 0; i < faces.size(); i++) {                // Calc the rect of faces                CvRect cvrect = faces[i];                CGRect face_rect = CGContextConvertRectToDeviceSpace(contextRef, CGRectMake(cvrect.x*scale, cvrect.y*scale , cvrect.width*scale, cvrect.height*scale));                                CGContextStrokeRect(contextRef, face_rect);            }                        dispatch_async(dispatch_get_main_queue(), ^{                                CGImageRef rectImageRef = CGBitmapContextCreateImage(contextRef);                UIImage *rectImage = [UIImage imageWithCGImage:rectImageRef];                _imageView.image = rectImage;                                CGContextRelease(contextRef);            });        }

这里面_imageView是覆盖在摄像头预览图层上的，你就可以看到一个小框框跟着你拍到的人脸后面跑来跑去了。

3.哪里可以找到haarcascade_frontalface_alt2.xml或者其他的特征库？
在opencv的data目录有这几个目录


名字包含cascades的目录里面都是xml文件，这些就是opencv贡献者们训练出来用来跟踪特定物体的特征库。