Android下使用jni实现基于opencv与dlib的68关键点检测

关键点：

• 配置项目依赖项环境
• 从手机摄像头获取图片帧
• 将图片帧数据传到C++层及从C++层获取处理后的图片帧数据
• 不安装opencv manager运行基于android opencv库
• C++层68关键点检测
• 项目编译

效果图：

配置项目依赖项环境

NDK的安装配置请自行百度，Android opencv sdk下载
1.新建Android项目
2.右键项目—>Android Tools—>Add Native Supports…

3.在生成的jni目录下新建Application.mk

APP_PLATFORM := android-18APP_STL := gnustl_staticAPP_ABI := armeabi-v7a 

4.右键项目—>properties—>c/c++General—>Paths and Symbols—>Include—>add

${NDK_HOME}/platforms/android-18/arch-arm/usr/include${NDK_HOME}/sources/cxx-stl/gnu-libstdc++/4.9/include${NDK_HOME}/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi-v7a/include

5.在Project Explorer空白处右键import->General->Existing Projects into Workspace->选择下载的opencvSDK目录下sdk目录
如：E:\OpenCV-2.4.10-android-sdk\sdk，在Project中选择Opencv Library这个项目->Finish.
然后右键之前创建的项目->properties->android->在Library栏中点击Add，弹出框中选择OpenCV Library，点击OK，然后Apply，OK。

6.在项目根目录创建third_party目录，然后将E:\OpenCV-2.4.10-android-sdk\sdk\native拷贝进来具体结构可参考Demo。

从手机摄像头获取图片帧（下一篇博文讲述）

将图片帧数据传到C++层及从C++层获取处理后的图片

private Bitmap srcBitmap;Bitmap bitmap = BitmapFactory.decodeResource(getResources(), R.drawable.person);srcBitmap = bitmap;Mat input = new Mat();Mat output = new Mat();Utils.bitmapToMat(srcBitmap, input);Landmarks.landMarks(input.getNativeObjAddr(), output.getNativeObjAddr());Utils.matToBitmap(output, srcBitmap);  

不安装opencv manager运行基于android opencv库

1.在项目jni目录下创建prebuilt文件夹，在opencv sdk中将相应Android架构下的libopencv_java.so拷贝进来然后新建Androidusc.mk文件.

LOCAL_PATH := $(call my-dir)include $(CLEAR_VARS)LOCAL_MODULE := opencv_javaLOCAL_SRC_FILES := libopencv_java.soinclude $(PREBUILT_SHARED_LIBRARY)

在Android.mk文件最后添加include $(TOP_LEVEL_PATH)/prebuilt/Androidusc.mk（TOP_LEVEL_PATH路径根据项目进行修改）
在Java代码中引用

@Overridepublic void onResume(){    super.onResume();    if (!OpenCVLoader.initDebug()) {    } else {        System.loadLibrary("DlibLandmarks");      }}

C++层68关键点检测

void detect(cv::Mat input){    LOGE("detect");    Mat result;    cvtColor(input, result, CV_RGBA2BGR);    assign_image(img, cv_image<bgr_pixel>(result));    std::vector<dlib::rectangle> dets = detector(img);//检测人脸，获得边界框    int Max = 0;    int area = 0;    if (dets.size() != 0)    {        for (unsigned long t = 0; t < dets.size(); ++t)        {            if (area < dets[t].width()*dets[t].height())            {                area = dets[t].width()*dets[t].height();                Max = t;            }        }    }    full_object_detection shape = sp(img, dets[Max]);    box.x = dets[Max].left();    box.y = dets[Max].top();    box.width = dets[Max].width();    box.height = dets[Max].height();    pts2d.clear();    // 将结果保存至 pts2d 里面    for (size_t k = 0; k < shape.num_parts(); k++) {        Point2d p(shape.part(k).x(), shape.part(k).y());        pts2d.push_back(p);    }}

项目编译

项目Android.mk

LOCAL_PATH := $(call my-dir)# =======================================================include $(CLEAR_VARS)# Define 3rd party pathTOP_LEVEL_PATH = $(LOCAL_PATH)/..$(info TOP_LEVEL_PATH: $(TOP_LEVEL_PATH))THIRD_PARTY_PATH = $(LOCAL_PATH)/../../third_partyOPENCV_PATH = $(THIRD_PARTY_PATH)/opencvOpenCV_INSTALL_MODULES := onOPENCV_LIB_TYPE := SHAREinclude $(OPENCV_PATH)/jni/OpenCV.mkLOCAL_MODULE := DlibLandmarksLOCAL_C_INCLUDES :=$(OPENCV_PATH)/jni/include \    $(LOCAL_PATH)/../dlib \LOCAL_SRC_FILES := \    ../dlib//dlib/threads/threads_kernel_shared.cpp \    ../dlib/dlib/entropy_decoder/entropy_decoder_kernel_2.cpp \    ../dlib/dlib/base64/base64_kernel_1.cpp \    ../dlib/dlib/threads/threads_kernel_1.cpp \    ../dlib/dlib/threads/threads_kernel_2.cpp LOCAL_SRC_FILES += \           DlibLandmarks.cppLOCAL_LDLIBS := -lm -llog -ldl -lz -ljnigraphicsLOCAL_CPPFLAGS += -fexceptions -frtti -std=c++11LOCAL_ARM_MODE := armifeq ($(TARGET_ARCH_ABI), armeabi-v7a)    LOCAL_ARM_NEON := trueendifinclude $(BUILD_SHARED_LIBRARY)#-----------------------------------------------------------------------------include $(TOP_LEVEL_PATH)/prebuilt/Androidusc.mk

注意：请仔细查看各级文件路径，具体文件层级请参考Demo。

Demo（DlibLandmarks）下载