tensorflow从0开始（7）——利用tensorflow进行开发的准备工作

tensorflow开发流程——表情分析

前期准备

在利用tensorflow做表情分析时，需要很多处理模块进行辅助，由于对这些模块并不熟悉，因此，本文中会针对每个模块进行测试。

CK+数据提取与label标识

本文采用CK+作为数据库，网上可以下载到（自行google）。该数据库是一个基于视频帧的表情库。目前，我们第一版本的表情分析，利用图片作为输入，对这个表情库进行提取，每个人的每种表情提取一张图片并表上标签，python代码如下：

import oslist_tuple=[]list_filepath=[]list_label=[]for root, dir, files in os.walk('/home/beast/Code/emotiondata/cohn-kanade/'):files_num = len(files)if files_num > 0:file_fullpath=os.path.join(root,files[files_num/2])label = int(file_fullpath.split('_')[1])list_tuple.append([file_fullpath, label])list_filepath.append(file_fullpath)list_label.append(label)print list_filepath

cohn-kanade即为下载的CK+数据库解压后的存放位置。

opencv读取数据

• opencv读取图像数据与显示：

import cv2im = cv2.imread(list_filepath[0])cv2.namedWindow('emotion')cv2.imshow('emotion',im)cv2.waitKey(-1)

显示结果如下：

• opencv数据转换： 
  opencv读取数据后，数据的排布格式如下：

import cv2im = cv2.imread(list_filepath[0])print im.sizeprint im.shapeprint im

显示结果如下： 

opencv读取的图像的格式.png 
图中是rgb的图像，490行640列，每个像素的rgb通道是连续排列的。

• 利用opencv的数据是可以直接初始化tensorflow中的tensor的，但是能不能直接使用，这是后话，测试代码如下：

import tensorflow as tfflags = tf.app.flagsFLAGS = flags.FLAGSflags.DEFINE_string('summaries_dir', './tf_logs', 'Summaries directory')t1=tf.constant(im)t2=tf.Variable(im)print t1print t2with tf.Session() as sess:    sess.run(tf.initialize_all_variables())    merged = tf.merge_all_summaries()    train_writer = tf.train.SummaryWriter(FLAGS.summaries_dir + '/',                                      sess.graph)

显示结果如下： 

• 利用numpy将opencv提取出来的数据，针对某一个分量（如R），单独提取出来，代码如下：

import numpyimport cv2im_r = im[:,:,0].astype(numpy.float32)/255print im_r.shapeprint im_r

numpy的数据格式，也是可以直接用来初始化tensorflow变量的。

python中glob的使用

可以用来提取文件：

import globfor i in glob.glob('/home/beast/Code/emotiondata/cohn-kanade/S010/001/*.png'):    print i

python中zip的使用

zip用来合并两个list,实例代码如下：

python dlib的使用

import sysimport osimport dlibimport globfrom skimage import iopredictor_path = '/home/beast/Code/model/shape_predictor_68_face_landmarks.dat'faces_folder_path = '/home/beast/Code/Pic/haijun'detector = dlib.get_frontal_face_detector()predictor = dlib.shape_predictor(predictor_path)win = dlib.image_window()for f in glob.glob(os.path.join(faces_folder_path, "*.png")):    print("Processing file: {}".format(f))    img = io.imread(f)    win.clear_overlay()    win.set_image(img)    # Ask the detector to find the bounding boxes of each face. The 1 in the    # second argument indicates that we should upsample the image 1 time. This    # will make everything bigger and allow us to detect more faces.    dets = detector(img, 1)    print("Number of faces detected: {}".format(len(dets)))    for k, d in enumerate(dets):        print("Detection {}: Left: {} Top: {} Right: {} Bottom: {}".format(            k, d.left(), d.top(), d.right(), d.bottom()))        # Get the landmarks/parts for the face in box d.        shape = predictor(img, d)        print("Part 0: {}, Part 1: {} ...".format(shape.part(0),                                                  shape.part(1)))        # Draw the face landmarks on the screen.        win.add_overlay(shape)    win.add_overlay(dets)    dlib.hit_enter_to_continue()

opencv和dlib结合使用

经验证在python中，opencv和dlib的数据结构是可以通用的，本例中，利用opencv开启摄像头采集数据，利用dlib进行人脸的检测以及人脸关键点的检测。该示例的目的是能够在视频中检测出人脸并作为机器学习的输入，python代码如下：

• 单张图片的dlib检测代码如下：

import sysimport osimport dlibimport globfrom skimage import iopredictor_path = '/home/beast/Code/model/shape_predictor_68_face_landmarks.dat'faces_folder_path = '/home/beast/Code/Pic/haijun'detector = dlib.get_frontal_face_detector()predictor = dlib.shape_predictor(predictor_path)win = dlib.image_window()for f in glob.glob(os.path.join(faces_folder_path, "*.png")):    print("Processing file: {}".format(f))    img = io.imread(f)    win.clear_overlay()    win.set_image(img)    # Ask the detector to find the bounding boxes of each face. The 1 in the    # second argument indicates that we should upsample the image 1 time. This    # will make everything bigger and allow us to detect more faces.    dets = detector(img, 1)    print("Number of faces detected: {}".format(len(dets)))    for k, d in enumerate(dets):        print("Detection {}: Left: {} Top: {} Right: {} Bottom: {}".format(            k, d.left(), d.top(), d.right(), d.bottom()))        # Get the landmarks/parts for the face in box d.        shape = predictor(img, d)        print("Part 0: {}, Part 1: {} ...".format(shape.part(0),                                                  shape.part(1)))        # Draw the face landmarks on the screen.        win.add_overlay(shape)    win.add_overlay(dets)    dlib.hit_enter_to_continue()

显示结果如下： 

dlib对图片的人脸关键点定位.png

• 基于视频的人脸关键点检测代码如下：

import numpy as npimport cv2import cv2.cv as cvfrom video import create_capturefrom common import clock, draw_strimport dlibpredictor_path = '/home/beast/Code/model/shape_predictor_68_face_landmarks.dat'detector = dlib.get_frontal_face_detector()predictor = dlib.shape_predictor(predictor_path)win = dlib.image_window()cam = create_capture(0, fallback='synth:bg=../cpp/lena.jpg:noise=0.05')while True:    ret, img = cam.read()    dets = detector(img, 1)    print("Number of faces detected: {}".format(len(dets)))    win.clear_overlay()    for k, d in enumerate(dets):        print("Detection {}: Left: {} Top: {} Right: {} Bottom: {}".format(            k, d.left(), d.top(), d.right(), d.bottom()))        # Get the landmarks/parts for the face in box d.        shape = predictor(img, d)        print("Part 0: {}, Part 1: {} ...".format(shape.part(0),                                                  shape.part(1)))        # Draw the face landmarks on the screen.        win.add_overlay(shape)    win.set_image(img)    win.add_overlay(dets)    dlib.hit_enter_to_continue()    if 0xFF & cv2.waitKey(5) == 27:        break