caffe样例_R-CNN detection基于Ubuntu_Caffe

R-CNN detection

1.背景

R-CNN是通过Caffe微调模型拟分类区域的检测器，详细内容可以参考项目网址和论文：

Rich feature hierarchies for accurate object detection and semantic segmentation. Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik. CVPR 2014. Arxiv 2013.

2.实现

2.1.Selective Search

Selective Search是R-CNN的区域拟提。selective_search_ijcv_with_pythonPython模块负责通过selective search MATLAB实现来提取拟提。先下载模块放置入caffe的根目录中（否则找不到模块中的函数）并重命名为selective_search_ijcv_with_python
$ cd ~/selectiveselective_search_ijcv_with_python
$ matlab
在MATLAB中运行demo.m编译必要的函数，并添加PYTHONPATH
$ export PYTHONPATH=/path/to/caffe/selectiveselective_search_ijcv_with_python

2.2.下载Caffe R-CNN ImageNet model

$ ./scripts/download_model_binary.py models/bvlc_reference_rcnn_ilsvrc13

2.3.产生h5文件(包含DataFrame , selected windows, and their detection scores)

$ mkdir -p _temp #建立临时文件
$ echo /path/to/caffe/images/fish-bike.jpg > _temp/det_input.txt #图像路径
$ ./python/detect.py --crop_mode=selective_search --pretrained_model=./models/bvlc_reference_rcnn_ilsvrc13/bvlc_reference_rcnn_ilsvrc13.caffemodel --model_def=./models/bvlc_reference_rcnn_ilsvrc13/deploy.prototxt --gpu --raw_scale=255 _temp/det_input.txt _temp/det_output.h5

2.4.数据处理

$ jupyter notebook

import numpy as npimport pandas as pdimport matplotlib.pyplot as plt%matplotlib inline​df = pd.read_hdf('_temp/det_output.h5', 'df')print(df.shape)print(df.iloc[0])

输出：

(1570, 5)prediction    [-2.62247, -2.84579, -2.85122, -3.20838, -1.94...ymin                                                     79.846xmin                                                       9.62ymax                                                     246.31xmax                                                    339.624Name: /home/dawin/caffe/examples/images/fish-bike.jpg, dtype: object

注释：1570个区域由selective search的R-CNN配置产生，区域的数量是基于每张图片的内容和大小变化的，因为selective search并不是尺度不变的。

接着，加载ILSVRC13检测类名录和建立一个预测数据表，要使用ILSVRC12辅佐数据，下载ILSVRC12辅佐数据:
$ sudo ./data/ilsvrc12/get_ilsvrc12_aux.sh

with open('./data/ilsvrc12/det_synset_words.txt') as f:    labels_df = pd.DataFrame([        {            'synset_id': l.strip().split(' ')[0],            'name': ' '.join(l.strip().split(' ')[1:]).split(',')[0]        }        for l in f.readlines()    ])labels_df.sort_values('synset_id')predictions_df = pd.DataFrame(np.vstack(df.prediction.values), columns=labels_df['name'])print(predictions_df.iloc[0])

输出：

nameaccordion         -2.622472airplane          -2.845789ant               -2.851220antelope          -3.208377apple             -1.949951armadillo         -2.472936artichoke         -2.201685axe               -2.327404baby bed          -2.737925backpack          -2.176763bagel             -2.681062balance beam      -2.722539banana            -2.390628band aid          -1.598909banjo             -2.298197baseball          -2.311024basketball        -2.733875bathing cap       -2.800533beaker            -2.405005bear              -3.286877bee               -2.990223bell pepper       -2.543806bench             -1.843543bicycle           -2.775216binder            -2.523010bird              -3.543336bookshelf         -2.767997bow tie           -2.532697bow               -2.003349bowl              -2.873867                     ...   strawberry        -2.559138stretcher         -2.048682sunglasses        -2.313832swimming trunks   -2.062061swine             -2.795033syringe           -2.287062table             -2.234039tape player       -3.081044tennis ball       -2.099031tick              -2.808466tie               -2.337841tiger             -2.615820toaster           -2.382439traffic light     -2.645303train             -3.441731trombone          -2.582362trumpet           -2.352854turtle            -2.360860tv or monitor     -2.761044unicycle          -2.218469vacuum            -1.907717violin            -2.757080volleyball        -2.723690waffle iron       -2.418540washer            -2.408994water bottle      -2.174899watercraft        -2.837426whale             -3.120339wine bottle       -2.772961zebra             -2.742914Name: 0, dtype: float32

查看激活：

plt.gray()plt.matshow(predictions_df.values)plt.xlabel('Classes')plt.ylabel('Windows')

输出:

<matplotlib.text.Text at 0x114f15f90><matplotlib.figure.Figure at 0x114254b50>

在所有窗口取最大值和绘制顶级类：

max_s = predictions_df.max(0)max_s.sort(ascending=False)print(max_s[:10])

输出：

nameperson          1.835771bicycle         0.866110unicycle        0.057080motorcycle     -0.006123banjo          -0.028209turtle         -0.189833electric fan   -0.206787cart           -0.214236lizard         -0.393519helmet         -0.477942dtype: float32

注释：实际上顶部的检测是人和自行车。选取好定位是一项进行中的工作，我们选取顶层得分的人和自行车检测。

# Find, print, and display the top detections: person and bicycle.i = predictions_df['person'].argmax()j = predictions_df['bicycle'].argmax()# Show top predictions for top detection.f = pd.Series(df['prediction'].iloc[i], index=labels_df['name'])print('Top detection:')print(f.order(ascending=False)[:5])print('')# Show top predictions for second-best detection.f = pd.Series(df['prediction'].iloc[j], index=labels_df['name'])print('Second-best detection:')print(f.order(ascending=False)[:5])# Show top detection in red, second-best top detection in blue.im = plt.imread('examples/images/fish-bike.jpg')plt.imshow(im)currentAxis = plt.gca()det = df.iloc[i]coords = (det['xmin'], det['ymin']), det['xmax'] - det['xmin'], det['ymax'] - det['ymin']currentAxis.add_patch(plt.Rectangle(*coords, fill=False, edgecolor='r', linewidth=5))det = df.iloc[j]coords = (det['xmin'], det['ymin']), det['xmax'] - det['xmin'], det['ymax'] - det['ymin']currentAxis.add_patch(plt.Rectangle(*coords, fill=False, edgecolor='b', linewidth=5))

输出：

Top detection:nameperson             1.835771swimming trunks   -1.150371rubber eraser     -1.231106turtle            -1.266038plastic bag       -1.303266dtype: float32Second-best detection:namebicycle     0.866110unicycle   -0.359140scorpion   -0.811621lobster    -0.982891lamp       -1.096809dtype: float32<matplotlib.patches.Rectangle at 0x7f02e8099d10>

让我们检测所有的 ‘自行车’ 和 用NMS使得他们摆脱重叠窗口：

def nms_detections(dets, overlap=0.3):    """    Non-maximum suppression: Greedily select high-scoring detections and    skip detections that are significantly covered by a previously    selected detection.    This version is translated from Matlab code by Tomasz Malisiewicz,    who sped up Pedro Felzenszwalb's code.    Parameters    ----------    dets: ndarray        each row is ['xmin', 'ymin', 'xmax', 'ymax', 'score']    overlap: float        minimum overlap ratio (0.3 default)    Output    ------    dets: ndarray        remaining after suppression.    """    x1 = dets[:, 0]    y1 = dets[:, 1]    x2 = dets[:, 2]    y2 = dets[:, 3]    ind = np.argsort(dets[:, 4])    w = x2 - x1    h = y2 - y1    area = (w * h).astype(float)    pick = []    while len(ind) > 0:        i = ind[-1]        pick.append(i)        ind = ind[:-1]        xx1 = np.maximum(x1[i], x1[ind])        yy1 = np.maximum(y1[i], y1[ind])        xx2 = np.minimum(x2[i], x2[ind])        yy2 = np.minimum(y2[i], y2[ind])        w = np.maximum(0., xx2 - xx1)        h = np.maximum(0., yy2 - yy1)        wh = w * h        o = wh / (area[i] + area[ind] - wh)        ind = ind[np.nonzero(o <= overlap)[0]]    return dets[pick, :]

scores = predictions_df['bicycle']windows = df[['xmin', 'ymin', 'xmax', 'ymax']].valuesdets = np.hstack((windows, scores[:, np.newaxis]))nms_dets = nms_detections(dets)

显示顶层 3 个图像中NMS检测到的 ‘自行车’ ，并注意顶部的得分框 （红色） 和其余的框之间的差距。

plt.imshow(im)currentAxis = plt.gca()colors = ['r', 'b', 'y']for c, det in zip(colors, nms_dets[:3]):    currentAxis.add_patch(        plt.Rectangle((det[0], det[1]), det[2]-det[0], det[3]-det[1],        fill=False, edgecolor=c, linewidth=5)    )print 'scores:', nms_dets[:3, 4]

输出：

scores: [ 0.86610961 -0.70051467 -1.34796405]