Caffe_Windows学习笔记（二）用自己的数据训练和测试CaffeNet

0、参考文献

[1]薛开宇 《学习笔记3 用自己的数据训练和测试“CaffeNet”》

[2]悠望南山 http://www.cnblogs.com/NanShan2016/p/5532589.html

1、准备数据集

1）全景网下载的50张鸟、50张猫，批量重命名，放在新建文件夹中，如下。

bird

cat

2）全景网下载的10张鸟、10张猫，批量重命名，放在新建文件夹中，如下。

3）制作train.txt（注意要指明子文件夹）、val.txt，放在data/myself下。生成标签代码如下：

import osdef IsSubString(SubStrList,Str):    flag=True    for substr in SubStrList:        if not(substr in Str):            flag=False       return flagdef GetFileList(FindPath,FlagStr=[]):    FileList=[]    FileNames=os.listdir(FindPath)    if len(FileNames)>0:        for fn in FileNames:            if len(FlagStr)>0:                if IsSubString(FlagStr,fn):                    fullfilename=os.path.join(FindPath,fn)                    FileList.append(fullfilename)            else:                fullfilename=os.path.join(FindPath,fn)                FileList.append(fullfilename)        if len(FileList)>0:        FileList.sort()           return FileListtrain_txt=open('train.txt','w')imgfile=GetFileList('train/train_dhj/')for img in imgfile:    str1=img+' '+'1'+'\n'            train_txt.writelines(str1)    imgfile=GetFileList('train/train_zzz/')for img in imgfile:    str2=img+' '+'0'+'\n'    train_txt.writelines(str2)train_txt.close()test_txt=open('val.txt','w')imgfile=GetFileList('val/test_dhj/')for img in imgfile:    str3=img+' '+'1'+'\n'    test_txt.writelines(str3)    imgfile=GetFileList('val/test_zzz/')for img in imgfile:    str4=img+' '+'0'+'\n'    test_txt.writelines(str4)test_txt.close()print("succeed")

 

2、将训练集与测试集转成leveldb格式

1）将examples/imagenet的create_imagenet.sh复制到examples/myself文件夹下，代码修改如下

#!/usr/bin/env sh# Create the imagenet lmdb inputs# N.B. set the path to the imagenet train + val data dirsset -eEXAMPLE=examples/myselfDATA=data/myselfTOOLS=Build/x64/ReleaseTRAIN_DATA_ROOT=data/myself/train/VAL_DATA_ROOT=data/myself/val/# Set RESIZE=true to resize the images to 256x256. Leave as false if images have# already been resized using another tool.RESIZE=trueif $RESIZE; then  RESIZE_HEIGHT=256  RESIZE_WIDTH=256else  RESIZE_HEIGHT=0  RESIZE_WIDTH=0fiif [ ! -d "$TRAIN_DATA_ROOT" ]; then  echo "Error: TRAIN_DATA_ROOT is not a path to a directory: $TRAIN_DATA_ROOT"  echo "Set the TRAIN_DATA_ROOT variable in create_imagenet.sh to the path" \       "where the ImageNet training data is stored."  exit 1fiif [ ! -d "$VAL_DATA_ROOT" ]; then  echo "Error: VAL_DATA_ROOT is not a path to a directory: $VAL_DATA_ROOT"  echo "Set the VAL_DATA_ROOT variable in create_imagenet.sh to the path" \       "where the ImageNet validation data is stored."  exit 1fiecho "Creating train lmdb..."GLOG_logtostderr=1 $TOOLS/convert_imageset \    --resize_height=$RESIZE_HEIGHT \    --resize_width=$RESIZE_WIDTH \--backend=leveldb \    --shuffle \    $TRAIN_DATA_ROOT \    $DATA/train.txt \    $DATA/myself_train_leveldbecho "Creating val lmdb..."GLOG_logtostderr=1 $TOOLS/convert_imageset \    --resize_height=$RESIZE_HEIGHT \    --resize_width=$RESIZE_WIDTH \--backend=leveldb \    --shuffle \    $VAL_DATA_ROOT \    $DATA/val.txt \    $DATA/myself_val_leveldbecho "Done."

2）在cygwin64中运行create_imagenet.sh，得到

3、计算图像均值

1）将examples/imagenet的make_imagenet_mean.sh复制到examples/myself文件夹下，代码修改如下

#!/usr/bin/env sh# Compute the mean image from the imagenet training lmdb# N.B. this is available in data/ilsvrc12EXAMPLE=examples/myselfDATA=data/myselfTOOLS=Build/x64/Release$TOOLS/compute_image_mean $DATA/myself_train_leveldb $DATA/imagenet_mean.binaryproto --backend=leveldbecho "Done."

2)在cygwin64中运行make_imagenet_mean.sh，得到均值文件

4、网络定义

1）复制models/bvlc_reference_caffenet的train_val.prototxt和solver.prototxt到examples/myself

2）设置train_val.prototxt文件，对数据层进行修改。

name: "CaffeNet"layer {  name: "data"  type: "Data"  top: "data"  top: "label"  include {    phase: TRAIN  }  transform_param {    mirror: true    crop_size: 227    mean_file: "D:/caffe_test/caffe-master/data/myself/imagenet_mean.binaryproto"  }# mean pixel / channel-wise mean instead of mean image#  transform_param {#    crop_size: 227#    mean_value: 104#    mean_value: 117#    mean_value: 123#    mirror: true#  }  data_param {    source: "D:/caffe_test/caffe-master/data/myself/myself_train_leveldb"    batch_size: 5    backend: LEVELDB  }}layer {  name: "data"  type: "Data"  top: "data"  top: "label"  include {    phase: TEST  }  transform_param {    mirror: false    crop_size: 227    mean_file: "D:/caffe_test/caffe-master/data/myself/imagenet_mean.binaryproto"  }# mean pixel / channel-wise mean instead of mean image#  transform_param {#    crop_size: 227#    mean_value: 104#    mean_value: 117#    mean_value: 123#    mirror: false#  }  data_param {    source: "D:/caffe_test/caffe-master/data/myself/myself_val_leveldb"    batch_size: 5    backend: LEVELDB  }}

3）设置solver.prototxt文件

net: "D:/caffe_test/caffe-master/examples/myself/train_val.prototxt"test_iter: 10test_interval: 500base_lr: 0.001lr_policy: "step"gamma: 0.1stepsize: 100000display: 20max_iter: 450000momentum: 0.9weight_decay: 0.0005snapshot: 2000snapshot_prefix: "D:/caffe_test/caffe-master/data/myself/result"solver_mode: GPU

4）复制examples/imagenet的train_caffenet.sh到examples/myself下，修改

#!/usr/bin/env shset -e./Build/x64/Release/caffe.exe train \    --solver=examples/myself/solver.prototxt $@

5、训练网络

在cygwin64中运行train_caffenet.sh，就完成了。