Caffe编译 Mnist训练测试

Caffe编译

进入目录/caffe-master/，输入下列命令编译整个caffe工程

make cleanmake all 

补充说明：很多添加编译选项，据说是为利用cpu的4个核并行编译加快编译效率，然而这里编译时报错所以就make all。其实make all 编译些什么可以看同目录下的Makefile文件。
补充说明：在make all 之前需要修改Makefile.config.example文件具体修改说明如下：

## Refer to http://caffe.berkeleyvision.org/installation.html# Contributions simplifying and improving our build system are welcome!# cuDNN acceleration switch (uncomment to build with cuDNN).# USE_CUDNN := 1//CuDNN是NVIDIA 出的专门针对Deep Learning框架设计的一套GPU计算加速库，用于实现高性能的并行计算。在有GPU并且安装cudnn的情况下可以打开即将注释去掉。# CPU-only switch (uncomment to build without GPU support).#CPU_ONLY := 1//表示是否用gpu如果只有cpu这里要打开，如果只有一个gpu建议不要打开，测试过只有一个gpu的情况下，速度并未得到提升反而更慢，当然可以打开gpu尝试下啦～# uncomment to disable IO dependencies and corresponding data layersUSE_OPENCV := 1# USE_LEVELDB := 0# USE_LMDB := 0//因为要用到opencv库所以要打开，下面这两个选项表示是选择caffe的数据管理第三方库，两者都不打开 caffe默认用的是LMDB，这两者均是嵌入式数据库管理系统编程库，详细介绍：http://www.cnblogs.com/yymn/p/4479216.html# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)#   You should not set this flag if you will be reading LMDBs with any#   possibility of simultaneous read and write# ALLOW_LMDB_NOLOCK := 1//打开这个注释是当需要读取LMDB文件时，默认不打开，所以不取消注释# Uncomment if you're using OpenCV 3OPENCV_VERSION := 2.4.10//用pkg-config --modversion opencv命令查看opencv版本# To customize your choice of compiler, uncomment and set the following.# N.B. the default for Linux is g++ and the default for OSX is clang++# CUSTOM_CXX := g++//linux系统一般用得都是g++编译器# CUDA directory contains bin/ and lib/ directories that we need.CUDA_DIR := /usr/local/cuda//cuda的安装目录# On Ubuntu 14.04, if cuda tools are installed via# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:# CUDA_DIR := /usr# CUDA architecture setting: going with all of them.# For CUDA < 6.0, comment the *_50 lines for compatibility.CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \        -gencode arch=compute_20,code=sm_21 \        -gencode arch=compute_30,code=sm_30 \        -gencode arch=compute_35,code=sm_35 \        -gencode arch=compute_50,code=sm_50 \        -gencode arch=compute_50,code=compute_50//这些是指gpu的计算能力，6.0以下的版本不支持×_50的计算能力，这里有个链接介绍：http://www.tuicool.com/articles/qUN322z# BLAS choice:# atlas for ATLAS (default)# mkl for MKL# open for OpenBlasBLAS := open//BLAS这个值如果用的是atlas计算库则赋值ATLAS，mkl计算库则用MKL赋值，OpenBlas则赋值open# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.# Leave commented to accept the defaults for your choice of BLAS# (which should work)!BLAS_INCLUDE := /usr/local/OpenBlas/includeBLAS_LIB := /usr/local/OpenBlas/lib//blas库安装目录# Homebrew puts openblas in a directory that is not on the standard search path# BLAS_INCLUDE := $(shell brew --prefix openblas)/include# BLAS_LIB := $(shell brew --prefix openblas)/lib//如果不是安装在标准路径则要指明# This is required only if you will compile the matlab interface.# MATLAB directory should contain the mex binary in /bin.# MATLAB_DIR := /usr/local# MATLAB_DIR := /Applications/MATLAB_R2012b.app//matlab安装库的目录# NOTE: this is required only if you will compile the python interface.# We need to be able to find Python.h and numpy/arrayobject.h.PYTHON_INCLUDE := /usr/include/python2.7 \        /usr/lib/python2.7/dist-packages/numpy/core/include//python安装目录# Anaconda Python distribution is quite popular. Include path:# Verify anaconda location, sometimes it's in root.# ANACONDA_HOME := $(HOME)/anaconda# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \        # $(ANACONDA_HOME)/include/python2.7 \        # $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \# Uncomment to use Python 3 (default is Python 2)# PYTHON_LIBRARIES := boost_python3 python3.5m# PYTHON_INCLUDE := /usr/include/python3.5m \#                 /usr/lib/python3.5/dist-packages/numpy/core/include# We need to be able to find libpythonX.X.so or .dylib.PYTHON_LIB := /usr/lib//python库位置# PYTHON_LIB := $(ANACONDA_HOME)/lib# Homebrew installs numpy in a non standard path (keg only)# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include# PYTHON_LIB += $(shell brew --prefix numpy)/lib# Uncomment to support layers written in Python (will link against Python libs)WITH_PYTHON_LAYER := 1# Whatever else you find you need goes here.INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/includeLIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies# INCLUDE_DIRS += $(shell brew --prefix)/include# LIBRARY_DIRS += $(shell brew --prefix)/lib# Uncomment to use `pkg-config` to specify OpenCV library paths.# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)# USE_PKG_CONFIG := 1# N.B. both build and distribute dirs are cleared on `make clean`BUILD_DIR := buildDISTRIBUTE_DIR := distribute# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171# DEBUG := 1# The ID of the GPU that 'make runtest' will use to run unit tests.TEST_GPUID := 0//所用的gpu的ID编号# enable pretty build (comment to see full commands)Q ?= @

这样改动之后还是会报错？？？哈哈哈，那是因为Makefile引用的不是这个编译配置文件，把Makefile里第二行改成：CONFIG_FILE := Makefile.config.example

Mnist训练

首先进入/caffe-master/目录，在当前目录下执行下列命令：

1、./data/mnist/get_mnist.sh

获得mnist数据训训练的4个数据包

2、./examples/mnist/create_mnist.sh

利用caffe-master/build/examples/mnist/的convert_mnist_data.bin工具，将mnist date转化为可用的lmdb格式的文件。并将新生成的2个文件mnist-train-lmdb 和 mnist-test-lmdb放于create_mnist.sh同目录下。

3、./examples/mnist/train_lenet.sh

训练数据，下面再看下train_lenet.sh中 –solver=examples/mnist/lenet_solver.prototxt ，这个文件定义了具体的训练参数

# The train/test net protocol buffer definitionnet: "examples/mnist/lenet_train_test.prototxt"//网络协议具体定义# test_iter specifies how many forward passes the test should carry out.# In the case of MNIST, we have test batch size 100 and 100 test iterations,# covering the full 10,000 testing images.test_iter: 100//test迭代次数 如果batch_size =100,则100张图一批，训练100次，则可以覆盖10000张图的需求# Carry out testing every 500 training iterations.test_interval: 500//训练迭代500次，测试一次# The base learning rate, momentum and the weight decay of the network.base_lr: 0.01momentum: 0.9weight_decay: 0.0005//网络参数：学习率，动量，权重的衰减# The learning rate policylr_policy: "inv"gamma: 0.0001power: 0.75//学习策略：有固定学习率和每步递减学习率# Display every 100 iterationsdisplay: 100//每迭代100次显示一次# The maximum number of iterationsmax_iter: 10000//最大迭代次数# snapshot intermediate resultssnapshot: 5000//每5000次迭代存储一次数据，路径前缀是<</span>span style="font-family: Arial, Helvetica, sans-serif;">examples/mnist/lenet</</span>span>snapshot_prefix: "examples/mnist/lenet"# solver mode: CPU or GPUsolver_mode: GPU//是否使用GPU还是CPU

下面看看文件lenet_solver.prototxt ，这个文件定义了具体的lenet网络参数

name: "LeNet"           网络名layer {  name: "mnist"         本层名称  type: "Data"              层类型  top: "data"               下一层接口  top: "label"              下一层接口  include {    phase: TRAIN  }  transform_param {    scale: 0.00390625           #1/256，预处理如减均值，尺寸变换，随机剪，镜像等  }  data_param {    source: "examples/mnist/mnist_train_lmdb"   训练数据位置    batch_size: 64                  一次训练的样本数    backend: LMDB                   读入的训练数据格式，默认leveldb  }}layer {  name: "mnist"  type: "Data"  top: "data"  top: "label"  include {    phase: TEST  }  transform_param {    scale: 0.00390625  }  data_param {    source: "examples/mnist/mnist_test_lmdb"    batch_size: 100                 一次测试使用100个数据    backend: LMDB  }}layer {  name: "conv1"  type: "Convolution"               卷积层  bottom: "data"                上一层名“data”  top: "conv1"                  下一层接口“conv1”  param {    lr_mult: 1                  （weights的学习率与全局相同）  }  param {    lr_mult: 2                  （biases的学习率是全局的2倍）  }  convolution_param {    num_output: 20              卷积核20个    kernel_size: 5              卷积核尺寸5×5    stride: 1                   步长1    weight_filler {      type: "xavier"                （随机的初始化权重和偏差）    }    bias_filler {      type: "constant"              bias用0初始化    }  }}layer {  name: "pool1"  type: "Pooling"               池化层  bottom: "conv1"               上层“conv1”  top: "pool1"                  下层接口“pool1”  pooling_param {    pool: MAX                   池化函数用MAX    kernel_size: 2              池化核函数大小2×2    stride: 2                   步长2  }}layer {  name: "conv2"  type: "Convolution"  bottom: "pool1"  top: "conv2"  param {    lr_mult: 1  }  param {    lr_mult: 2  }  convolution_param {    num_output: 50              卷积核50个    kernel_size: 5    stride: 1    weight_filler {      type: "xavier"    }    bias_filler {      type: "constant"    }  }}layer {  name: "pool2"  type: "Pooling"  bottom: "conv2"  top: "pool2"  pooling_param {    pool: MAX    kernel_size: 2    stride: 2  }}layer {  name: "ip1"  type: "InnerProduct"              全连接层  bottom: "pool2"               上层连接“pool2”  top: "ip1"                    “下层输出接口ip1”  param {    lr_mult: 1  }  param {    lr_mult: 2  }  inner_product_param {    num_output: 500             输出数量500    weight_filler {      type: "xavier"    }    bias_filler {      type: "constant"    }  }}layer {  name: "relu1"  type: "ReLU"              激活函数  bottom: "ip1"  top: "ip1"    （这个地方还是ip1，底层与顶层相同减少开支，下一层全连接层的输入也还是ip1）}layer {  name: "ip2"  type: "InnerProduct"  bottom: "ip1"  top: "ip2"  param {    lr_mult: 1  }  param {    lr_mult: 2  }  inner_product_param {    num_output: 10              输出结果10个    weight_filler {      type: "xavier"    }    bias_filler {      type: "constant"    }  }}layer {  name: "accuracy"  type: "Accuracy"  bottom: "ip2"         上层连接ip2全连接层  bottom: "label"           上层连接label层  top: "accuracy"           输出接口为accuracy  include {    phase: TEST          }}layer {  name: "loss"  type: "SoftmaxWithLoss"       损失函数  bottom: "ip2"  bottom: "label"  top: "loss"}

4、./build/tools/caffe.bin test -model=examples/mnist/lenet_train_test.prototxt -weights=examples/mnist/lenet_iter_10000.caffemodel -gpu=0   或者不使用GPU       ./build/tools/caffe.bin test -model=examples/mnist/lenet_train_test.prototxt -weights=examples/mnist/lenet_iter_10000.caffemodel

所有数据都训练好之后，接下来就是如何将模型应用到实际数据了。test：表示对训练好的模型进行Testing，而不是training。其他参数包括train, time, device_query。
-model=XXX：指定模型prototxt文件，这是一个文本文件，详细描述了网络结构和数据集信息。

下面解释下caffe程序的命令行格式

caffe的c++主程序（caffe.cpp)放在根目录下的tools文件夹内, 当然还有一些其它的功能文件，如：convert_imageset.cpp, train_net.cpp, test_net.cpp等也放在这个文件夹内。经过编译后，这些文件都被编译成了可执行文件，放在了 ./build/tools/ 文件夹内。因此我们要执行caffe程序，都需要加 ./build/tools/ 前缀。如上述的train_lenet.sh脚本文件:

sudo sh ./build/tools/caffe train --solver=examples/mnist/train_lenet.sh

caffe程序的命令行执行格式如下：

caffe <command> <args>

其中command命令有如下四种：

1. train：：：训练或finetune模型（model)
2. test ：：：测试模型
3. device_query：：：显示gpu信息
4. time：：：显示程序执行时间

其中args参数有如下几种：

-solver：必选参数。一个protocol buffer类型的文件，即模型的配置文件。：

./build/tools/caffe train -solver examples/mnist/lenet_solver.prototxt

-gpu: 可选参数。该参数用来指定用哪一块gpu运行，根据gpu的id进行选择，如果设置为’-gpu all’则使用所有的gpu运行。如使用第二块gpu运行：

./build/tools/caffe train -solver examples/mnist/lenet_solver.prototxt -gpu 2

-snapshot:可选参数。该参数用来从快照（snapshot)中恢复训练。可以在solver配置文件设置快照，保存solverstate。如：

./build/tools/caffe train -solver examples/mnist/lenet_solver.prototxt -snapshot examples/mnist/lenet_iter_5000.solverstate

-weights:可选参数。用预先训练好的权重来fine-tuning模型，需要一个caffemodel，不能和-snapshot同时使用。如：

./build/tools/caffe train -solver examples/finetuning_on_flickr_style/solver.prototxt -weights models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel

-iterations: 可选参数，迭代次数，默认为50。 如果在配置文件文件中没有设定迭代次数，则默认迭代50次。
-model:可选参数，定义在protocol buffer文件中的模型。也可以在solver配置文件中指定。
-sighup_effect：可选参数。用来设定当程序发生挂起事件时，执行的操作，可以设置为snapshot, stop或none, 默认为snapshot
-sigint_effect: 可选参数。用来设定当程序发生键盘中止事件时（ctrl+c), 执行的操作，可以设置为snapshot, stop或none, 默认为stop

test参数用在测试阶段，用于最终结果的输出，要模型配置文件中我们可以设定需要输入accuracy还是loss. 假设我们要在验证集中验证已经训练好的模型，就可以这样写

./build/tools/caffe test -model examples/mnist/lenet_train_test.prototxt -weights examples/mnist/lenet_iter_10000.caffemodel -gpu 0 -iterations 100

这个例子比较长，不仅用到了test参数，还用到了-model, -weights, -gpu和-iteration四个参数。意思是利用训练好了的权重（-weight)，输入到测试模型中(-model)，用编号为0的gpu(-gpu)测试100次(-iteration)。

time参数用来在屏幕上显示程序运行时间。如：

./build/tools/caffe time -model examples/mnist/lenet_train_test.prototxt -iterations 10

这个例子用来在屏幕上显示lenet模型迭代10次所使用的时间。包括每次迭代的forward和backward所用的时间，也包括每层forward和backward所用的平均时间。

device_query参数用来诊断gpu信息：

./build/tools/caffe device_query -gpu 0

会显示如下关于gpu信息

Querying GPUs 0Device id:                     0Major revision number:         2Minor revision number:         1Name:                          GeForce GT 630Total global memory:           2146631680Total shared memory per block: 49152Total registers per block:     32768Warp size:                     32Maximum memory pitch:          2147483647Maximum threads per block:     1024Maximum dimension of block:    1024, 1024, 64Maximum dimension of grid:     65535, 65535, 65535Clock rate:                    1500000Total constant memory:         65536Texture alignment:             512Concurrent copy and execution: YesNumber of multiprocessors:     2Kernel execution timeout:      Yes

因为我只有一块gpu所以用-gpu运行时速度反而更慢