（Caffe）基本类Solver、Caffe、Batch（二）

本文地址：http://blog.csdn.net/mounty_fsc/article/details/51088173

1 Solver

1.1 简介

其对网络进行求解，其作用有：

1. 提供优化日志支持、创建用于学习的训练网络、创建用于评估的测试网络
2. 通过调用forward / backward迭代地优化，更新权值
3. 周期性地评估测试网络
4. 通过优化了解model及solver的状态

1.2 源代码

<code class="language-c++ hljs cpp has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;"><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">/** * @brief An interface for classes that perform optimization on Net%s. * * Requires implementation of ApplyUpdate to compute a parameter update * given the current state of the Net parameters. */</span><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">template</span> <<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">typename</span> Dtype><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">class</span> Solver { <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span>:  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">explicit</span> Solver(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> SolverParameter& param,      <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> Solver* root_solver = NULL);  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">explicit</span> Solver(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">string</span>& param_file, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> Solver* root_solver = NULL);  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> Init(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> SolverParameter& param);  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> InitTrainNet();  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> InitTestNets(); ...  <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// The main entry of the solver function. In default, iter will be zero. Pass</span>  <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// in a non-zero iter number to resume training for a pre-trained net.</span>  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> Solve(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">char</span>* resume_file = NULL);  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">inline</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> Solve(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">string</span> resume_file) { Solve(resume_file.c_str()); }  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> Step(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> iters);... <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">protected</span>:  <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// Make and apply the update value for the current iteration.</span>  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> ApplyUpdate() = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>;  ...  SolverParameter param_;  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> iter_;  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> current_step_;  <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">shared_ptr</span><Net<Dtype> > net_;  <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">shared_ptr</span><Net<Dtype></span> > > test_nets_;  <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Callback*></span> callbacks_;  <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Dtype></span> losses_;  Dtype smoothed_loss_;  <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// The root solver that holds root nets (actually containing shared layers)</span>  <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// in data parallelism</span>  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> Solver* <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> root_solver_;...};</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li></ul>

说明：

1. shared_ptr<Net<Dtype>> net_为训练网络的指针，vector<shared_ptr<Net<Dtype>>> test_nets为测试网络的指针组，可见测试网络可以有多个

2. 一般来说训练网络跟测试网络在实现上会有区别，但是绝大部分网络层是相同的。

3. 不同的模型训练方法通过重载函数ComputeUpdateValue( )实现计算update参数的核心功能

4. caffe.cpp中的train( )函数训练模型，在这里实例化一个Solver对象，初始化后调用了Solver中的Solve( )方法。而这个Solve( )函数主要就是在迭代运行下面这两个函数。ComputeUpdateValue(); 
net_->Update();

1.3 Solver的方法

• Stochastic Gradient Descent (type: “SGD”)
• AdaDelta (type: “AdaDelta”)
• Adaptive Gradient (type: “AdaGrad”)
• Adam (type: “Adam”)
• Nesterov’s Accelerated Gradient (type: “Nesterov”)
• RMSprop (type: “RMSProp”)

详细参见引用1

2 Caffe类

Caffe类为一个包含常用的caffe成员的单例类。如caffe使用的cuda库cublas,curand的句柄等，以及生成Caffe中的随机数等。

<code class="language-c++ hljs cs has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;"><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// common.hpp</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// A singleton class to hold common caffe stuff, such as the handler that</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// caffe is going to use for cublas, curand, etc.</span>class Caffe { <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span>:  ~<span class="hljs-title" style="box-sizing: border-box;">Caffe</span>();  <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// Thread local context for Caffe. Moved to common.cpp instead of</span>  <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// including boost/thread.hpp to avoid a boost/NVCC issues (#1009, #1010)</span>  <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// on OSX. Also fails on Linux with CUDA 7.0.18.</span>  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">static</span> Caffe& Get();  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">enum</span> Brew { CPU, GPU };...<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">protected</span>:#ifndef CPU_ONLY  cublasHandle_t cublas_handle_;  curandGenerator_t curand_generator_;<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#<span class="hljs-keyword" style="box-sizing: border-box;">endif</span></span>  shared_ptr<RNG> random_generator_;  Brew mode_;  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> solver_count_;  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">bool</span> root_solver_; <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">private</span>:  // The <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">private</span> constructor to avoid duplicate instantiation.  <span class="hljs-title" style="box-sizing: border-box;">Caffe</span>();  DISABLE_COPY_AND_ASSIGN(Caffe);};</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li></ul>

<code class="language-c++ hljs r has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">//common.cppnamespace caffe {// Make sure each thread can have different values.static boost::thread_specific_ptr<Caffe> thread_instance_;Caffe& Caffe::Get() {  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">if</span> (!thread_instance_.get()) {    thread_instance_.reset(new Caffe());  }  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> *(thread_instance_.get());}<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">...</span>Caffe::Caffe()    : cublas_handle_(<span class="hljs-literal" style="color: rgb(0, 102, 102); box-sizing: border-box;">NULL</span>), curand_generator_(<span class="hljs-literal" style="color: rgb(0, 102, 102); box-sizing: border-box;">NULL</span>), random_generator_(),    mode_(Caffe::CPU), solver_count_(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>), root_solver_(true) {  // Try to create a cublas handler, and report an error <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">if</span> failed (but we will  // keep the program running as one might just want to run CPU code).  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">if</span> (cublasCreate(&cublas_handle_) != CUBLAS_STATUS_SUCCESS) {    LOG(ERROR) << <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Cannot create Cublas handle. Cublas won't be available."</span>;  }  // Try to create a curand handler.  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">if</span> (curandCreateGenerator(&curand_generator_, CURAND_RNG_PSEUDO_DEFAULT)      != CURAND_STATUS_SUCCESS ||      curandSetPseudoRandomGeneratorSeed(curand_generator_, cluster_seedgen())      != CURAND_STATUS_SUCCESS) {    LOG(ERROR) << <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Cannot create Curand generator. Curand won't be available."</span>;  }}<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">...</span>}  // namespace caffe</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li></ul>

说明：

1. Caffe类为一个单例类，构造方法私有
2. 该单例由static boost::thread_specific_ptr<Caffe> thread_instance_维护，确保多线程环境下，不同的线程有不同的Caffe单例版本
3. 获取该单例由Get()方法执行，即Caffe::Get()方法返回thread_instance_维护的单例，
4. thread_instance_的初值为NULL，若是第一次访问，则new Caffe()
5. new Caffe()执行构造方法，其实只是创建了cublas,curand的句柄
6. 单步调试可发现cublasCreate()创建cublas的句柄，生成了额外的两个线程

3 Batch

<code class="language-c++ hljs css has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;"><span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">template</span> <<span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">typename</span> <span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">Dtype</span>><span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">class</span> <span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">Batch</span> <span class="hljs-rules" style="box-sizing: border-box;">{ <span class="hljs-rule" style="box-sizing: border-box;"><span class="hljs-attribute" style="box-sizing: border-box;">public</span>:<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 102, 102);">  Blob<Dtype> data_, label_</span></span>;<span class="hljs-rule" style="box-sizing: border-box;">}</span></span>;</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul>

说明：

• Batch是对一个样本的封装，与Datum不同，Datum是面向数据库的，且一个Datum对应一个样本（图像、标签）；而Batch是面向网络的，一个Batch对应一批样本

[1].http://caffe.berkeleyvision.org/tutorial/solver.html