Caffe学习：Solver

来源：互联网发布：苹果糯米网络不可用编辑：程序博客网时间：2024/06/06 01:53

原文
Solver
Methods
- SGD
- AdaGrad
- NAG
Scaffolding
Updating Parameters
Snapshotting and Resuming

原文

Solver

Solver通过协调Net的前向loss计算和反向梯度计算（forward inference and backward gradients），对参数进行更新，以减小loss。Caffe模型的学习被分为两个部分：由Solver进行优化、更新参数，由Net计算出loss和gradient。

Caffe提供了3种Solver方法：Stochastic Gradient Descent（SGD，随机梯度下降），Adaptive Gradient（ADAGRAD，自适应梯度下降）和Nesterov’s Accelerated Gradient（NESTEROV，Nesterov提出的加速梯度下降）。

Solve步骤：

设计优化对象、训练网络（用于学习）和测试网络（用于评估）
通过forward和backward迭代以优化和更新参数
周期性地评估测试网络
在优化过程中显示（snapshot）模型和solver的状态

每一次迭代中：

调用Net的forward pass计算出output 和loss
调用Net的backward pass计算出gradients
根据Solver方法，利用gradients更新参数
根据learning rate，history和method更新solver的状态

Caffe solvers有CPU / GPU两种模式。

Methods

Solver Methods用于最小化loss值。给定一个数据集D，优化的目标是D中所有数据的平均loss，即使平均loss取最小值。

计算公式

其中 fW(X(i))是data中X(i)项的loss，r(W)是正则项，权重为λ。当数据量很大时（直接采用普通的梯度下降方法计算量很大），在每一次迭代中，我们采用数据集的一个子集（mini-batch）来近似代替，其数据量远小于整个数据集：

计算公式

在forward pass中计算fW（即loss），在backward pass中计算∇fW（即gradient）。

SGD

类型：SGD

随机梯度下降（Stochastic gradient）利用（负的）gradient ∇L(W) 和上一次权重的更新值Vt的线性组合来更新权重W。学习速率（learning rate）α是（负的）gradient的权重。动量（momentum ）μ是上一次更新值得权重。

有如下公式，根据上一次计算的更新值Vt和当前权重来计算本次的更新值Vt+1和权重 Wt+1：

计算公式

学习的参数（α和μ）需要一定的调整才能达到最好的效果。如果你不是很清楚该怎样做的话，请看下面的“经验”，更多的细节请参考Leon Bottou的Stochastic Gradient Descent Tricks

设置学习速率（ learning rate α）和动量（momentum μ）的经验：

一个比较好的建议是，将学习速率（ learning rate α）初始化为α≈0.01，然后在训练（training）中当loss达到稳定时，将α除以一个常数（例如，10）。对于，动量（momentum μ）一般设置为μ=0.9，μ使weight的更新更为平缓，是学习过程更为稳定、快速。

这是Krizhevsky提出的技巧【文献1】，Caffe实现了SolverParameter让这个技巧易于实现。详见：./examples/imagenet/alexnet_solver.prototxt

要使用上述技巧，可以将下面的代码添加到自定义的solver prototxt文件中：

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-label" style="box-sizing: border-box;">base_lr:</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.01</span>     <span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 开始学习速率为：α = 0.01</span><span class="hljs-label" style="box-sizing: border-box;">lr_policy:</span> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"step"</span> <span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 学习策略: 每stepsize次迭代之后，将α乘以gamma</span><span class="hljs-label" style="box-sizing: border-box;">gamma:</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.1</span>        <span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 学习速率变化因子</span><span class="hljs-label" style="box-sizing: border-box;">stepsize:</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100000</span>  <span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 每100K次迭代，下降学习速率</span><span class="hljs-label" style="box-sizing: border-box;">max_iter:</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">350000</span>  <span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 训练的最大迭代次数</span><span class="hljs-label" style="box-sizing: border-box;">momentum:</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.9</span>     <span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># momentum为：μ = 0.01</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>

上面的例子中，我们将动量μ设为常数0.9。刚开始前100K次迭代时学习速率α设置为0.01，然后第100K~200K次迭代时将学习速率α乘以gamma (γ)，即为α′=αγ=(0.01)*(0.1)=0.001，之后第200K~300K次迭代时学习速率为α′′=0.00001，最后第301K~350K次迭代时学习速率为α′′=0.000001.

上述例子中，当训练次数达到一定量后，计算量会扩大到1/（1-μ）倍，所以如果增加μ的话，最好是减少α值（反之相反）。举个例子，设μ=0.9，则计算量会扩大1/（1-μ）=10倍。如果将μ扩大为0.99，那么计算量会扩大100倍，所以α应该除以10.

上述技巧也只是经验之谈，不保证绝对有用，甚至有副作用。如果训练过程中出现了发散现象（例如，loss或者output值非常大），试着减小开始学习速率（base_lr）再训练，重复这个过程，直到找到一个比较合适的学习速率。

【文献1】：A. Krizhevsky, I. Sutskever, and G. Hinton. ImageNet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems, 2012.

AdaGrad

类型：ADAGRAD

自适应梯度下降方法（ Adaptive gradient）【文献1】跟随机梯度下降（Stochastic gradient）一样是基于梯度的优化方法。给定之前更新的信息：之前的更新信息，按照下面的公式更新参数：
计算公式

实际操作中，对于权重，自适应梯度（AdaGrad ）只需要额外空间存储历史信息，而不是的存储空间来存储所有的历史纪录。

【文献1】： Duchi, E. Hazan, and Y. Singer. Adaptive Subgradient Methods for Online Learning and Stochastic Optimization. The Journal of Machine Learning Research, 2011.

NAG

类型：NESTEROV

Nesterov提出的加速梯度下降（Nesterov’s accelerated gradient）是Nesterov提出的凸优化的最佳方法【文献1】，其收敛速度可以达到，而不是。尽管在Caffe中并不是一定能够满足收敛于的条件（例如，由于非平滑non-smoothness和non-convexity非凸）实际中NAG对于某类特定结构的深度学习模型是一个非常有效的方法【文献2】。

权重weight更新参数与随机梯度下降（Stochastic gradient）非常相似：

计算公式

与SGD的不同之处在于梯度∇L(W)项中取值不同：在NAG中，对于权重W梯度的计算要机上动量=>∇L(Wt+μVt)；在SGD中，只是简单的计算当前权重的动量=>∇L(Wt)。

【文献1】：Y. Nesterov. A Method of Solving a Convex Programming Problem with Convergence Rate O(1/k√). Soviet Mathematics Doklady, 1983.
【文献2】：I. Sutskever, J. Martens, G. Dahl, and G. Hinton. On the Importance of Initialization and Momentum in Deep Learning. Proceedings of the 30th International Conference on Machine Learning, 2013.

Scaffolding

Solver::Presolve()方法：准备用于优化的方法及初始化模型。

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">> caffe train -solver examples/mnist/lenet_solver<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.prototxt</span>I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56.474978</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> caffe<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.cpp</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">90</span>] Starting OptimizationI0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56.475190</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> solver<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.cpp</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span>] Initializing solver from parameters:<span class="hljs-label" style="box-sizing: border-box;">test_iter:</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span><span class="hljs-label" style="box-sizing: border-box;">test_interval:</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">500</span><span class="hljs-label" style="box-sizing: border-box;">base_lr:</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.01</span><span class="hljs-label" style="box-sizing: border-box;">display:</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span><span class="hljs-label" style="box-sizing: border-box;">max_iter:</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10000</span><span class="hljs-label" style="box-sizing: border-box;">lr_policy:</span> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"inv"</span><span class="hljs-label" style="box-sizing: border-box;">gamma:</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.0001</span><span class="hljs-label" style="box-sizing: border-box;">power:</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.75</span><span class="hljs-label" style="box-sizing: border-box;">momentum:</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.9</span><span class="hljs-label" style="box-sizing: border-box;">weight_decay:</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.0005</span><span class="hljs-label" style="box-sizing: border-box;">snapshot:</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5000</span><span class="hljs-label" style="box-sizing: border-box;">snapshot_prefix:</span> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"examples/mnist/lenet"</span><span class="hljs-label" style="box-sizing: border-box;">solver_mode:</span> GPU<span class="hljs-label" style="box-sizing: border-box;">net:</span> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"examples/mnist/lenet_train_test.prototxt"</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li></ul>

Net initialization（网络初始化）

<code class="hljs dos has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">655681</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> solver.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">72</span>] Creating training <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span> from <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span> file: examples/mnist/lenet_train_test.prototxt[...]I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">656740</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>] Memory required <span class="hljs-flow" style="box-sizing: border-box;">for</span> data: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">656791</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">67</span>] Creating Layer mnistI0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">656811</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">356</span>] mnist -> dataI0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">656846</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">356</span>] mnist -> labelI0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">656874</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">96</span>] Setting up mnistI0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">694052</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> data_layer.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">135</span>] Opening lmdb examples/mnist/mnist_train_lmdbI0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">701062</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> data_layer.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">195</span>] output data size: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span>,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span>I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">701146</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> data_layer.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">236</span>] Initializing prefetchI0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">701196</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> data_layer.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">238</span>] Prefetch initialized.I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">701212</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">103</span>] Top shape: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">50176</span>)I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">701230</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">103</span>] Top shape: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span>)[...]I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">703737</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">67</span>] Creating Layer ip1I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">703753</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">394</span>] ip1 <- pool2I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">703778</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">356</span>] ip1 -> ip1I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">703797</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">96</span>] Setting up ip1I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728127</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">103</span>] Top shape: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">500</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32000</span>)I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728142</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">113</span>] Memory required <span class="hljs-flow" style="box-sizing: border-box;">for</span> data: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5039360</span>I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728175</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">67</span>] Creating Layer relu1I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728194</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">394</span>] relu1 <- ip1I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728219</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">345</span>] relu1 -> ip1 (<span class="hljs-flow" style="box-sizing: border-box;">in</span>-place)I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728240</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">96</span>] Setting up relu1I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728256</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">103</span>] Top shape: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">500</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32000</span>)I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728270</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">113</span>] Memory required <span class="hljs-flow" style="box-sizing: border-box;">for</span> data: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5167360</span>I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728287</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">67</span>] Creating Layer ip2I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728304</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">394</span>] ip2 <- ip1I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728333</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">356</span>] ip2 -> ip2I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728356</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">96</span>] Setting up ip2I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728690</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">103</span>] Top shape: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">640</span>)I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728705</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">113</span>] Memory required <span class="hljs-flow" style="box-sizing: border-box;">for</span> data: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5169920</span>I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728734</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">67</span>] Creating Layer lossI0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728747</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">394</span>] loss <- ip2I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728767</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">394</span>] loss <- labelI0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728786</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">356</span>] loss -> lossI0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728811</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">96</span>] Setting up lossI0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728837</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">103</span>] Top shape: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728849</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">109</span>]     with loss weight <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728878</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">113</span>] Memory required <span class="hljs-flow" style="box-sizing: border-box;">for</span> data: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5169924</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li></ul>

Loss（损失）

<code class="hljs dos has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728893</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">170</span>] loss needs backward computation.I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728909</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">170</span>] ip2 needs backward computation.I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728924</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">170</span>] relu1 needs backward computation.I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728938</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">170</span>] ip1 needs backward computation.I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728953</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">170</span>] pool2 needs backward computation.I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728970</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">170</span>] conv2 needs backward computation.I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728984</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">170</span>] pool1 needs backward computation.I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">728998</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">170</span>] conv1 needs backward computation.I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">729014</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">172</span>] mnist does <span class="hljs-flow" style="box-sizing: border-box;">not</span> need backward computation.I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">729027</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">208</span>] This network produces output lossI0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">729053</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">467</span>] Collecting Learning Rate and Weight Decay.I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">729071</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">219</span>] Network initialization done.I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">729085</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span>.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">220</span>] Memory required <span class="hljs-flow" style="box-sizing: border-box;">for</span> data: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5169924</span>I0902 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">729277</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16020</span> solver.cpp:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">156</span>] Creating test <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span> (#<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>) specified by <span class="hljs-winutils" style="color: rgb(0, 0, 136); box-sizing: border-box;">net</span> file: examples/mnist/lenet_train_test.prototxt</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li></ul>

Completion（完成）

<code class="hljs css has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">I0902</span> 13<span class="hljs-pseudo" style="color: rgb(0, 0, 0); box-sizing: border-box;">:35</span><span class="hljs-pseudo" style="color: rgb(0, 0, 0); box-sizing: border-box;">:56</span><span class="hljs-class" style="box-sizing: border-box; color: rgb(155, 112, 63);">.806970</span> 16020 <span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">solver</span><span class="hljs-class" style="box-sizing: border-box; color: rgb(155, 112, 63);">.cpp</span><span class="hljs-pseudo" style="color: rgb(0, 0, 0); box-sizing: border-box;">:46</span>] <span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">Solver</span> <span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">scaffolding</span> <span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">done</span>.<span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">I0902</span> 13<span class="hljs-pseudo" style="color: rgb(0, 0, 0); box-sizing: border-box;">:35</span><span class="hljs-pseudo" style="color: rgb(0, 0, 0); box-sizing: border-box;">:56</span><span class="hljs-class" style="box-sizing: border-box; color: rgb(155, 112, 63);">.806984</span> 16020 <span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">solver</span><span class="hljs-class" style="box-sizing: border-box; color: rgb(155, 112, 63);">.cpp</span><span class="hljs-pseudo" style="color: rgb(0, 0, 0); box-sizing: border-box;">:165</span>] <span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">Solving</span> <span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">LeNet</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

Updating Parameters

权重更新是在Solver::ComputeUpdateValue()中完成的。ComputeUpdateValue方法利用权重衰减项（weight decay）计算出权重梯度（weight gradients）（现阶段还只有误差梯度），进而计算出每个权重最后的梯度。然后经过与学习速率α相乘后，该值被存储于每一个参数（Blob的diff域）中。最后，调用每个参数的Blob::Update()方法进行最后的参数更新（Blob的data值减去diff值）。

Snapshotting and Resuming

Solver使用Solver::Snapshot()方法保存权重的值，使用Solver::SnapshotSolverState()保存状态。保存的权重表示当前学习到的模型，并且允许从该状态继续进行学习。训练可以通过Solver::Restore()方法和Solver::RestoreSolverState()方法继续进行。

权重保存文件没有拓展名，状态保存文件的拓展名是.solverstate extension，每种文件都有一个_iter_N后缀，表明保存时的迭代次数。

在Solver定义文件（*_solver.prototxt）中，保存方法是如下定义：

<code class="hljs vala has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 保存间隔（单位：迭代次数）</span>snapshot: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5000</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 保存文件的前缀，注意：是相对于caffe根目录，而不是具体Solver定义文件目录</span>snapshot_prefix: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"/path/to/model"</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 是否保存Blob的diff域（存储梯度值），有助于debug，但是耗费更多存储空间</span>snapshot_diff: <span class="hljs-literal" style="color: rgb(0, 102, 102); box-sizing: border-box;">false</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 训练完成后是否保存最后的状态，默认是True</span>snapshot_after_train: <span class="hljs-literal" style="color: rgb(0, 102, 102); box-sizing: border-box;">true</span></code>

0 0