caffe solver configuration

来源：互联网发布：淘宝二手手机靠谱么编辑：程序博客网时间：2024/05/17 09:47

(用到一个加一个, 并非完整的介绍)

lr_policy

基本的learning rate 在solver.prototxt中由参数base_lr配置.
配合lr_policy和其余的一些参数制定learning rate的变化策略.

lr_policy="fixed"

在整个训练过程中learning rate不变.

lr_policy="step"

需要另外几个参数配合:

base_lr: 0.01     # begin training at a learning rate of 0.01 = 1e-2lr_policy: "step" # learning rate policy: drop the learning rate in "stepsize"                  # by a factor of gamma every stepsize iterationsgamma: 0.1        # drop the learning rate by a factor of 10                  # (i.e., multiply it by a factor of gamma = 0.1)stepsize: 100000  # drop the learning rate every 100K iterations

average_loss

相当于做了一个平滑. 控制台打印训练loss时, 当前loss为最近20个iteration的loss的平均数. 仅仅是为了显示好看, 不影响训练.

http://stackoverflow.com/questions/40190377/what-is-average-loss-field-in-caffe-solver-for

iter_size

在显存不够用时很管用.
它产生的效果是forward iter_size次后才backpropogate一次, 相当于将batch_size增大了iter_size倍.

简单来说, real batch_size = batch_size * iter_size.
每执行一次solver.step(1), 会执行batch_size * iter_size次forward与1次backward.

https://www.zhihu.com/question/37270367

max_iter

最大iteration次数. 但如果是通过solver.step(n)来forward-backward, 这个配置是无效的.
例如以下代码, total iterations = 100 * 10 = 1000

for _ in xrange(100):    solver.step(10)

http://caffe.berkeleyvision.org/tutorial/solver.html

0 0