caffe常见优化器使用参数

caffe中solver不同优化器的一些使用方法(只记录一些常用的)

下面是一些公用的参数

测试时需要前向传播的次数,比如你有1000个数据,批处理大小为10,那么这个值就应该是100,这样才能够将所有的数据覆盖
test_iter: 100
每多少次迭代进行一次测试.
test_interval: 500

weight_decay防止过拟合的参数,使用方式:
1 样本越多,该值越小
2 模型参数越多,该值越大
weight_decay: 0.0005

rmsprop:net: "examples/mnist/lenet_train_test.prototxt"test_iter: 100test_interval: 500#The base learning rate, momentum and the weight decay of the network.base_lr: 0.01momentum: 0.0weight_decay: 0.0005#The learning rate policylr_policy: "inv"gamma: 0.0001power: 0.75display: 100max_iter: 10000snapshot: 5000snapshot_prefix: "examples/mnist/lenet_rmsprop"solver_mode: GPUtype: "RMSProp"rms_decay: 0.98Adam:net: "examples/mnist/lenet_train_test.prototxt"test_iter: 100test_interval: 500#All parameters are from the cited paper abovebase_lr: 0.001momentum: 0.9momentum2: 0.999#since Adam dynamically changes the learning rate, we set the base learning#rate to a fixed valuelr_policy: "fixed"display: 100#The maximum number of iterationsmax_iter: 10000snapshot: 5000snapshot_prefix: "examples/mnist/lenet"type: "Adam"solver_mode: GPUmultistep:net: "examples/mnist/lenet_train_test.prototxt"test_iter: 100test_interval: 500#The base learning rate, momentum and the weight decay of the network.base_lr: 0.01momentum: 0.9weight_decay: 0.0005#The learning rate policylr_policy: "multistep"gamma: 0.9stepvalue: 5000stepvalue: 7000stepvalue: 8000stepvalue: 9000stepvalue: 9500# Display every 100 iterationsdisplay: 100#The maximum number of iterationsmax_iter: 10000#snapshot intermediate resultssnapshot: 5000snapshot_prefix: "examples/mnist/lenet_multistep"#solver mode: CPU or GPUsolver_mode: GPU