pytorch-fineturn the network and adjust learning rate
来源:互联网 发布:手机期货软件排行 编辑:程序博客网 时间:2024/06/05 08:24
1.
ignored_params = list(map(id, model.fc.parameters()))
base_params = filter(lambda p: id(p) not in ignored_params, model.parameters())
optimizer = torch.optim.SGD([
{'params': base_params},
{'params': model.fc.parameters(), 'lr': opt.lr}
], lr=opt.lr*0.1, momentum=0.9)
2. adjust learning rate
def adjust_lr(optimizer, e, base_lr=base_lr, step_size=step_size):
_lr = base_lr * 0.1 ** ( e // step_size)
for param_group in optimizer.param_groups:
param_group['lr'] = _ lr
return optimizer
def adjust_lr(optimizer, e, step_size=step_size, decay=0.1):
if e==0 or e%step_size:
return optimizer
for param_group in optimizer.param_groups:
param_group['lr'] *= decay
return optimizer
3. lr_scheduler
scheduler = optim.lr_scheduler( optimizer, xxx)
scheduler.step() # per epoch update onece
for I in range(epoch):
scheduler.step()
train(i)
valid()
------------------------------------------reference-----------------------------
1. https://discuss.pytorch.org/t/how-to-perform-finetuning-in-pytorch/419/7
2. https://discuss.pytorch.org/t/adaptive-learning-rate/320/4
- pytorch-fineturn the network and adjust learning rate
- MDF结合Learning rate adjust应用
- weight decay and learning rate
- Momentum and Learning Rate Adaptation
- pytorch学习笔记(十):learning rate decay(学习率衰减)
- pytorch学习笔记(十):learning rate decay(学习率衰减)
- 解释Learning Rate在Neural Network的作用
- learning rate
- The Difference between Bit Rate and Baud Rate
- Neural Network and Deep Learning
- Batch Gradiant Descent - Sample Magnitute and Learning Rate
- 学习率Learning rate
- keras learning rate
- Learning Rate Annealing
- Plot Learning Rate
- 学习率 Learning Rate
- neural network and deep learning (1)
- neural network and deep learning (2)
- 使用endnote在word中插入参考文献
- 【Redis学习】:hash数据类型详解
- 2017年五月二十九日 NO.3 HTML(二)——基本文档元素(笔记)
- 动态规划——72. Edit Distance[HARD]
- 【Redis学习】:list数据类型详解
- pytorch-fineturn the network and adjust learning rate
- 数据结构(图--邻接矩阵法实现)
- 【Redis学习】:set数据类型详解
- 旧版漏洞 Bug 被发现可致 Win7/8.1 “崩溃”
- 数据库连接池
- python3 类的学习
- (179)编辑材质函数
- sqlite数据定义语言
- 【Redis学习】:sortedset数据类型详解