keras learning rate

http://machinelearningmastery.com/using-learning-rate-schedules-deep-learning-models-python-keras/

https://stackoverflow.com/questions/39779710/setting-up-a-learningratescheduler-in-keras    (打印出每一个周期的学习率lr的代码)

https://github.com/fchollet/keras/issues/898

直接使用命令设置

import keras.backend as Ksgd = SGD(lr=0.1, decay=0, momentum=0.9, nesterov=True)K.set_value(sgd.lr, 0.5 * K.get_value(sgd.lr))

或者编写回调类

class decay_lr(Callback):    '''         n_epoch = no. of epochs after decay should happen.        decay = decay value    '''      def __init__(self, n_epoch, decay):        super(decay_lr, self).__init__()        self.n_epoch=n_epoch        self.decay=decay    def on_epoch_begin(self, epoch, logs={}):        old_lr = self.model.optimizer.lr.get_value()        if epoch > 1 and epoch%self.n_epoch == 0 :            new_lr= self.decay*old_lr            k.set_value(self.model.optimizer.lr, new_lr)        else:            k.set_value(self.model.optimizer.lr, old_lr)decaySchedule=decay_lr(10, 0.95)

With TF backend, I did this (for inception-V3)

from keras.callbacks import LearningRateSchedulerdef scheduler(epoch):    if epoch%2==0 and epoch!=0:        lr = K.get_value(model.optimizer.lr)        K.set_value(model.optimizer.lr, lr*.9)        print("lr changed to {}".format(lr*.9))    return K.get_value(model.optimizer.lr)lr_decay = LearningRateScheduler(scheduler)model.fit_generator(train_gen, (nb_train_samples//batch_size)*batch_size,                  nb_epoch=100, verbose=1,                  validation_data=valid_gen,    nb_val_samples=val_size,                  callbacks=[lr_decay])以上都是以epoch为周期的，其实每一次minibatch就算一次update（例如model.train_on_batch()）， iteration的状态+1， 这就是学习率中decay作用时的iteration数值，并不等于epoch. 一个epoch有多少minibatch,就有多少iteration