tensorflow05 《TensorFlow实战Google深度学习框架》笔记-04-03学习率设置
来源:互联网 发布:斗龙战士1玩具淘宝 编辑:程序博客网 时间:2024/06/09 14:50
# 《TensorFlow实战Google深度学习框架》04 深层神经网络# win10 Tensorflow1.0.1 python3.5.3# CUDA v8.0 cudnn-8.0-windows10-x64-v5.1# filename:ts04.03.py 学习率设置# 假设我们要最小化函数 $y=x^2$, 选择初始点$x_0=5$# 1. 学习率为1的时候,x在5和-5之间震荡import tensorflow as tfTRAINING_STEPS = 10LEARNING_RATE = 1x = tf.Variable(tf.constant(5, dtype=tf.float32), name="x")y = tf.square(x)train_op = tf.train.GradientDescentOptimizer(LEARNING_RATE).minimize(y)with tf.Session() as sess: sess.run(tf.global_variables_initializer()) for i in range(TRAINING_STEPS): sess.run(train_op) x_value = sess.run(x) print("After %s iteration(s): x%s is %f."% (i+1, i+1, x_value))'''After 1 iteration(s): x1 is -5.000000.After 2 iteration(s): x2 is 5.000000.After 3 iteration(s): x3 is -5.000000.After 4 iteration(s): x4 is 5.000000.After 5 iteration(s): x5 is -5.000000.After 6 iteration(s): x6 is 5.000000.After 7 iteration(s): x7 is -5.000000.After 8 iteration(s): x8 is 5.000000.After 9 iteration(s): x9 is -5.000000.After 10 iteration(s): x10 is 5.000000.'''# 2. 学习率为0.001的时候,下降速度过慢,在901轮时才收敛到0.823355TRAINING_STEPS = 1000LEARNING_RATE = 0.001x = tf.Variable(tf.constant(5, dtype=tf.float32), name="x")y = tf.square(x)train_op = tf.train.GradientDescentOptimizer(LEARNING_RATE).minimize(y)with tf.Session() as sess: sess.run(tf.global_variables_initializer()) for i in range(TRAINING_STEPS): sess.run(train_op) if i % 100 == 0: x_value = sess.run(x) print("After %s iteration(s): x%s is %f."% (i+1, i+1, x_value))'''After 1 iteration(s): x1 is 4.990000.After 101 iteration(s): x101 is 4.084646.After 201 iteration(s): x201 is 3.343555.After 301 iteration(s): x301 is 2.736923.After 401 iteration(s): x401 is 2.240355.After 501 iteration(s): x501 is 1.833880.After 601 iteration(s): x601 is 1.501153.After 701 iteration(s): x701 is 1.228794.After 801 iteration(s): x801 is 1.005850.After 901 iteration(s): x901 is 0.823355.'''# 3. 使用指数衰减的学习率,在迭代初期得到较高的下降速度,可以在较小的训练轮数下取得不错的收敛程度TRAINING_STEPS = 100global_step = tf.Variable(0)LEARNING_RATE = tf.train.exponential_decay(0.1, global_step, 1, 0.96, staircase=True)x = tf.Variable(tf.constant(5, dtype=tf.float32), name="x")y = tf.square(x)train_op = tf.train.GradientDescentOptimizer(LEARNING_RATE).minimize(y, global_step=global_step)with tf.Session() as sess: sess.run(tf.global_variables_initializer()) for i in range(TRAINING_STEPS): sess.run(train_op) if i % 10 == 0: LEARNING_RATE_value = sess.run(LEARNING_RATE) x_value = sess.run(x) print("After %s iteration(s): x%s is %f, learning rate is %f."% (i+1, i+1, x_value, LEARNING_RATE_value))'''After 1 iteration(s): x1 is 4.000000, learning rate is 0.096000.After 11 iteration(s): x11 is 0.690561, learning rate is 0.063824.After 21 iteration(s): x21 is 0.222583, learning rate is 0.042432.After 31 iteration(s): x31 is 0.106405, learning rate is 0.028210.After 41 iteration(s): x41 is 0.065548, learning rate is 0.018755.After 51 iteration(s): x51 is 0.047625, learning rate is 0.012469.After 61 iteration(s): x61 is 0.038558, learning rate is 0.008290.After 71 iteration(s): x71 is 0.033523, learning rate is 0.005511.After 81 iteration(s): x81 is 0.030553, learning rate is 0.003664.After 91 iteration(s): x91 is 0.028727, learning rate is 0.002436.'''
0 0
- tensorflow05 《TensorFlow实战Google深度学习框架》笔记-04-03学习率设置
- Tensorflow实战Google深度学习框架 笔记
- tensorflow02 《TensorFlow实战Google深度学习框架》笔记-03
- tensorflow03 《TensorFlow实战Google深度学习框架》笔记-04-01
- Tensorflow实战Google深度学习框架-学习笔记
- tensorflow26《TensorFlow实战Google深度学习框架》笔记-10-03 分布式TensorFlow code
- TensorFlow实战Google深度学习框架(一)
- 关于《TensorFlow 实战Google深度学习框架》
- TensorFlow实战Google深度学习框架
- tensorflow14《TensorFlow实战Google深度学习框架》笔记-06-03 迁移学习 code
- tensorflow06 《TensorFlow实战Google深度学习框架》笔记-04-04正则化
- tensorflow10 《TensorFlow实战Google深度学习框架》笔记-05-03模型持久化code
- tensorflow18《TensorFlow实战Google深度学习框架》笔记-08-03 SKlearn封装例子 code
- tensorflow22《TensorFlow实战Google深度学习框架》笔记-09-03 TensorBoard改造后的mnist_train code
- tensorflow04 《TensorFlow实战Google深度学习框架》笔记-04-02自定义损失函数
- tensorflow07 《TensorFlow实战Google深度学习框架》笔记-04-05滑动平均模型
- tensorflow11 《TensorFlow实战Google深度学习框架》笔记-05-04mnist-nn最佳实践code
- tensorflow19《TensorFlow实战Google深度学习框架》笔记-08-04 预测正弦函数 code
- 【NOI2017】无限棋盘
- 运算符重载
- MFC中CFileDialog的用法 如何设置文件类型
- jok
- Python 学习笔记19
- tensorflow05 《TensorFlow实战Google深度学习框架》笔记-04-03学习率设置
- 蓝桥杯 历届试题 李白打酒
- 嵌入式NandFlash和NorFlash详解
- git的一些基本操作
- 分类排序 同辈元素只在数据上的层级关系
- java中的静态初始化块
- 自定义WEB MVC框架 三 自定义配置和响应处理
- bzoj1972: [Sdoi2010]猪国杀
- linux CentOS 安装rz和sz命令 lrzsz