tensorflow 学习笔记8 dropout
来源:互联网 发布:mysql utf8 utf8mb4 编辑:程序博客网 时间:2024/04/27 06:53
dropout用于解决过拟合问题。打个比方Dropout类似于性别在生物进化中的角色:物种为了生存往往会倾向于适应这种环境,环境突变则会导致物种难以做出及时反应,性别的出现可以繁衍出适应新环境的变种,有效的阻止过拟合,即避免环境改变时物种可能面临的灭绝。代码例子是sklearn数据集进行简单训练后的模型:
import tensorflow as tffrom sklearn.datasets import load_digitsfrom sklearn.model_selection import train_test_splitfrom sklearn.preprocessing import LabelBinarizer# load datadigits = load_digits()X = digits.datay = digits.targety = LabelBinarizer().fit_transform(y)X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=.3)def add_layer(inputs, in_size, out_size, layer_name, activation_function=None, ): # add one more layer and return the output of this layer Weights = tf.Variable(tf.random_normal([in_size, out_size])) biases = tf.Variable(tf.zeros([1, out_size]) + 0.1, ) Wx_plus_b = tf.matmul(inputs, Weights) + biases # here to dropout Wx_plus_b = tf.nn.dropout(Wx_plus_b, keep_prob) if activation_function is None: outputs = Wx_plus_b else: outputs = activation_function(Wx_plus_b, ) tf.summary.histogram(layer_name + '/outputs', outputs) return outputs# define placeholder for inputs to network#保持多少的结果不被dropout掉keep_prob = tf.placeholder(tf.float32)xs = tf.placeholder(tf.float32, [None, 64]) # 8x8ys = tf.placeholder(tf.float32, [None, 10])# add output layerl1 = add_layer(xs, 64, 50, 'l1', activation_function=tf.nn.tanh)prediction = add_layer(l1, 50, 10, 'l2', activation_function=tf.nn.softmax)# the loss between prediction and real datacross_entropy = tf.reduce_mean(-tf.reduce_sum(ys * tf.log(prediction), reduction_indices=[1])) # losstf.summary.scalar('loss', cross_entropy)train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)sess = tf.Session()merged = tf.summary.merge_all()# summary writer goes in heretrain_writer = tf.summary.FileWriter("桌面/logs/train", sess.graph)test_writer = tf.summary.FileWriter("桌面/logs/test", sess.graph)# tf.initialize_all_variables() no long valid frominit = tf.global_variables_initializer()sess.run(init)for i in range(500): # here to determine the keeping probability,50%dropout掉 sess.run(train_step, feed_dict={xs: X_train, ys: y_train, keep_prob: 0.5}) if i % 50 == 0: # record loss,记录的时候不需要dropout掉任何东西 train_result = sess.run(merged, feed_dict={xs: X_train, ys: y_train, keep_prob: 1}) test_result = sess.run(merged, feed_dict={xs: X_test, ys: y_test, keep_prob: 1}) train_writer.add_summary(train_result, i) test_writer.add_summary(test_result, i)在终端执行:
tensorboard --logdir=~/桌面/logs
tensorboard可视化loss(橘色:testing sets 绿色:training sets):
如果不用dropout就会出现过拟合,training sets效果特别好,而testing sets效果就不行了,这就是训练集训练的时候出现过拟合,对于测试集测试的时候误差就比较大,结果如下:
阅读全文
0 0
- tensorflow 学习笔记8 dropout
- tensorflow学习笔记(八):dropout
- Tensorflow学习笔记(用哪学哪)tf.nn.dropout
- tensorflow function笔记: dropout
- TensorFlow学习笔记(六):如何理解dropout?
- TensorFlow Dropout
- 深度学习理解tensorflow之Dropout
- PyTorch笔记8-Dropout
- 【深度学习】Dropout网络笔记
- TensorFlow学习笔记4——正则化方法:L1和L2 regularization、数据集扩增、dropout
- (七)Tensorflow学习之旅——理解dropout
- TensorFlow学习---tf.nn.dropout防止过拟合
- 【转载】TensorFlow学习---tf.nn.dropout防止过拟合
- 【Tensorflow】tf.nn.dropout函数
- TensorFlow学习笔记8----TensorFlow Linear Model Tutorial
- 【tensorflow1.0学习笔记005】dropout解决overfitting
- TensorFlow学习笔记-1
- TensorFlow学习笔记
- 判断一个二叉树是否是AVL树
- 多线程基础三、实例变量和线程安全
- JAVAWeb报错:The absolute uri: [http://java.sun.com/jstl/core] cannot be resolved in eith
- 摩天楼
- 为类的每一个对象生成一个唯一的序号
- tensorflow 学习笔记8 dropout
- Help Jimmy
- java.sql.SQLException: No suitable driver found for jdbc:mysql://localhost:3306/
- ES6学习笔记
- 【强化学习】DQN(Deep reinforcement learning) Basic
- laravel 模块化管理 插件 caffeinated
- SPI、IIC与UART的区别
- ajax上传excel
- 阿里云学习路径