tensorflow代码全解析 -1- TensorBoard 入门案例

本文概要：通过mnist识别案例讲解TensorFlow中TensorBoard的使用方法

源代码

本人源代码
https://github.com/zhuyin521/Tensorflow-work_example
原作者代码
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/tutorials/mnist/mnist_with_summaries.py

TensorBoard概述

TensorBoard 可以将模型训练过程中的各种汇总数据展示出来。包括

• 标量 Scalars - tf.summary.scalar
• 图片 Images - tf.summary.image
• 音频 Audio
• 计算图 Graphs
• 数据分布 Distributions
• 直方图 Histogram
• 嵌入向量 Embeddings
  ——这些向量将会经常用到
  suammary 节点需要专门去运行才能起作用，使用tf.summary.merge_all可以将所有summary节点合并成一个节点，只要运行这个节点，就能产生之前设置的所有summary
  使用tf.summary.FileWriter将运行后输出的数据都保存到本地磁盘中

启动程序后，在使用命令行进入相对应目录输入tensorboard指定，才可以查看可视化文件

使用TensorBoard 展示数据需要在执行Tensoflow计算图的过程中，将各类数据汇总并记录到日志文件中，然后在使用tensorBoard读取这些日志文件，解析并生产数据可视化的web页面。

代码框架

1. 读取mnist数据集 read_data_sets()，定义初始化参数方法,定义输入数据feed_dict()
   
   1. 定义输入数据 x,xs
   2. 定义输人标签 y,ys
2. 定义数据汇总方法 variable_summaries()
3. 创建神经网络框架 nn_layer（）
   
   1. 第一层 hidden1 = nn_layer(x,784,500,’layer1’)
   2. dropout层 dropped = tf.nn.droupout(hidden1 )
   3. 第二层 y = nn_layer(dropped, 500, 10, ‘layer2’, act = tf.identity)
   4. 输出 y
4. 创建损失函数 cross_entropy
5. 创建训练优化器 AdamOptimizer
6. 定义准确度tf.summary.scalar.correct_ prediction
   合并所有summary节点merged = tf.summary.merge_all()
7. 模型训练 sess.run

运行环境

操作系统

• win10
• python 3.5
• tensorflow-gpu 1.0.0

注意事项及BUG

1 . BUG 运行中间发生Python执行非法指令错误，同时运行窗口报告：
Couldn’t open CUDA library cupti64_80.dll无法继续执行
原因：CUDA的cupti64_80.dll的路径没有加入PATH
解决办法：
将目录C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\extras\CUPTI\libx64下的cupti64_80.dll 复制到C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin即可
参考：

I have encountered this problem before. When you use CUDA 8.0,the file cupti64_80.dll lies in C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\extras\CUPTI\libx64. I just fixed the problem by copying the dll into C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin, and the file cupti.lib in the same location into C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\lib\x64. And it works!

2 . 初始化目录问题及启动tensorboard

parser.add_argument('--data_dir', type=str, default='input_data',                        help='Directory for storing input data')parser.add_argument('--log_dir', type=str, default='logs',                        help='Summaries logs directory')

目录设置好后，要在自己代码所在的文件夹里面新建这两个文件夹，如果没有新建自己不会主动创建，反正我的是没有，有的人是可以
调用tensorboard要进入目标命令行里面

tensorboard --logdir=logs --debug

要开启 debug 模式 就可以看到是不是读取了日志文件
这个地方头疼了好久，按照如上设置就可以正确的在网页上显示

3 . BUG 报如下错误

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'input/x-input' with dtype float     [[Node: input/x-input = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/gpu:0"]()]]     [[Node: layer2_1/weights/summaries/stddev/Sqrt/_21 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_710_layer2_1/weights/summaries/stddev/Sqrt", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]

这个问题花费了非常非常的时间去解决，多方查找资料
因为是自己一个代码一个代码敲进去的，所以反复核对了好几遍，但都没有发现问题
原因：因为spyder运行一次session后会组织其再次运行
解决办法：在命令行列里面运行就可以了
参考：

I find out that once you run it once in spyder it prevents you from runing it again on the same session

这个问题告诉我们，有时候真的不是代码写错了，是IDE出错了
只要是人做的东西都可能出错，犯错的总不一定是自己

代码详解

这个代码是原作者的代码
我自己写的比较碎片化

from __future__ import absolute_importfrom __future__ import divisionfrom __future__ import print_functionimport argparseimport sysimport tensorflow as tffrom tensorflow.examples.tutorials.mnist import input_dataFLAGS = Nonedef train():    # 读取数据    mnist = input_data.read_data_sets(FLAGS.data_dir,                                      one_hot=True,                                      fake_data=FLAGS.fake_data)    # 启动默认回话    sess = tf.InteractiveSession()    # 创建模型    # 创建输入数据的占位符，分别创建特征数据x，标签数据y_    with tf.name_scope('input'):        x = tf.placeholder(tf.float32, [None, 784], name='x-input')        y_ = tf.placeholder(tf.float32, [None, 10], name='y-input')    # 准备输入数据和训练数据    # 如果train=true，从mnist.train中取一个batch样本，设置dropout值；    # 如果train==false,获取minist.test的测试数据，设置keep_prob为1，即保留所有神经元开启。    def feed_dict(train):        """Make a TensorFlow feed_dict: maps data onto Tensor placeholders."""        if train or FLAGS.fake_data:            xs, ys = mnist.train.next_batch(100, fake_data=FLAGS.fake_data)            k = FLAGS.dropout        else:            xs, ys = mnist.test.images, mnist.test.labels            k = 1.0        return {x: xs, y_: ys, keep_prob: k}    # 使用summary.image记录图片，要注意需要转换成对应的格式    with tf.name_scope('input_reshape'):        image_shaped_input = tf.reshape(x, [-1, 28, 28, 1])        tf.summary.image('input', image_shaped_input, 10)    # We can't initialize these variables to 0 - the network will get stuck.    # 我们初始默认参数，不能设置为零，初始化为零会难以收敛    # w 采用 truncated_normal 函数进行初始化一个标准差的正态分布    # b 0.1初始化就可以    def weight_variable(shape):        """Create a weight variable with appropriate initialization."""        initial = tf.truncated_normal(shape, stddev=0.1)        return tf.Variable(initial)    def bias_variable(shape):        """Create a bias variable with appropriate initialization."""        initial = tf.constant(0.1, shape=shape)        return tf.Variable(initial)    # 记录每一次迭代的参数信息    def variable_summaries(var):        """Attach a lot of summaries to a Tensor (for TensorBoard visualization)."""        with tf.name_scope('summaries'):            mean = tf.reduce_mean(var)            tf.summary.scalar('mean', mean)            # 记录参数的标准差            with tf.name_scope('stddev'):                stddev = tf.sqrt(tf.reduce_mean(tf.square(var - mean)))            tf.summary.scalar('stddev', stddev)            tf.summary.scalar('max', tf.reduce_max(var))            tf.summary.scalar('min', tf.reduce_min(var))            tf.summary.histogram('histogram', var)    # 构建神经网络    # 应该明确输入参数    # input_tensor：特征数据    # input_dim：输入数据的维度大小    # output_dim：输出数据的维度大小( = 隐层神经元个数）    # layer_name：命名空间    # act = tf.nn.relu：激活函数（默认是relu)    # 该神经网络是一个MLP多层神经网络，每一层会对模型参数进行数据汇总tf.summary.histogram    def nn_layer(input_tensor, input_dim, output_dim, layer_name, act=tf.nn.relu):        """Reusable code for making a simple neural net layer.        It does a matrix multiply, bias add, and then uses ReLU to nonlinearize.        It also sets up name scoping so that the resultant graph is easy to read,        and adds a number of summary ops.        """        # Adding a name scope ensures logical grouping of the layers in the graph.        with tf.name_scope(layer_name):            # This Variable will hold the state of the weights for the layer            with tf.name_scope('weights'):                weights = weight_variable([input_dim, output_dim])                variable_summaries(weights)            with tf.name_scope('biases'):                biases = bias_variable([output_dim])                variable_summaries(biases)            # y = wx +b            with tf.name_scope('Wx_plus_b'):                preactivate = tf.matmul(input_tensor, weights) + biases                tf.summary.histogram('pre_activations', preactivate)            # 调用激励函数对数据进行响应            # result = relu(y)            activations = act(preactivate, name='activation')            tf.summary.histogram('activations', activations)            return activations    # 隐藏层 输入数据维度784 输出维度500    hidden1 = nn_layer(x, 784, 500, 'layer1')    # dropout 随机删除一些神经元，参数 keep_prob    with tf.name_scope('dropout'):        keep_prob = tf.placeholder(tf.float32)        tf.summary.scalar('dropout_keep_probability', keep_prob)        dropped = tf.nn.dropout(hidden1, keep_prob)    # 输出层 输入数据500维 输出类别 10    y = nn_layer(dropped, 500, 10, 'layer2', act=tf.identity)    # 创建损失函数 y 模型输出 y_ 数据标签    with tf.name_scope('cross_entropy'):        diff = tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y)        with tf.name_scope('total'):            cross_entropy = tf.reduce_mean(diff)    tf.summary.scalar('cross_entropy', cross_entropy)    # 使用AdamOptimizer优化器训练模型，最小化交叉熵损失    with tf.name_scope('train'):        train_step = tf.train.AdamOptimizer(FLAGS.learning_rate).minimize(            cross_entropy)    # 计算准确率，并用tf.summary 进行合并    with tf.name_scope('accuracy'):        with tf.name_scope('correct_prediction'):            correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))        with tf.name_scope('accuracy'):            accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))    tf.summary.scalar('accuracy', accuracy)    # 将所有summary合并，这个直接在后面的session.run()里面运行    merged = tf.summary.merge_all()    # 日志数据存放位置    # 定义两个不同的文件记录器，分别存放训练和测试数据的日志数据，这样就可以不互相干扰    train_writer = tf.summary.FileWriter(FLAGS.log_dir + '/train', sess.graph)    test_writer = tf.summary.FileWriter(FLAGS.log_dir + '/test')    tf.global_variables_initializer().run()    # Train the model, and also write summaries.    # Every 10th step, measure test-set accuracy, and write test summaries    # All other steps, run train_step on training data, & add training summaries    # 开始训练，每十次，进行一次数据汇总，并打印测试数据的准确率，并将测试数据中的参数写入日志    # 每100次，记录元信息    for i in range(FLAGS.max_steps):        if i % 10 == 0:  # Record summaries and test-set accuracy            summary, acc = sess.run([merged, accuracy], feed_dict=feed_dict(False))            test_writer.add_summary(summary, i)            print('Accuracy at step %s: %s' % (i, acc))        else:  # Record train set summaries, and train            if i % 100 == 99:  # Record execution stats                run_options = tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE)                run_metadata = tf.RunMetadata()                summary, _ = sess.run([merged, train_step],                                      feed_dict=feed_dict(True),                                      options=run_options,                                      run_metadata=run_metadata)                # 记录训练运算时间和内存占用                train_writer.add_run_metadata(run_metadata, 'step%03d' % i)                train_writer.add_summary(summary, i)                print('Adding run metadata for', i)            else:  # Record a summary                summary, _ = sess.run([merged, train_step], feed_dict=feed_dict(True))                train_writer.add_summary(summary, i)    # 关闭文件记录器    train_writer.close()    test_writer.close()def main(_):    if tf.gfile.Exists(FLAGS.log_dir):        tf.gfile.DeleteRecursively(FLAGS.log_dir)    tf.gfile.MakeDirs(FLAGS.log_dir)    train()# 初始化参数if __name__ == '__main__':    parser = argparse.ArgumentParser()    parser.add_argument('--fake_data', nargs='?', const=True, type=bool,                        default=False,                        help='If true, uses fake data for unit testing.')    parser.add_argument('--max_steps', type=int, default=100000,                        help='Number of steps to run trainer.')    parser.add_argument('--learning_rate', type=float, default=0.001,                        help='Initial learning rate')    parser.add_argument('--dropout', type=float, default=0.9,                        help='Keep probability for training dropout.')    # 这个要注意 主要要在自己的文件夹里面新建好    parser.add_argument('--data_dir', type=str, default='input_data',                        help='Directory for storing input data')    parser.add_argument('--log_dir', type=str, default='logs',                        help='Summaries logs directory')    FLAGS, unparsed = parser.parse_known_args()    tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)