Deep Learning-TensorFlow (2) CNN卷积神经网络_TensorBoard可视化使用及MNIST代码实例

环境：Win8.1 TensorFlow1.0.1

软件：Anaconda3 （集成Python3及开发环境）

TensorFlow安装：pip install tensorflow (CPU版) pip install tensorflow-gpu (GPU版)

代码参考：Lib\site-packages\tensorflow\examples\tutorials\mnist 的示例代码

为了更方便 TensorFlow 程序的理解、调试与优化，谷歌发布了一套叫做 TensorBoard 的可视化工具。你可以用 TensorBoard 来展现你的 TensorFlow 图像，绘制图像生成的定量指标图以及附加数据。

示例：

作用域

TensorFlow 在构建图表的时候，可以将每一层都创建于一个唯一的 tf.name_scope 之下，创建于该作用域之下的所有元素都将带有其前缀，例如上一篇 MNIST 的实例中，我们可以将输入的训练数据和标签创建于 input 作用域下：

  with tf.name_scope('input'):    x = tf.placeholder(tf.float32, [None, 784], name='x-input')    y_ = tf.placeholder(tf.float32, [None, 10], name='y-input')

如此在 tensorboard 生成的 GRAPHS 中我们可以观察到input节点：

Summary

TensorBoard 通过读取 TensorFlow 的事件文件来运行。TensorFlow 的事件文件包括了你会在 TensorFlow 运行中涉及到的主要数据。

首先，创建你想汇总数据的 TensorFlow 图，然后再选择你想在哪个节点进行汇总(Summary)操作。

比如，假设你正在训练一个卷积神经网络，用于识别 MNIST 标签。你可能希望记录学习速度(learning rate)的如何变化，以及目标函数如何变化。通过向节点附加 tf.summary.scalar 操作来分别输出学习速度和期望误差。然后你可以给每个 scalar 分配一个有意义的 标签，比如 'accuracy' 和 'cross_entropy'：

tf.summary.scalar('cross_entropy', cross_entropy)

如此在 TensorBoard 生成的 SCALARS 中我们可以观察到 tensor 随迭代次数的变化：

此外我们还可以对一个 tensor 变量汇总多项统计(mean, min, max, std, histogram)：

def variable_summaries(var):    """Attach a lot of summaries to a Tensor (for TensorBoard visualization)."""    with tf.name_scope('summaries'):      mean = tf.reduce_mean(var)      tf.summary.scalar('mean', mean)      with tf.name_scope('stddev'):        stddev = tf.sqrt(tf.reduce_mean(tf.square(var - mean)))      tf.summary.scalar('stddev', stddev)      tf.summary.scalar('max', tf.reduce_max(var))      tf.summary.scalar('min', tf.reduce_min(var))      tf.summary.histogram('histogram', var)

或者你还希望显示一个特殊层中激活的分布，或者梯度权重的分布。可以通过分别附加 tf.summary.histogram 运算来收集权重变量和梯度输出，并显示在 HISTOGRAMS中；或者你还可以通过 tf.summary.image在 IMAGES 中可视化输入数据或者中间结果的图像。

FileWriter

在 TensorFlow 中，所有的操作只有当你执行，或者另一个操作依赖于它的输出时才会运行。我们刚才创建的这些节点(summary nodes)都围绕着你的数据图，但没有任何操作依赖于它们的结果。因此，为了生成汇总信息，我们需要运行所有这些节点，为避免乏味的手动工作，可以使用 tf.summary.merge_all() 来将他们合并为一个操作。

执行合并命令的时候，它会依据 merge 步骤将所有数据生成一个序列化的 Summary protobuf 对象。为了将汇总数据写入磁盘，需要将汇总的 protobuf 对象传递给 tf.summary.FileWriter。FileWriter 的构造函数中包含了参数log_dir，所有事件都会写到它所指的目录下。最后通过 global_variables_initializer()  初始化所有变量：

  # Merge all the summaries and write them out to /tmp/mnist_logs (by default)  merged = tf.summary.merge_all()  train_writer = tf.summary.FileWriter(FLAGS.log_dir + '/train', sess.graph)  test_writer = tf.summary.FileWriter(FLAGS.log_dir + '/test')  tf.global_variables_initializer().run()

在所有的 summary nodes 定义完成之后，我们需要在每一次迭代中通过 add_summary 将训练或者测试得到的数据写入刚定义的 FileWriter ：

summary, _ = sess.run([merged, train_step], feed_dict=feed_dict(True))train_writer.add_summary(summary, i)      summary, acc = sess.run([merged, accuracy], feed_dict=feed_dict(False))test_writer.add_summary(summary, i)

最后迭代完成后，我们关闭 FileWriter：

  train_writer.close()  test_writer.close()

启动TensorBoard

当以上步骤顺利完成，我们可以在 log_dir 下得到汇总数据，如果你已经通过pip安装了 TensorBoard，你可以通过执行简单的命令来访问 TensorBoard，例如在CMD下：

tensorboard --logdir=C:\Users\DiamonJoy\Desktop\tensorflow\mnist-tensorboard\logs\mnist_with_summaries

一旦 TensorBoard 开始运行，你可以通过在浏览器中输入 localhost:6006 来查看 TensorBoard。进入 TensorBoard 的界面时，你会在右上角看到导航选项卡，每一个选项卡将展现一组可视化的序列化数据集 。如果 TensorBoard 中没有数据与这个选项卡相关的话，则会显示一条提示信息指示你如何序列化相关数据。

建议

浏览器选择谷歌(Chrome)最优，或者Chrome内核的浏览器，查看 EMBEDDINGS 需要 WebGL 支持。

源码(mnist_with_summaries.py)

# Copyright 2015 The TensorFlow Authors. All Rights Reserved.## Licensed under the Apache License, Version 2.0 (the 'License');# you may not use this file except in compliance with the License.# You may obtain a copy of the License at##     http://www.apache.org/licenses/LICENSE-2.0## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an 'AS IS' BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.# See the License for the specific language governing permissions and# limitations under the License.# =============================================================================="""A simple MNIST classifier which displays summaries in TensorBoard. This is an unimpressive MNIST model, but it is a good example of usingtf.name_scope to make a graph legible in the TensorBoard graph explorer, and ofnaming summary tags so that they are grouped meaningfully in TensorBoard.It demonstrates the functionality of every TensorBoard dashboard."""from __future__ import absolute_importfrom __future__ import divisionfrom __future__ import print_functionimport argparseimport sysimport tensorflow as tffrom tensorflow.examples.tutorials.mnist import input_dataFLAGS = Nonedef train():  # Import data  mnist = input_data.read_data_sets(FLAGS.data_dir,                                    one_hot=True,                                    fake_data=FLAGS.fake_data)  sess = tf.InteractiveSession()  # Create a multilayer model.  # Input placeholders  with tf.name_scope('input'):    x = tf.placeholder(tf.float32, [None, 784], name='x-input')    y_ = tf.placeholder(tf.float32, [None, 10], name='y-input')  with tf.name_scope('input_reshape'):    image_shaped_input = tf.reshape(x, [-1, 28, 28, 1])    tf.summary.image('input', image_shaped_input, 10)  # We can't initialize these variables to 0 - the network will get stuck.  def weight_variable(shape):    """Create a weight variable with appropriate initialization."""    initial = tf.truncated_normal(shape, stddev=0.1)    return tf.Variable(initial)  def bias_variable(shape):    """Create a bias variable with appropriate initialization."""    initial = tf.constant(0.1, shape=shape)    return tf.Variable(initial)  def variable_summaries(var):    """Attach a lot of summaries to a Tensor (for TensorBoard visualization)."""    with tf.name_scope('summaries'):      mean = tf.reduce_mean(var)      tf.summary.scalar('mean', mean)      with tf.name_scope('stddev'):        stddev = tf.sqrt(tf.reduce_mean(tf.square(var - mean)))      tf.summary.scalar('stddev', stddev)      tf.summary.scalar('max', tf.reduce_max(var))      tf.summary.scalar('min', tf.reduce_min(var))      tf.summary.histogram('histogram', var)  def nn_layer(input_tensor, input_dim, output_dim, layer_name, act=tf.nn.relu):    """Reusable code for making a simple neural net layer.    It does a matrix multiply, bias add, and then uses relu to nonlinearize.    It also sets up name scoping so that the resultant graph is easy to read,    and adds a number of summary ops.    """    # Adding a name scope ensures logical grouping of the layers in the graph.    with tf.name_scope(layer_name):      # This Variable will hold the state of the weights for the layer      with tf.name_scope('weights'):        weights = weight_variable([input_dim, output_dim])        variable_summaries(weights)      with tf.name_scope('biases'):        biases = bias_variable([output_dim])        variable_summaries(biases)      with tf.name_scope('Wx_plus_b'):        preactivate = tf.matmul(input_tensor, weights) + biases        tf.summary.histogram('pre_activations', preactivate)      activations = act(preactivate, name='activation')      tf.summary.histogram('activations', activations)      return activations  hidden1 = nn_layer(x, 784, 500, 'layer1')  with tf.name_scope('dropout'):    keep_prob = tf.placeholder(tf.float32)    tf.summary.scalar('dropout_keep_probability', keep_prob)    dropped = tf.nn.dropout(hidden1, keep_prob)  # Do not apply softmax activation yet, see below.  y = nn_layer(dropped, 500, 10, 'layer2', act=tf.identity)  with tf.name_scope('cross_entropy'):    # The raw formulation of cross-entropy,    #    # tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(tf.softmax(y)),    #                               reduction_indices=[1]))    #    # can be numerically unstable.    #    # So here we use tf.nn.softmax_cross_entropy_with_logits on the    # raw outputs of the nn_layer above, and then average across    # the batch.    diff = tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y)    with tf.name_scope('total'):      cross_entropy = tf.reduce_mean(diff)  tf.summary.scalar('cross_entropy', cross_entropy)  with tf.name_scope('train'):    train_step = tf.train.AdamOptimizer(FLAGS.learning_rate).minimize(        cross_entropy)  with tf.name_scope('accuracy'):    with tf.name_scope('correct_prediction'):      correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))    with tf.name_scope('accuracy'):      accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))  tf.summary.scalar('accuracy', accuracy)  # Merge all the summaries and write them out to /tmp/mnist_logs (by default)  merged = tf.summary.merge_all()  train_writer = tf.summary.FileWriter(FLAGS.log_dir + '/train', sess.graph)  test_writer = tf.summary.FileWriter(FLAGS.log_dir + '/test')  tf.global_variables_initializer().run()  # Train the model, and also write summaries.  # Every 10th step, measure test-set accuracy, and write test summaries  # All other steps, run train_step on training data, & add training summaries  def feed_dict(train):    """Make a TensorFlow feed_dict: maps data onto Tensor placeholders."""    if train or FLAGS.fake_data:      xs, ys = mnist.train.next_batch(100, fake_data=FLAGS.fake_data)      k = FLAGS.dropout    else:      xs, ys = mnist.test.images, mnist.test.labels      k = 1.0    return {x: xs, y_: ys, keep_prob: k}  for i in range(FLAGS.max_steps):    if i % 10 == 0:  # Record summaries and test-set accuracy      summary, acc = sess.run([merged, accuracy], feed_dict=feed_dict(False))      test_writer.add_summary(summary, i)      print('Accuracy at step %s: %s' % (i, acc))    else:  # Record train set summaries, and train      if i % 100 == 99:  # Record execution stats        run_options = tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE)        run_metadata = tf.RunMetadata()        summary, _ = sess.run([merged, train_step],                              feed_dict=feed_dict(True),                              options=run_options,                              run_metadata=run_metadata)        train_writer.add_run_metadata(run_metadata, 'step%03d' % i)        train_writer.add_summary(summary, i)        print('Adding run metadata for', i)      else:  # Record a summary        summary, _ = sess.run([merged, train_step], feed_dict=feed_dict(True))        train_writer.add_summary(summary, i)  train_writer.close()  test_writer.close()def main(_):  if tf.gfile.Exists(FLAGS.log_dir):    tf.gfile.DeleteRecursively(FLAGS.log_dir)  tf.gfile.MakeDirs(FLAGS.log_dir)  train()if __name__ == '__main__':  parser = argparse.ArgumentParser()  parser.add_argument('--fake_data', nargs='?', const=True, type=bool,                      default=False,                      help='If true, uses fake data for unit testing.')  parser.add_argument('--max_steps', type=int, default=1000,                      help='Number of steps to run trainer.')  parser.add_argument('--learning_rate', type=float, default=0.001,                      help='Initial learning rate')  parser.add_argument('--dropout', type=float, default=0.9,                      help='Keep probability for training dropout.')  parser.add_argument('--data_dir', type=str, default='MNIST_data',                      help='Directory for storing input data')  parser.add_argument('--log_dir', type=str, default='logs/mnist_with_summaries',                      help='Summaries log directory')  FLAGS, unparsed = parser.parse_known_args()  tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)