tensorboard的可视化及模型可视化

待整理

How to Check-Point Deep Learning Models in Keras

---

LossWise
Tensorboard 中文社区

---

谷歌发布TensorBoard API，让你自定义机器学习中的可视化

---

查找tensorflow安装的位置

---

pip show tensorflow-gpu

Name: tensorflow-gpuVersion: 1.0.1Summary: TensorFlow helps the tensors flowHome-page: http://tensorflow.org/Author: Google Inc.Author-email: [e-mail]: Apache 2.0Location: /home/bids/.local/lib/python2.7/site-packagesRequires: mock, numpy, protobuf, wheel, six

或者

pythonimport tensorflowtensorflow.__path__dir(tensorflow)

终止某个程序的进程

pkill -f "tensorboard"

---

什么是 TensorBoard

---

TensorBoard 是 TensorFlow 上一个非常酷的功能，神经网络很多时候就像是个黑盒子，里面到底是什么样，是什么样的结构，是怎么训练的，可能很难搞清楚。而 TensorBoard 的作用就是可以把复杂的神经网络训练过程给可视化，可以更好地理解，调试并优化程序。
TensorBoard可以将训练过程中的各种绘制数据展示出来，包括标量（scalars），图片（images），音频（Audio）,计算图（graph）,数据分布，直方图（histograms）和嵌入式向量。

在 scalars 下可以看到 accuracy，cross entropy，dropout，layer1 和 layer2 的 bias 和 weights 等的趋势。在 images 和 audio 下可以看到输入的数据。展示训练过程中记录的图像和音频。在 graphs 中可以看到模型的结构。在 histogram 可以看到 activations，gradients 或者 weights 等变量的每一步的分布，越靠前面就是越新的步数的结果。展示训练过程中记录的数据的分布图distribution 和 histogram 是两种不同的形式，可以看到整体的状况。在 embedding 中可以看到用 PCA 主成分分析方法将高维数据投影到 3D 空间后的数据的关系。Event: 展示训练过程中的统计数据（最值，均值等）变化情况

使用TensorBoard展示数据，需要在执行Tensorflow就算图的过程中，将各种类型的数据汇总并记录到日志文件中。然后使用TensorBoard读取这些日志文件，解析数据并生产数据可视化的Web页面，让我们可以在浏览器中观察各种汇总数据。

---

汇总数据的日志－－callbacks回调执行结果

---

log_filepath = '/tmp/keras_log' model.compile(loss='categorical_crossentropy', optimizer=SGD(lr=0.001), metrics=['accuracy'])    tb_cb = keras.callbacks.TensorBoard(log_dir=log_filepath, write_images=1, histogram_freq=1)  # 设置log的存储位置，将网络权值以图片格式保持在tensorboard中显示，设置每一个周期计算一次网络的  #权值，每层输出值的分布直方图  cbks = [tb_cb]  history = model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch,   verbose=1, callbacks=cbks, validation_data=(X_test, Y_test))  

tensorboard　默认的slcar一栏只记录了训练集和验证集上的loss，如何想记录展示其他指标，在model.compile的metric中进行添加，例如：

model.compile(        loss = 'mean_squared_error',        optimizer = 'sgd',        metrics= c('mae', 'acc')  # 可视化mae和acc      )  

而在keras的call back模块中,tensorborad class类实现源码可以看出，keras默认将模型的所有层的所有weights, bias以及每一层输出的distribution, histogram等传送到tensorborad，方便在浏览器中观察网络的运行情况。实现源码如下：

    def set_model(self, model):          self.model = model          self.sess = K.get_session()          if self.histogram_freq and self.merged is None:              for layer in self.model.layers:                  for weight in layer.weights:                      tf.summary.histogram(weight.name, weight)                      if self.write_images:                          w_img = tf.squeeze(weight)                          shape = w_img.get_shape()                          if len(shape) > 1 and shape[0] > shape[1]:                              w_img = tf.transpose(w_img)                          if len(shape) == 1:                              w_img = tf.expand_dims(w_img, 0)                          w_img = tf.expand_dims(tf.expand_dims(w_img, 0), -1)                          tf.summary.image(weight.name, w_img)                  if hasattr(layer, 'output'):                      tf.summary.histogram('{}_out'.format(layer.name),                                           layer.output)          self.merged = tf.summary.merge_all()  

---

可视化结果

---

python /home/bids/.local/lib/python2.7/site-packages/tensorboard/tensorboard.py --logdir='/tmp/keras_log'

Starting TensorBoard 54 at http://bids:6006(Press CTRL+C to quit)

---

训练误差和测试误差在同一个图中（tensorflow）

---

具体参看Windows下tensorflow的tensorboard的使用

TensorBoard: Visualizing Learning

使用Tensorboard查看训练过程

TensorFlow深度学习笔记 Tensorboard入门
Tensorflow 自带可视化Tensorboard使用方法 附项目代码

学习TensorFlow，TensorBoard可视化网络结构和参数

tensorflow中如何进行可视化和减轻过拟合

TensorFlow-7-TensorBoard Embedding可视化

from __future__ import absolute_import  from __future__ import division  from __future__ import print_function  import argparse  import sys  import tensorflow as tf  from tensorflow.examples.tutorials.mnist import input_data  FLAGS = None  def train():    # Import data    mnist = input_data.read_data_sets(FLAGS.data_dir,                                      one_hot=True,                                      fake_data=FLAGS.fake_data)    sess = tf.InteractiveSession()    # Create a multilayer model.    # Input placeholders    with tf.name_scope('input'):      x = tf.placeholder(tf.float32, [None, 784], name='x-input')      y_ = tf.placeholder(tf.float32, [None, 10], name='y-input')    with tf.name_scope('input_reshape'):      image_shaped_input = tf.reshape(x, [-1, 28, 28, 1])      tf.summary.image('input', image_shaped_input, 10)    # We can't initialize these variables to 0 - the network will get stuck.    def weight_variable(shape):      """Create a weight variable with appropriate initialization."""      initial = tf.truncated_normal(shape, stddev=0.1)      return tf.Variable(initial)    def bias_variable(shape):      """Create a bias variable with appropriate initialization."""      initial = tf.constant(0.1, shape=shape)      return tf.Variable(initial)    def variable_summaries(var):      """Attach a lot of summaries to a Tensor (for TensorBoard visualization)."""      with tf.name_scope('summaries'):        mean = tf.reduce_mean(var)        tf.summary.scalar('mean', mean)        with tf.name_scope('stddev'):          stddev = tf.sqrt(tf.reduce_mean(tf.square(var - mean)))        tf.summary.scalar('stddev', stddev)        tf.summary.scalar('max', tf.reduce_max(var))        tf.summary.scalar('min', tf.reduce_min(var))        tf.summary.histogram('histogram', var)    def nn_layer(input_tensor, input_dim, output_dim, layer_name, act=tf.nn.relu):      """Reusable code for making a simple neural net layer.      It does a matrix multiply, bias add, and then uses relu to nonlinearize.      It also sets up name scoping so that the resultant graph is easy to read,      and adds a number of summary ops.      """      # Adding a name scope ensures logical grouping of the layers in the graph.      with tf.name_scope(layer_name):        # This Variable will hold the state of the weights for the layer        with tf.name_scope('weights'):          weights = weight_variable([input_dim, output_dim])          variable_summaries(weights)        with tf.name_scope('biases'):          biases = bias_variable([output_dim])          variable_summaries(biases)        with tf.name_scope('Wx_plus_b'):          preactivate = tf.matmul(input_tensor, weights) + biases          tf.summary.histogram('pre_activations', preactivate)        activations = act(preactivate, name='activation')        tf.summary.histogram('activations', activations)        return activations    hidden1 = nn_layer(x, 784, 500, 'layer1')    with tf.name_scope('dropout'):      keep_prob = tf.placeholder(tf.float32)      tf.summary.scalar('dropout_keep_probability', keep_prob)      dropped = tf.nn.dropout(hidden1, keep_prob)    # Do not apply softmax activation yet, see below.    y = nn_layer(dropped, 500, 10, 'layer2', act=tf.identity)    with tf.name_scope('cross_entropy'):      # The raw formulation of cross-entropy,      #      # tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(tf.softmax(y)),      #                               reduction_indices=[1]))      #      # can be numerically unstable.      #      # So here we use tf.nn.softmax_cross_entropy_with_logits on the      # raw outputs of the nn_layer above, and then average across      # the batch.      diff = tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y)      with tf.name_scope('total'):        cross_entropy = tf.reduce_mean(diff)    tf.summary.scalar('cross_entropy', cross_entropy)    with tf.name_scope('train'):      train_step = tf.train.AdamOptimizer(FLAGS.learning_rate).minimize(          cross_entropy)    with tf.name_scope('accuracy'):      with tf.name_scope('correct_prediction'):        correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))      with tf.name_scope('accuracy'):        accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))    tf.summary.scalar('accuracy', accuracy)    # Merge all the summaries and write them out to /tmp/tensorflow/mnist/logs/mnist_with_summaries (by default)    merged = tf.summary.merge_all()    train_writer = tf.summary.FileWriter(FLAGS.log_dir + '/train', sess.graph)    test_writer = tf.summary.FileWriter(FLAGS.log_dir + '/test')    tf.global_variables_initializer().run()    # Train the model, and also write summaries.    # Every 10th step, measure test-set accuracy, and write test summaries    # All other steps, run train_step on training data, & add training summaries    def feed_dict(train):      """Make a TensorFlow feed_dict: maps data onto Tensor placeholders."""      if train or FLAGS.fake_data:        xs, ys = mnist.train.next_batch(100, fake_data=FLAGS.fake_data)        k = FLAGS.dropout      else:        xs, ys = mnist.test.images, mnist.test.labels        k = 1.0      return {x: xs, y_: ys, keep_prob: k}    for i in range(FLAGS.max_steps):      if i % 10 == 0:  # Record summaries and test-set accuracy        summary, acc = sess.run([merged, accuracy], feed_dict=feed_dict(False))        test_writer.add_summary(summary, i)        print('Accuracy at step %s: %s' % (i, acc))      else:  # Record train set summaries, and train        if i % 100 == 99:  # Record execution stats          run_options = tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE)          run_metadata = tf.RunMetadata()          summary, _ = sess.run([merged, train_step],                                feed_dict=feed_dict(True),                                options=run_options,                                run_metadata=run_metadata)          train_writer.add_run_metadata(run_metadata, 'step%03d' % i)          train_writer.add_summary(summary, i)          print('Adding run metadata for', i)        else:  # Record a summary          summary, _ = sess.run([merged, train_step], feed_dict=feed_dict(True))          train_writer.add_summary(summary, i)    train_writer.close()    test_writer.close()  def main(_):    if tf.gfile.Exists(FLAGS.log_dir):      tf.gfile.DeleteRecursively(FLAGS.log_dir)    tf.gfile.MakeDirs(FLAGS.log_dir)    train()  if __name__ == '__main__':    parser = argparse.ArgumentParser()    parser.add_argument('--fake_data', nargs='?', const=True, type=bool,                        default=False,                        help='If true, uses fake data for unit testing.')    parser.add_argument('--max_steps', type=int, default=1000,                        help='Number of steps to run trainer.')    parser.add_argument('--learning_rate', type=float, default=0.001,                        help='Initial learning rate')    parser.add_argument('--dropout', type=float, default=0.9,                        help='Keep probability for training dropout.')    parser.add_argument('--data_dir', type=str, default='/tmp/tensorflow/mnist/input_data',                        help='Directory for storing input data')    parser.add_argument('--log_dir', type=str, default='/tmp/tensorflow/mnist/logs/mnist_with_summaries',                        help='Summaries log directory')    FLAGS, unparsed = parser.parse_known_args()    tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)

上面是官方给出的例子实现，下面我给出要修改的地方：

    parser.add_argument('--log_dir', type=str, default='/tmp/tensorflow/mnist/logs/mnist_with_summaries',                           help='Summaries log directory')  

修改后的代码：

parser.add_argument('--log_dir', type=str, default='C:/tmp/tensorflow/mnist/logs/mnist_with_summaries',                      help='Summaries log directory')

这里目录取决你放在哪个盘，这里我放在C盘，打开cmd。在终端输入，如下图：

tensorboard --logdir= C:\tmp\tensorflow\mnist\logs\mnist_with_summaries

打开Google Chrome输入localhost:6006，结果如下图

---

Display Deep Learning Model Training History in Keras

---

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])# Fit the modelhistory = model.fit(X, Y, validation_split=0.33, epochs=150, batch_size=10, verbose=0)# list all data in historyprint(history.history.keys())# summarize history for accuracyplt.plot(history.history['acc'])plt.plot(history.history['val_acc'])plt.title('model accuracy')plt.ylabel('accuracy')plt.xlabel('epoch')plt.legend(['train', 'test'], loc='upper left')plt.tight_layout()plt.savefig('accuracyVSepoch.png')plt.show()# summarize history for lossplt.plot(history.history['loss'])plt.plot(history.history['val_loss'])plt.title('model loss')plt.ylabel('loss')plt.xlabel('epoch')plt.legend(['train', 'test'], loc='upper left')plt.tight_layout()plt.savefig('lossVSepoch.png')plt.show()

就会得到如下类似的图

---

Keras输出的loss，val这些值如何保存到文本中去

---

hist=model.fit(train_set_x,train_set_y,batch_size=256,shuffle=True,nb_epoch=nb_epoch,validation_split=0.1)with open('log_sgd_big_32.txt','w') as f:    f.write(str(hist.history))

---

同时可视化多个模型运行结果

---

第一个模型的运行结果

log_filepath = '/tmp/keras_log/run_a' model.compile(loss='categorical_crossentropy', optimizer=SGD(lr=0.001), metrics=['accuracy'])    tb_cb = keras.callbacks.TensorBoard(log_dir=log_filepath, write_images=1, histogram_freq=1)  # 设置log的存储位置，将网络权值以图片格式保持在tensorboard中显示，设置每一个周期计算一次网络的  #权值，每层输出值的分布直方图  batch_size=10,cbks = [tb_cb],  history = model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch,   verbose=1, callbacks=cbks, validation_data=(X_test, Y_test))  

第二个模型的运行结果

log_filepath = '/tmp/keras_log/run_b' model.compile(loss='categorical_crossentropy', optimizer=adam(lr=0.001), metrics=['accuracy'])    tb_cb = keras.callbacks.TensorBoard(log_dir=log_filepath, write_images=1, histogram_freq=1)  # 设置log的存储位置，将网络权值以图片格式保持在tensorboard中显示，设置每一个周期计算一次网络的  #权值，每层输出值的分布直方图batchsize=8  cbks = [tb_cb],  history = model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch,   verbose=1, callbacks=cbks, validation_data=(X_test, Y_test))  

可视化结果

python /home/bids/.local/lib/python2.7/site-packages/tensorboard/tensorboard.py --logdir='/tmp/keras_log'

Starting TensorBoard 54 at http://bids:6006(Press CTRL+C to quit)

就会看到如下类似的图像

---

实时可视化

---

jiandanjinxin－tensorflow-mnist-tutorial
没有博士学位如何玩转TensorFlow和深度学习

Visualizing MNIST: An Exploration of Dimensionality Reduction
Visualizing Representations: Deep Learning and Human Beings

---

Hyperparameter Search

---

jiandanjinxin-tf-dev-summit-tensorboard-tutoria

TensorFlow-dev-summit：TF发展历史以及有趣的应用

---

经典示例 1

---

#-*- coding: UTF-8 -*-from __future__ import divisionfrom __future__ import print_functionfrom __future__ import unicode_literalsimport gzipimport structimport numpy as npfrom sklearn.linear_model import LogisticRegressionfrom sklearn import preprocessingfrom sklearn.metrics import accuracy_scoreimport tensorflow as tf# MNIST data is stored in binary format, # and we transform them into numpy ndarray objects by the following two utility functionsdef read_image(file_name):    with gzip.open(file_name, 'rb') as f:        buf = f.read()        index = 0        magic, images, rows, columns = struct.unpack_from('>IIII' , buf , index)        index += struct.calcsize('>IIII')        image_size = '>' + str(images*rows*columns) + 'B'        ims = struct.unpack_from(image_size, buf, index)        im_array = np.array(ims).reshape(images, rows, columns)        return im_arraydef read_label(file_name):    with gzip.open(file_name, 'rb') as f:        buf = f.read()        index = 0        magic, labels = struct.unpack_from('>II', buf, index)        index += struct.calcsize('>II')        label_size = '>' + str(labels) + 'B'        labels = struct.unpack_from(label_size, buf, index)        label_array = np.array(labels)        return label_arrayprint("Start processing MNIST handwritten digits data...")

train_x_data = read_image("MNIST_data/train-images-idx3-ubyte.gz")print(type(train_x_data),train_x_data.dtype, train_x_data.shape)print(train_x_data.max(), train_x_data.min()) train_y_data = read_label("MNIST_data/train-labels-idx1-ubyte.gz")print(type(train_y_data),train_y_data.dtype, train_y_data.shape)print(train_y_data.max(), train_y_data.min())test_x_data = read_image("MNIST_data/t10k-images-idx3-ubyte.gz")print(type(test_x_data),test_x_data.dtype, test_x_data.shape)print(test_x_data.max(), test_x_data.min()) test_x_data = test_x_data.reshape(test_x_data.shape[0], -1).astype(np.float32)print(type(test_x_data),test_x_data.dtype, test_x_data.shape)print(test_x_data.max(), test_x_data.min()) test_y_data = read_label("MNIST_data/t10k-labels-idx1-ubyte.gz")print(type(test_y_data),test_y_data.dtype, test_y_data.shape)print(test_y_data.max(), test_y_data.min())

train_x_minmax = train_x_data / 255.0test_x_minmax = test_x_data / 255.0# We evaluate the softmax regression model by sklearn firsteval_sklearn = Trueif eval_sklearn:    print("Start evaluating softmax regression model by sklearn...")    reg = LogisticRegression(solver="lbfgs", multi_class="multinomial")    reg.fit(train_x_minmax, train_y_data)    #np.savetxt('coef_softmax_sklearn.txt', reg.coef_, fmt='%.6f')  # Save coefficients to a text file    test_y_predict = reg.predict(test_x_minmax)    print("Accuracy of test set: %f" % accuracy_score(test_y_data, test_y_predict))eval_tensorflow = Truebatch_gradient = Falsedef variable_summaries(var):    with tf.name_scope('summaries'):        mean = tf.reduce_mean(var)        tf.summary.scalar('mean', mean)        stddev = tf.sqrt(tf.reduce_mean(tf.square(var - mean)))        tf.summary.scalar('stddev', stddev)        tf.summary.scalar('max', tf.reduce_max(var))        tf.summary.scalar('min', tf.reduce_min(var))        tf.summary.histogram('histogram', var)

if eval_tensorflow:    print("Start evaluating softmax regression model by tensorflow...")    # reformat y into one-hot encoding style    lb = preprocessing.LabelBinarizer()    lb.fit(train_y_data)    train_y_data_trans = lb.transform(train_y_data)    test_y_data_trans = lb.transform(test_y_data)    x = tf.placeholder(tf.float32, [None, 784])    with tf.name_scope('weights'):        W = tf.Variable(tf.zeros([784, 10]))        variable_summaries(W)    with tf.name_scope('biases'):        b = tf.Variable(tf.zeros([10]))        variable_summaries(b)    with tf.name_scope('Wx_plus_b'):        V = tf.matmul(x, W) + b        tf.summary.histogram('pre_activations', V)    with tf.name_scope('softmax'):        y = tf.nn.softmax(V)        tf.summary.histogram('activations', y)    y_ = tf.placeholder(tf.float32, [None, 10])    with tf.name_scope('cross_entropy'):        loss = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))        tf.summary.scalar('cross_entropy', loss)    with tf.name_scope('train'):        optimizer = tf.train.GradientDescentOptimizer(0.5)        train = optimizer.minimize(loss)    with tf.name_scope('evaluate'):        with tf.name_scope('correct_prediction'):            correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))        with tf.name_scope('accuracy'):            accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))            tf.summary.scalar('accuracy', accuracy)    init = tf.global_variables_initializer()    sess = tf.Session()    sess.run(init)    merged = tf.summary.merge_all()    train_writer = tf.summary.FileWriter('/tmp/log/train', sess.graph)    test_writer = tf.summary.FileWriter('/tmp/log/test')    if batch_gradient:        for step in range(300):            sess.run(train, feed_dict={x: train_x_minmax, y_: train_y_data_trans})            if step % 10 == 0:                print("Batch Gradient Descent processing step %d" % step)        print("Finally we got the estimated results, take such a long time...")    else:        for step in range(1000):            if step % 10 == 0:                summary, acc = sess.run([merged, accuracy], feed_dict={x: test_x_minmax, y_: test_y_data_trans})                test_writer.add_summary(summary, step)                print("Stochastic Gradient Descent processing step %d accuracy=%.2f" % (step, acc))            else:                sample_index = np.random.choice(train_x_minmax.shape[0], 100)                batch_xs = train_x_minmax[sample_index, :]                batch_ys = train_y_data_trans[sample_index, :]                summary, _ = sess.run([merged, train], feed_dict={x: batch_xs, y_: batch_ys})                train_writer.add_summary(summary, step)    #np.savetxt('coef_softmax_tf.txt', np.transpose(sess.run(W)), fmt='%.6f')  # Save coefficients to a text file    print("Accuracy of test set: %f" % sess.run(accuracy, feed_dict={x: test_x_minmax, y_: test_y_data_trans}))

转自：TensorFlow学习笔记（7）：TensorBoard——Tensor与Graph可视化

---

经典示例 2

---

import tensorflow as tfimport numpy as nptf.set_random_seed(1)np.random.seed(1)# fake datax = np.linspace(-1, 1, 100)[:, np.newaxis]          # shape (100, 1)noise = np.random.normal(0, 0.1, size=x.shape)y = np.power(x, 2) + noise                          # shape (100, 1) + some noisewith tf.variable_scope('Inputs'):    tf_x = tf.placeholder(tf.float32, x.shape, name='x')    tf_y = tf.placeholder(tf.float32, y.shape, name='y')with tf.variable_scope('Net'):    l1 = tf.layers.dense(tf_x, 10, tf.nn.relu, name='hidden_layer')    output = tf.layers.dense(l1, 1, name='output_layer')    # add to histogram summary    tf.summary.histogram('h_out', l1)    tf.summary.histogram('pred', output)loss = tf.losses.mean_squared_error(tf_y, output, scope='loss')train_op = tf.train.GradientDescentOptimizer(learning_rate=0.5).minimize(loss)tf.summary.scalar('loss', loss)     # add loss to scalar summarysess = tf.Session()sess.run(tf.global_variables_initializer())writer = tf.summary.FileWriter('./log', sess.graph)     # write to filemerge_op = tf.summary.merge_all()                       # operation to merge all summaryfor step in range(100):    # train and net output    _, result = sess.run([train_op, merge_op], {tf_x: x, tf_y: y})    writer.add_summary(result, step)

Tensorflow-Tutorial/tutorial-contents/305_tensorboard.py

---

逐层可视化

---

卷积神经网络实战（可视化部分）——使用keras识别猫咪
Visualizing parts of Convolutional Neural Networks using Keras and Cats
All code is on Github: https://github.com/erikreppel/visualizing_cnns

这张图长400像素宽320像素，有三个通道（rgb）的颜色。
那么经过一层卷积运算之后会变成什么样子呢？

这是用一个3*3的卷积核和三个滤波器处理的效果（如果我们有超过3个的滤波器，那么我可以画出猫的2d图像。更高维的话就很难处理）
我们可以看到，图中的猫非常的模糊，因为我们使用了一个随机的初始值，而且我们还没有训练网络。他们都在彼此的顶端，即使每层都有细节，我们将无法看到它。但我们可以制作出与眼睛和背景相同颜色的猫的区域。如果我们将内核大小增加到10x10，会发生什么呢？

我们可以看到，由于内核太大，我们失去了一些细节。还要注意，从数学角度来看，卷积核越大，图像的形状会变得越小。
如果我们把它压扁一点，我们可以更好的看到色彩通道会发生什么？

这张看起来好多了！现在我们可以看到我们的过滤器看到的一些事情。看起来红色替换掉了黑色的鼻子和黑色眼睛，蓝色替换掉了猫边界的浅灰色。我们可以开始看到图层如何捕获照片中的一些更重要的细节。

如果我们增加内核大小，我们得到的细节就会越来越明显，当然图像也比其他两个都小。

增加一个激活层

我们通过添加一个relu，去掉了很多不是蓝色的部分。

增加一个池化层

我们添加一个池化层（摆脱激活层最大限度地让图片更加更容易显示）。

正如预期的那样，猫咪变成了斑驳的，而我们可以让它更加斑驳。

现在图片大约成了原来的三分之一。

激活和最大池化

LeNet

如果我们将猫咪的图片放到LeNet模型中做卷积和池化，那么效果会怎么样呢?

ConvNets功能强大，因为它们能够提取图像的核心特征，并使用这些特征来识别包含其中的特征的图像。即使我们的两层CNN，我们也可以开始看到网络正在对猫的晶须，鼻子和眼睛这样的地区给予很多的关注。这些是让CNN将猫与鸟区分开的特征的类型。
CNN是非常强大的，虽然这些可视化并不完美，但我希望他们能够帮助像我这样正在尝试更好地理解ConvNets的人。

---

可视化模型

---

keras.utils.vis_utils模块提供了画出Keras模型的函数（利用graphviz）

该函数将画出模型结构图，并保存成图片：

from keras.utils import plot_modelplot_model(model, to_file='model.png')

plot_model接收两个可选参数：

show_shapes：指定是否显示输出数据的形状，默认为Falseshow_layer_names:指定是否显示层名称,默认为True

我们也可以直接获取一个pydot.Graph对象，然后按照自己的需要配置它，例如，如果要在ipython中展示图片

from IPython.display import SVGfrom keras.utils.vis_utils import model_to_dotSVG(model_to_dot(model).create(prog='dot', format='svg'))

【Tips】依赖 pydot-ng 和 graphviz，若出现错误，用命令行输入pip install pydot-ng & brew install graphviz

---

keras回调函数中的Tensorboard

---

keras.callbacks.TensorBoard(log_dir='./Graph', histogram_freq=0,            write_graph=True, write_images=True)

tbCallBack = keras.callbacks.TensorBoard(log_dir='./Graph', histogram_freq=0, write_graph=True, write_images=True)...model.fit(...inputs and parameters..., callbacks=[tbCallBack])

tensorboard --logdir path_to_current_dir/Graph 

或者

from keras.callbacks import TensorBoardtensorboard = TensorBoard(log_dir='./logs', histogram_freq=0,                          write_graph=True, write_images=False)# define modelmodel.fit(X_train, Y_train,          batch_size=batch_size,          epochs=nb_epoch,          validation_data=(X_test, Y_test),          shuffle=True,          callbacks=[tensorboard])

https://stackoverflow.com/questions/42112260/how-do-i-use-the-tensorboard-callback-of-keras

---

参考文献

---

Display Deep Learning Model Training History in Keras

Edward－Tensorboard

详解TensorBoard如何调参
https://gist.github.com/dandelionmane/4f02ab8f1451e276fea1f165a20336f1#file-mnist-py

TensorFlow深度学习笔记 Tensorboard入门

Keras中文文档－模型可视化
极客学院－TensorBoard:可视化学习

Tensorboard 可视化之图层

Tensorboard 可视化之训练过程

TensorBoard可视化详解

TensorFlow学习笔记（七）：TensorBoard可视化助手

https://www.tensorflow.org/get_started/summaries_and_tensorboard
https://www.tensorflow.org/get_started/summaries_and_tensorboard
https://www.youtube.com/watch?v=eBbEDRsCmv4

How do I use the Tensorboard callback of Keras?
keras tensorboard的使用, 设置GPU使用的内存
tensorboard的一些问题
模型可视化
Keras中文文档

卷积神经网络实战（可视化部分）——使用keras识别猫咪
Visualizing parts of Convolutional Neural Networks using Keras and Cats
All code is on Github: https://github.com/erikreppel/visualizing_cnns