【tensorflow学习】卷积神经网络CNN（mnist数据集）

一、 导入数据

from tensorflow.examples.tutorials.mnist import input_datamnist = input_data.read_data_sets('MNIST_data/', one_hot=True)

每个example的类label以one-hot形式设定。

二、 运行TensorFlow的InteractiveSession

sess = tf.InteractiveSession()

Tensorflow依赖于一个高效的C++后端来进行计算。与后端的这个连接叫做session。一般而言，使用TensorFlow程序的流程是先创建一个图，然后在session中启动它。
InteractiveSession和普通的tf.Session()的区别在于，InteractiveSession能把自己作为默认的session，tf.Tensor.eval 和tf.Operation.run 就是用这个默认session运行ops的。
如果是普通的session，达到同样的效果则需要用‘with’。

二、 定义超参数

# hyperparamslearning_rate = 1e-4batch_size = 50step = 20000display_step = 100drop_out = 0.5

三、创建图（build graph）

3.1 声明图的输入（graph input）

# graph inputx = tf.placeholder(tf.float32, [None, 784]) #  none ：不确定个batchy = tf.placeholder(tf.float32, [None, 10])keep_prob = tf.placeholder(tf.float32)

3.2 创建cnn模型（feedforward /prediction）

首先为了简便起见定义一些函数：

# Create some wrappers for simplicitydef weight_var(shape):    return tf.Variable(tf.truncated_normal(shape, stddev=0.1))def bias_var(shape):    return tf.Variable(tf.random_normal(shape))def conv2d(x, W, b):    # conv with bias and 0-padding so images size stay the same    # strides:[batch, height, width, channels], batch和channels一定为1    x = tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')    return tf.nn.bias_add(x, b)def max_pool(x):    # [batch, height, width, channels]    return tf.nn.max_pool(x, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')

然后用以上的函数定义cnn模型结构(cov1+pool1+cov2+pool2+fc1+dropout+softmax)：

# create modeldef cnn(x, y, keep_prob):    # Reshape input picture    x = tf.reshape(x, [-1, 28, 28, 1])    # 5x5 conv, 1 input, 32 outputs    w_conv1 = weight_var([5, 5, 1, 32])    b_conv1 = bias_var([32])    # Convolution Layer    h_conv1 = tf.nn.relu(conv2d(x, w_conv1, b_conv1))    # Max Pooling (down-sampling)    h_pool1 = max_pool(h_conv1)     # 5x5 conv, 32 inputs, 64 outputs    w_conv2 = weight_var([5, 5, 32, 64])    b_conv2 = bias_var([64])     # Convolution Layer    h_conv2 = tf.nn.relu(conv2d(h_pool1, w_conv2, b_conv2))    # Max Pooling (down-sampling)    h_pool2 = max_pool(h_conv2)    # fully connected, 7*7*64 inputs, 1024 outputs    w_fc1 = weight_var([7 * 7 * 64, 1024])    b_fc1 = bias_var([1024])    # Fully connected layer    # Reshape conv2 output to fit fully connected layer input    h_pool2_flat = tf.reshape(h_pool2, [-1, 7 * 7 * 64])    h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, w_fc1) + b_fc1)    # Apply Dropout    h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)     # 1024 inputs, 10 outputs (class prediction)    w_fc2 = weight_var([1024, 10])    b_fc2 = bias_var([10])    # Output, class prediction    logit = tf.matmul(h_fc1_drop, w_fc2) + b_fc2    return logit# construct modelpred = cnn(x, y, keep_prob)

3.3 定义loss function和优化方法（Optimization）

loss function 用交叉熵，优化方法用adam 而不用gradient descent，因为在mnist数据集上adam效果更好。

# define loss and optimizercross_entropy = tf.reduce_mean(    tf.nn.softmax_cross_entropy_with_logits(labels=y, logits=pred))optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cross_entropy)

3.4 模型评估 (accuracy )

# evaluate modelcorrect_pred = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))

四、 用session.run 训练并测试模型

4.1 初始化所有变量

# init variablessess.run(tf.global_variables_initializer())

4.2 训练模型

for i in range(step):    batch = mnist.train.next_batch(batch_size)    if i % display_step == 0:        loss, acc = sess.run([cross_entropy, accuracy], feed_dict={                             x: batch[0], y: batch[1], keep_prob: 1.0})        print('step: %d' %              i, " minibatch loss: {0:.6f},training accuracy:{1:.5f} ".format(loss, acc))    optimizer.run(feed_dict={x: batch[0], y: batch[1], keep_prob: drop_out})#用默认session运行optimizer

4.3 测试模型

# test accuracyprint('Testing accuracy:', sess.run(accuracy, feed_dict={      x: mnist.test.images[:500], y: mnist.test.labels[:500], keep_prob: 1.}))

五、测试结果

......step: 17800  minibatch loss: 0.039849,training accracy:0.98000 step: 17900  minibatch loss: 0.051081,training accracy:0.96000 step: 18000  minibatch loss: 0.019745,training accracy:1.00000 step: 18100  minibatch loss: 0.002456,training accracy:1.00000 step: 18200  minibatch loss: 0.005077,training accracy:1.00000 step: 18300  minibatch loss: 0.000313,training accracy:1.00000 step: 18400  minibatch loss: 0.007955,training accracy:1.00000 step: 18500  minibatch loss: 0.007918,training accracy:1.00000 step: 18600  minibatch loss: 0.086002,training accracy:0.98000 step: 18700  minibatch loss: 0.002512,training accracy:1.00000 step: 18800  minibatch loss: 0.005236,training accracy:1.00000 step: 18900  minibatch loss: 0.002420,training accracy:1.00000 step: 19000  minibatch loss: 0.014115,training accracy:1.00000 step: 19100  minibatch loss: 0.010140,training accracy:1.00000 step: 19200  minibatch loss: 0.021536,training accracy:0.98000 step: 19300  minibatch loss: 0.001117,training accracy:1.00000 step: 19400  minibatch loss: 0.016043,training accracy:1.00000 step: 19500  minibatch loss: 0.002531,training accracy:1.00000 step: 19600  minibatch loss: 0.005710,training accracy:1.00000 step: 19700  minibatch loss: 0.002422,training accracy:1.00000 step: 19800  minibatch loss: 0.007958,training accracy:1.00000 step: 19900  minibatch loss: 0.067153,training accracy:0.98000 Testing accuracy: 0.994[Finished in 306.8s]

六、 完整代码

# -*- coding: utf-8 -*-# @Author: adrianna# @Date:   2017-05-19 20:35:55# @Last Modified by:   adrianna# @Last Modified time: 2017-06-02 11:54:56import tensorflow as tfimport osfrom tensorflow.examples.tutorials.mnist import input_dataos.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'sess = tf.InteractiveSession()# hyperparamslearning_rate = 1e-4batch_size = 50step = 20000display_step = 100drop_out = 0.5# load datamnist = input_data.read_data_sets('MNIST_data/', one_hot=True)# graph inputx = tf.placeholder(tf.float32, [None, 784])y = tf.placeholder(tf.float32, [None, 10])keep_prob = tf.placeholder(tf.float32)# Create some wrappers for simplicitydef weight_var(shape):    return tf.Variable(tf.truncated_normal(shape, stddev=0.1))def bias_var(shape):    return tf.Variable(tf.random_normal(shape))def conv2d(x, W, b):    # conv with bias    x = tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')    return tf.nn.bias_add(x, b)def max_pool(x):    return tf.nn.max_pool(x, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')# create modeldef cnn(x, y, keep_prob):    x = tf.reshape(x, [-1, 28, 28, 1])    w_conv1 = weight_var([5, 5, 1, 32])    b_conv1 = bias_var([32])    h_conv1 = tf.nn.relu(conv2d(x, w_conv1, b_conv1))    h_pool1 = max_pool(h_conv1)    w_conv2 = weight_var([5, 5, 32, 64])    b_conv2 = bias_var([64])    h_conv2 = tf.nn.relu(conv2d(h_pool1, w_conv2, b_conv2))    h_pool2 = max_pool(h_conv2)    w_fc1 = weight_var([7 * 7 * 64, 1024])    b_fc1 = bias_var([1024])    h_pool2_flat = tf.reshape(h_pool2, [-1, 7 * 7 * 64])    h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, w_fc1) + b_fc1)    h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)    w_fc2 = weight_var([1024, 10])    b_fc2 = bias_var([10])    logit = tf.matmul(h_fc1_drop, w_fc2) + b_fc2    return logit# construct modelpred = cnn(x, y, keep_prob)# define loss and optimizercross_entropy = tf.reduce_mean(    tf.nn.softmax_cross_entropy_with_logits(labels=y, logits=pred))optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cross_entropy)# evaluate modelcorrect_pred = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))# init variablessess.run(tf.global_variables_initializer())for i in range(step):    batch = mnist.train.next_batch(batch_size)    if i % display_step == 0:        # train_accuracy = accuracy.eval(feed_dict={        #     x: batch[0], y: batch[1], keep_prob: 1.0})        # print("step %d, training accuracy %g" % (i, train_accuracy))        loss, acc = sess.run([cross_entropy, accuracy], feed_dict={                             x: batch[0], y: batch[1], keep_prob: 1.0})        print('step: %d' %              i, " minibatch loss: {0:.6f},training accuracy:{1:.5f} ".format(loss, acc))    optimizer.run(feed_dict={x: batch[0], y: batch[1], keep_prob: drop_out})# test accuracyprint('Testing accuracy:', sess.run(accuracy, feed_dict={      x: mnist.test.images[:500], y: mnist.test.labels[:500], keep_prob: 1.}))

参考：

1. tensorflow 官网：https://www.tensorflow.org/get_started/mnist/pros
2. https://github.com/aymericdamien/TensorFlow-Examples