tensorflow入门(二)

TensorFlow实现卷积神经网络

学习Tensorflow实战个人笔记,共大家学习交流,欢迎拍砖
作者: 谢中朝

1. TensorFlow 实现简单的卷积神经网络

'''载入MNIST数据集,并创建默认的Interactive Session'''from tensorflow.examples.tutorials.mnist import input_dataimport tensorflow as tfmnist = input_data.read_data_sets("MNIST_data/", one_hot = True)sess = tf.InteractiveSession()

Extracting MNIST_data/train-images-idx3-ubyte.gzExtracting MNIST_data/train-labels-idx1-ubyte.gzExtracting MNIST_data/t10k-images-idx3-ubyte.gzExtracting MNIST_data/t10k-labels-idx1-ubyte.gz

'''创建卷及神经网络的权重和偏置(1)我们需要给权重制造随机噪声来打破完全对称,比如截断的正态分布噪声,标准差设为0.1(2)使用ReLU给偏置增加一些小的正值(0.1)用来避免死亡节点'''def weight_variable(shape):        initial = tf.truncated_normal(shape, stddev=0.1)        return tf.Variable(initial)def bias_variable(shape):        initial = tf.constant(0.1, shape = shape)        return tf.Variable(initial)

'''创建卷积层和池化层(1)参数x为输入,W为卷积参数,[5,5,1,32]:前面两个数字代表卷积尺寸,第三个数字代表有多少个channel,灰度单色为1,彩色RGB图片为3,最后一个数字代表卷积数量.(2)Strides代表模板卷积模板移动的步长,都是1代表会不遗漏地划过图片的每一个点.(3)Padding代表边界的处理方式,这里的SAME代表给边界加上Padding让卷积的输出和输入保持同样的(SAME)的尺寸.'''def conv2d(x, W):        return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')def max_pool_2x2(x):        return tf.nn.max_pool(x, ksize=[1,2,2,1], strides=[1,2,2,1], padding='SAME')

'''定义placeholder,x是特征,y_是真实的label(1)卷积神经网络会利用空间信息,因此需要将1D的输入向量转化为2D图片结构,1*784转化原始的28*28,只有一个颜色通道,最终尺寸[-1,28,28,1],前面的-1代表样本数量不固定,最后1代表颜色通道数量.'''x = tf.placeholder(tf.float32, [None, 784])y_ = tf.placeholder(tf.float32, [None, 10])x_image = tf.reshape(x, [-1,28,28,1])

'''定义第一个卷积层:(1)参数初始化,包括weights和bias,[5,5,1,32]代表卷积核尺寸为5*5,1个颜色通道,32个不同卷积核.(2)使用conv2d进行卷积,再加上偏置.(3)接着使用ReLU激活函数进行非线性处理.(4)最后用最大池化函数max_pool_2x2对卷积的结果进行池化.'''W_conv1 = weight_variable([5,5,1,32])b_conv1 = bias_variable([32])h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)h_pool1 = max_pool_2x2(h_conv1)

'''定义第二个卷积层:(1)参数初始化,包括weights和bias,[5,5,1,64]代表卷积核尺寸为5*5,1个颜色通道,64个不同卷积核.(2)使用conv2d进行卷积,再加上偏置.(3)接着使用ReLU激活函数进行非线性处理.(4)最后用最大池化函数max_pool_2x2对卷积的结果进行池化.''''''经历两次步长为2*2的最大池化,图片尺寸28*28变成7*7,由于第二层卷积核数量为64,输出尺寸即为7*7*64'''W_conv2 = weight_variable([5,5,32,64])b_conv2 = bias_variable([64])h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)h_pool2 = max_pool_2x2(h_conv2)print (h_pool2.shape)

(?, 7, 7, 64)

'''定义一个全连接层:(1)将第二层卷积的输出tensor进行变形,将其转化为1D向量,然后连接一个全连接层,隐含节点为1024(2)并使用ReLU激活函数'''W_fc1 = weight_variable([7 * 7 * 64, 1024])b_fc1 = bias_variable([1024])h_pool2_flat = tf.reshape(h_pool2, [-1, 7*7*64])print (h_pool2_flat.shape)h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1)print (h_fc1.shape)

(?, 3136)(?, 1024)

'''定义一个Dropout层:(1)为了减轻过拟合,使用Dropout,通过一个placeholder传入keep_prob比率进行控制.(2)在训练过程中,我们随机丢弃一部分节点的数据来减轻过拟合.(3)预测时则保留全部的数据来追求最好的预测性能.'''keep_prob = tf.placeholder(tf.float32)h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)print h_fc1_drop.shape

(?, 1024)

'''最后将Dropout层的输出连接一个Softmax层,得到最后的概率输出'''W_fc2 = weight_variable([1024, 10])b_fc2 = bias_variable([10])y_conv = tf.nn.softmax(tf.matmul(h_fc1_drop, W_fc2) + b_fc2)print y_conv.shape

(?, 10)

'''定义损失函数cross entropy, 优化器使用Adam, 并给予一个比较小的学习速率1e-4'''cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y_conv), reduction_indices=[1]))train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)

'''定义评定准确率的操作:'''correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y_, 1))accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

'''开始训练过程:(1)首先初始化所有参数(2)设置训练时的Dropout的keep_prob比率为0.5(3)然后使用大小为50的mini-batch,进行20000次的训练迭代,参与训练样本的数量总共为100万(4)其中每训练100次,对准确率进行一次评测(评测时keep_prob设为1),用于实时监控模型性能'''tf.global_variables_initializer().run()for i in range(20000):        batch = mnist.train.next_batch(50)#label:batch[1]代表50行*10列; input:batch[0]代表1024行*50列        if i%100 == 0:                train_accuracy = accuracy.eval(feed_dict={x:batch[0], y_:batch[1], keep_prob: 1.0})                print ("step %d     , training accuracy %g"%(i, train_accuracy))        train_step.run(feed_dict={x: batch[0], y_:batch[1], keep_prob: 0.5})

step 0     , training accuracy 0.06step 100     , training accuracy 0.8step 200     , training accuracy 0.86step 300     , training accuracy 0.92step 400     , training accuracy 0.94step 500     , training accuracy 0.9step 600     , training accuracy 0.98step 700     , training accuracy 0.92step 800     , training accuracy 0.96step 900     , training accuracy 0.98step 1000     , training accuracy 0.96step 1100     , training accuracy 0.94step 1200     , training accuracy 0.98step 1300     , training accuracy 1step 1400     , training accuracy 0.96step 1500     , training accuracy 0.98step 1600     , training accuracy 0.98step 1700     , training accuracy 1step 1800     , training accuracy 0.94step 1900     , training accuracy 0.98step 2000     , training accuracy 1step 2100     , training accuracy 1step 2200     , training accuracy 1step 2300     , training accuracy 0.98step 2400     , training accuracy 0.96step 2500     , training accuracy 1step 2600     , training accuracy 0.94step 2700     , training accuracy 0.98step 2800     , training accuracy 1step 2900     , training accuracy 0.94step 3000     , training accuracy 0.96step 3100     , training accuracy 0.96step 3200     , training accuracy 1step 3300     , training accuracy 0.98step 3400     , training accuracy 1step 3500     , training accuracy 0.96step 3600     , training accuracy 1step 3700     , training accuracy 1step 3800     , training accuracy 1step 3900     , training accuracy 0.98step 4000     , training accuracy 1step 4100     , training accuracy 1step 4200     , training accuracy 0.96step 4300     , training accuracy 0.98step 4400     , training accuracy 0.98step 4500     , training accuracy 0.96step 4600     , training accuracy 1step 4700     , training accuracy 1step 4800     , training accuracy 1step 4900     , training accuracy 1step 5000     , training accuracy 0.98step 5100     , training accuracy 1step 5200     , training accuracy 1step 5300     , training accuracy 1step 5400     , training accuracy 1step 5500     , training accuracy 0.98step 5600     , training accuracy 1step 5700     , training accuracy 1step 5800     , training accuracy 0.98step 5900     , training accuracy 1step 6000     , training accuracy 1step 6100     , training accuracy 1step 6200     , training accuracy 1step 6300     , training accuracy 1step 6400     , training accuracy 1step 6500     , training accuracy 0.98step 6600     , training accuracy 1step 6700     , training accuracy 0.98step 6800     , training accuracy 1step 6900     , training accuracy 1step 7000     , training accuracy 1step 7100     , training accuracy 0.98step 7200     , training accuracy 0.98step 7300     , training accuracy 1step 7400     , training accuracy 1step 7500     , training accuracy 1step 7600     , training accuracy 1step 7700     , training accuracy 1step 7800     , training accuracy 0.98step 7900     , training accuracy 0.98step 8000     , training accuracy 1step 8100     , training accuracy 0.96step 8200     , training accuracy 1step 8300     , training accuracy 0.98step 8400     , training accuracy 1step 8500     , training accuracy 1step 8600     , training accuracy 1step 8700     , training accuracy 1step 8800     , training accuracy 1step 8900     , training accuracy 1step 9000     , training accuracy 1step 9100     , training accuracy 1step 9200     , training accuracy 0.98step 9300     , training accuracy 0.98step 9400     , training accuracy 1step 9500     , training accuracy 0.98step 9600     , training accuracy 0.98step 9700     , training accuracy 0.98step 9800     , training accuracy 1step 9900     , training accuracy 1step 10000     , training accuracy 1step 10100     , training accuracy 1step 10200     , training accuracy 1step 10300     , training accuracy 0.98step 10400     , training accuracy 1step 10500     , training accuracy 1step 10600     , training accuracy 1step 10700     , training accuracy 1step 10800     , training accuracy 1step 10900     , training accuracy 0.98step 11000     , training accuracy 1step 11100     , training accuracy 1step 11200     , training accuracy 1step 11300     , training accuracy 1step 11400     , training accuracy 1step 11500     , training accuracy 1step 11600     , training accuracy 1step 11700     , training accuracy 1step 11800     , training accuracy 1step 11900     , training accuracy 1step 12000     , training accuracy 1step 12100     , training accuracy 1step 12200     , training accuracy 1step 12300     , training accuracy 1step 12400     , training accuracy 1step 12500     , training accuracy 1step 12600     , training accuracy 1step 12700     , training accuracy 1step 12800     , training accuracy 1step 12900     , training accuracy 1step 13000     , training accuracy 1step 13100     , training accuracy 1step 13200     , training accuracy 1step 13300     , training accuracy 1step 13400     , training accuracy 1step 13500     , training accuracy 1step 13600     , training accuracy 1step 13700     , training accuracy 1step 13800     , training accuracy 1step 13900     , training accuracy 0.96step 14000     , training accuracy 1step 14100     , training accuracy 1step 14200     , training accuracy 1step 14300     , training accuracy 1step 14400     , training accuracy 1step 14500     , training accuracy 1step 14600     , training accuracy 1step 14700     , training accuracy 1step 14800     , training accuracy 1step 14900     , training accuracy 1step 15000     , training accuracy 1step 15100     , training accuracy 1step 15200     , training accuracy 1step 15300     , training accuracy 1step 15400     , training accuracy 1step 15500     , training accuracy 1step 15600     , training accuracy 0.98step 15700     , training accuracy 1step 15800     , training accuracy 1step 15900     , training accuracy 1step 16000     , training accuracy 1step 16100     , training accuracy 1step 16200     , training accuracy 1step 16300     , training accuracy 1step 16400     , training accuracy 0.98step 16500     , training accuracy 1step 16600     , training accuracy 1step 16700     , training accuracy 1step 16800     , training accuracy 1step 16900     , training accuracy 1step 17000     , training accuracy 1step 17100     , training accuracy 1step 17200     , training accuracy 1step 17300     , training accuracy 1step 17400     , training accuracy 1step 17500     , training accuracy 1step 17600     , training accuracy 1step 17700     , training accuracy 1step 17800     , training accuracy 1step 17900     , training accuracy 1step 18000     , training accuracy 1step 18100     , training accuracy 1step 18200     , training accuracy 1step 18300     , training accuracy 1step 18400     , training accuracy 1step 18500     , training accuracy 1step 18600     , training accuracy 1step 18700     , training accuracy 1step 18800     , training accuracy 1step 18900     , training accuracy 1step 19000     , training accuracy 1step 19100     , training accuracy 1step 19200     , training accuracy 1step 19300     , training accuracy 1step 19400     , training accuracy 1step 19500     , training accuracy 1step 19600     , training accuracy 1step 19700     , training accuracy 1step 19800     , training accuracy 1step 19900     , training accuracy 1

'''在最终的数据集上进行全面测试,得到整体的分类的准确率'''print("test_accuracy %g"%accuracy.eval(feed_dict={x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0}))

test_accuracy 0.9923

2. TensorFlow 实现进阶的卷积神经网络

cd /home/xzz/code/python_tensorflow/models-master/tutorials/image/cifar10/

/home/xzz/code/python_tensorflow/models-master/tutorials/image/cifar10

#载入常用的库import cifar10import cifar10_inputimport tensorflow as tfimport numpy as npimport time

'''定义batch_size, 训练轮数max_steps,以及下载CIFAR-10数据的默认路径'''max_step = 100000batch_size = 128data_dir = '/home/xzz/download/cifar-10-batches-bin/'

'''(1)定义初始化weight的函数,使用tf.truncated_normal截断的正态分布来初始化权重,给weight加一个L2的loss,进行L2正则化处理.(2)在机器学习中,不管是分类任务还是回归任务,都可能因特征过多而导致过拟合,一般可以通过减少特征或者惩罚不重要特征的权重来缓解这一问题.(3)正则化就是帮组惩罚特征权重的,特征权重也会成为模型的损失函数的一部分.一般来说,L1正则会制造稀疏的特征,大部分无用的特征权重会被置为0,而L2正则会让特征的权重不过大,使得特征的权重比较平均.(4)我们使用w1控制L2 loss的大小, 使用tf.nn.l2_loss函数计算weight的L2 loss,再使用tf.multipy让L2 loss乘以w1得到weight loss.接着使用tf.add_to_collection把weight loss统一存在一个collection,这个colloction称为'losses',在计算神经网络的总体loss时被用上.'''def variable_with_weight_loss(shape, stddev, wl):    var = tf.Variable(tf.truncated_normal(shape, stddev = stddev))    if wl is not None:        weight_loss = tf.multiply(tf.nn.l2_loss(var), wl, name = 'weight_loss')        tf.add_to_collection('losses', weight_loss)    return var 

#使用cifar10类下载数据集,并解压,展开到默认位置cifar10.maybe_download_and_extract()

'''(1)使用cifar10_input类中的distorted_inputs函数产生训练需要使用的数据,包括特征及其对应的label.(2)对图像的进行数据增强的操作耗费大量的CPU时间,因此disorted_inputs使用了16个独立的线程来加速任务,函数内部会产生线程池,在需要使用的时候会通过TensorFlow queue进行调度'''images_train, labels_train = cifar10_input.distorted_inputs(data_dir = data_dir, batch_size = batch_size)

Filling queue with 20000 CIFAR images before starting to train. This will take a few minutes.

'''cifar10_input.inputs函数生成测试数据,不需要太多的处理,将图片裁剪为24*24大小的区块,进行数据的标准化处理'''images_test, labels_test = cifar10_input.inputs(eval_data=True, data_dir=data_dir, batch_size=batch_size)

'''(1)创建输入数据placeholder,包括特征和label(2)数据尺寸中的图片尺寸为24*24,即是裁剪后的大小,颜色通道数则设为3,代表图片是彩色有RGB三条通道'''image_holder = tf.placeholder(tf.float32, [batch_size, 24, 24, 3])label_holder =  tf.placeholder(tf.int32, [batch_size])

'''(1)开始创建第一个卷积层(2)先对variable_with_weight_loss函数创建卷积核参数初始化,第一个卷积层使用5*5卷积核大小,3个颜色通道,64个卷积核(3)不对第一个卷积层的weight进行L2正则,因此w1(weight loss)这项设为0(4)使用tf.nn.conv2d函数对输入数据image_holder进行卷积操作,步长为1,padding模式为SAME,将这层bias全部初始化为0,再将卷积结果加上bias,最后使用ReLU激活函数进行非线性化(5)在激活函数后,使用3*3且步长为2*2最大池化层处理数据,这里最大池化尺寸和步长不一致可以增加数据的丰富性(6)最后进行LRN,可以使其中响应比较大的值变得更大,并抑制其他反馈较小的神经元,增强模型的泛化能力,适合没有上界的激活函数'''weight1 = variable_with_weight_loss(shape=[5,5,3,64], stddev=5e-2, wl=0.0)kernel1 = tf.nn.conv2d(image_holder, weight1, [1,1,1,1], padding='SAME')print 'kernel', kernel1bias1 = tf.Variable(tf.constant(0.0, shape=[64]))print 'bias1', bias1conv1 = tf.nn.relu(tf.nn.bias_add(kernel1, bias1))print 'conv', conv1pool1 = tf.nn.max_pool(conv1, ksize=[1,3,3,1], strides=[1,2,2,1],padding='SAME')print 'pool', pool1norm1 = tf.nn.lrn(pool1, 4, bias=1.0, alpha=0.001/9.0, beta=0.75)print 'norm', norm1

kernel Tensor("Conv2D:0", shape=(128, 24, 24, 64), dtype=float32)bias1 Tensor("Variable_1/read:0", shape=(64,), dtype=float32)conv Tensor("Relu_2:0", shape=(128, 24, 24, 64), dtype=float32)pool Tensor("MaxPool:0", shape=(128, 12, 12, 64), dtype=float32)norm Tensor("LRN:0", shape=(128, 12, 12, 64), dtype=float32)

'''(1)开始创建第二个卷积层(2)64个通道,64个卷积核,注意bias全部初始化为0.1(3)注意该层先进行LRN层处理,再使用最大池化层'''weight2 = variable_with_weight_loss(shape=[5,5,64,64], stddev=5e-2, wl=0.0)kernel2 = tf.nn.conv2d(norm1, weight2, [1,1,1,1], padding='SAME')bias2 = tf.Variable(tf.constant(0.0, shape=[64]))conv2 = tf.nn.relu(tf.nn.bias_add(kernel2, bias2))norm2 = tf.nn.lrn(conv2, 4, bias=1.0, alpha=0.001/9.0, beta=0.75)pool2 = tf.nn.max_pool(norm2, ksize=[1,3,3,1], strides=[1,2,2,1],padding='SAME')print 'pool', pool2print batch_size

pool Tensor("MaxPool_1:0", shape=(128, 6, 6, 64), dtype=float32)128

'''(1)使用全连接层,将两个卷积层的输入结果全部flatten,使用tf.shape函数把每一个样本都变成一维向量,get_shape可以获得长度(2)variable_with_weight_loss函数对全连接层的weight进行初始化,隐含节点数384,正态分布标准差设为0.04,bias设置为0.1(3)为了这个全连接层不要过拟合,设置了非零weight loss即w1 = 0.004,让这一层所有参数都被L2正则所约束,最后ReLU激活函数进行非线性化'''reshape = tf.reshape(pool2, [batch_size, -1])print 'reshape', reshape.shapedim = reshape.get_shape()[1].valueprint reshape.get_shape()[0]print reshape.get_shape()[1]print 'dim', dimweight3 = variable_with_weight_loss(shape=[dim, 384], stddev = 0.04, wl = 0.004)print 'weight3', weight3bias3 = tf.Variable(tf.constant(0.1, shape=[384]))print 'bias3', bias3local3 = tf.nn.relu(tf.matmul(reshape, weight3) + bias3)print 'local3', local3

reshape (128, 2304)1282304dim 2304weight3 Tensor("Variable_4/read:0", shape=(2304, 384), dtype=float32)bias3 Tensor("Variable_5/read:0", shape=(384,), dtype=float32)local3 Tensor("Relu_4:0", shape=(128, 384), dtype=float32)

'''与前一个全连接层一样,就是隐藏节点下降一半,只有192个,其他的超参数保持不变'''weight4 = variable_with_weight_loss(shape=[384, 192], stddev = 0.04, wl = 0.004)bias4 = tf.Variable(tf.constant(0.1, shape=[192]))local4 = tf.nn.relu(tf.matmul(local3, weight4) + bias4)print 'local4', local4

local4 Tensor("Relu_5:0", shape=(128, 192), dtype=float32)

'''最后一层,先创建这一层的weight,其正态分布的标准差设为上一个隐藏层的节点数的倒数,并且不计入L2的正则'''weight5 = variable_with_weight_loss(shape=[192, 10], stddev = 1/192.0, wl = 0.0)print 'weight5',weight5bias5 = tf.Variable(tf.constant(0.0, shape=[10]))logits = tf.add(tf.matmul(local4, weight5), bias5)print 'logits', logits.shape

weight5 Tensor("Variable_8/read:0", shape=(192, 10), dtype=float32)logits (128, 10)

'''为了完成模型inference部分构建,接下来计算CNN的loss,将全部的loss整合到一起,其中包括cross entropy loss,还有后两个全连接层中的weight的L2 loss'''def loss(logits, labels):    labels = tf.cast(labels, tf.int64)    cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(        logits = logits, labels = labels, name = 'cross_entropy_per_example')    cross_entropy_mean = tf.reduce_mean(cross_entropy, name='cross_entropy')    tf.add_to_collection('losses', cross_entropy_mean)    return tf.add_n(tf.get_collection('losses'), name = 'total_loss' )

#将logits节点和label_placeholder传入loss函数获得最终的lossloss = loss(logits, label_holder)#优化器Adam Optimizer, 学习速率设置为1e-3train_op = tf.train.AdamOptimizer(1e-3).minimize(loss)#使用tf.nn.in_top_k函数求出输出结果中top k的准确率,默认使用top 1,输出得分最高的那一类准确率top_k_op = tf.nn.in_top_k(logits, label_holder, 1)#tf.InteractiveSession创建默认的session,接着初始化全部模型参数sess = tf.InteractiveSession()tf.global_variables_initializer().run()

'''启动图片数据增强线程队列,一共有16个线程来进行加速'''tf.train.start_queue_runners()

[<Thread(Thread-4, started daemon 139821564057344)>, <Thread(Thread-5, started daemon 139821555664640)>, <Thread(Thread-6, started daemon 139821547271936)>, <Thread(Thread-7, started daemon 139821538879232)>, <Thread(Thread-8, started daemon 139821530486528)>, <Thread(Thread-9, started daemon 139820781270784)>, <Thread(Thread-10, started daemon 139820772878080)>, <Thread(Thread-11, started daemon 139820764485376)>, <Thread(Thread-12, started daemon 139820756092672)>, <Thread(Thread-13, started daemon 139820747699968)>, <Thread(Thread-14, started daemon 139820739307264)>, <Thread(Thread-15, started daemon 139820730914560)>, <Thread(Thread-16, started daemon 139820244399872)>, <Thread(Thread-17, started daemon 139820236007168)>, <Thread(Thread-18, started daemon 139820227614464)>, <Thread(Thread-19, started daemon 139820219221760)>, <Thread(Thread-20, started daemon 139820210829056)>, <Thread(Thread-21, started daemon 139820202436352)>, <Thread(Thread-22, started daemon 139820194043648)>, <Thread(Thread-23, started daemon 139820185650944)>, <Thread(Thread-24, started daemon 139820177258240)>, <Thread(Thread-25, started daemon 139820168865536)>, <Thread(Thread-26, started daemon 139820160472832)>, <Thread(Thread-27, started daemon 139820152080128)>, <Thread(Thread-28, started daemon 139820143687424)>, <Thread(Thread-29, started daemon 139820135294720)>, <Thread(Thread-30, started daemon 139820126902016)>, <Thread(Thread-31, started daemon 139820118509312)>, <Thread(Thread-32, started daemon 139820110116608)>, <Thread(Thread-33, started daemon 139820101723904)>, <Thread(Thread-34, started daemon 139820093331200)>, <Thread(Thread-35, started daemon 139820084938496)>, <Thread(Thread-36, started daemon 139820076545792)>, <Thread(Thread-37, started daemon 139820068153088)>]

'''现在正式开始训练.(1)在每一个step训练过程中,我们需要使用session的run方法执行images_train, labels_train(2)获得一个batch的训练数据,在将这个batch的数据传入train_op和loss的计算(3)我们记录,每一个step花费的时间,每隔10个step会计算和展示当前的loss,每秒训练的样本数量,以及训练一个batch数据所花费的时间'''for step in range(max_step):    start_time = time.time()    image_batch, label_batch = sess.run([images_train, labels_train])    _, loss_value = sess.run([train_op, loss],                            feed_dict={image_holder: image_batch, label_holder: label_batch})    duration = time.time() - start_time    if step%10 == 0 :        examples_per_sec = batch_size / duration        sec_per_batch = float(duration)        format_str=('step %d, loss=%.2f  (%.1f  examples/sec;  %.3f  sec/batch)' )        print(format_str % (step, loss_value, examples_per_sec, sec_per_batch))

step 0, loss=4.68 (170.5 examples/sec; 0.751 sec/batch)
step 10, loss=3.66 (1896.5 examples/sec; 0.067 sec/batch)
step 20, loss=3.11 (1757.2 examples/sec; 0.073 sec/batch)
step 30, loss=2.75 (1859.3 examples/sec; 0.069 sec/batch)
step 40, loss=2.42 (1860.1 examples/sec; 0.069 sec/batch)
step 50, loss=2.25 (1932.3 examples/sec; 0.066 sec/batch)
step 60, loss=2.21 (1881.6 examples/sec; 0.068 sec/batch)
step 70, loss=2.02 (1929.8 examples/sec; 0.066 sec/batch)
step 80, loss=1.96 (1898.5 examples/sec; 0.067 sec/batch)
step 90, loss=2.01 (1902.7 examples/sec; 0.067 sec/batch)
step 100, loss=1.95 (1187.7 examples/sec; 0.108 sec/batch)
step 110, loss=2.03 (1928.6 examples/sec; 0.066 sec/batch)
step 120, loss=1.87 (1935.3 examples/sec; 0.066 sec/batch)
step 130, loss=1.88 (1802.0 examples/sec; 0.071 sec/batch)
step 140, loss=1.81 (1964.1 examples/sec; 0.065 sec/batch)
step 150, loss=1.86 (1899.4 examples/sec; 0.067 sec/batch)
step 160, loss=1.83 (1890.8 examples/sec; 0.068 sec/batch)
step 170, loss=1.99 (1129.3 examples/sec; 0.113 sec/batch)
step 180, loss=1.82 (1953.1 examples/sec; 0.066 sec/batch)
step 190, loss=1.75 (1931.8 examples/sec; 0.066 sec/batch)
step 200, loss=1.79 (1813.3 examples/sec; 0.071 sec/batch)
step 210, loss=1.55 (1550.7 examples/sec; 0.083 sec/batch)
step 220, loss=1.64 (1886.9 examples/sec; 0.068 sec/batch)
step 230, loss=1.71 (1889.5 examples/sec; 0.068 sec/batch)
step 240, loss=1.57 (1797.4 examples/sec; 0.071 sec/batch)
step 250, loss=1.78 (1909.6 examples/sec; 0.067 sec/batch)
step 260, loss=1.76 (1901.4 examples/sec; 0.067 sec/batch)
step 270, loss=1.73 (1283.8 examples/sec; 0.100 sec/batch)
step 280, loss=1.57 (1898.1 examples/sec; 0.067 sec/batch)
step 290, loss=1.53 (1974.6 examples/sec; 0.065 sec/batch)
step 300, loss=1.67 (2031.1 examples/sec; 0.063 sec/batch)
step 310, loss=1.69 (1917.7 examples/sec; 0.067 sec/batch)
step 320, loss=1.81 (1857.9 examples/sec; 0.069 sec/batch)
step 330, loss=1.52 (1961.6 examples/sec; 0.065 sec/batch)
step 340, loss=1.55 (1857.5 examples/sec; 0.069 sec/batch)
step 350, loss=1.63 (1730.8 examples/sec; 0.074 sec/batch)
step 360, loss=1.75 (1848.8 examples/sec; 0.069 sec/batch)
step 370, loss=1.59 (1582.6 examples/sec; 0.081 sec/batch)
step 380, loss=1.67 (1652.3 examples/sec; 0.077 sec/batch)
step 390, loss=1.64 (926.2 examples/sec; 0.138 sec/batch)
step 400, loss=1.62 (1666.3 examples/sec; 0.077 sec/batch)
step 410, loss=1.34 (1907.2 examples/sec; 0.067 sec/batch)
step 420, loss=1.40 (1929.3 examples/sec; 0.066 sec/batch)
step 430, loss=1.38 (1904.4 examples/sec; 0.067 sec/batch)
step 440, loss=1.57 (1899.8 examples/sec; 0.067 sec/batch)
step 450, loss=1.61 (1939.0 examples/sec; 0.066 sec/batch)
step 460, loss=1.42 (1908.8 examples/sec; 0.067 sec/batch)
step 470, loss=1.48 (1887.1 examples/sec; 0.068 sec/batch)
step 480, loss=1.44 (1865.2 examples/sec; 0.069 sec/batch)
step 490, loss=1.47 (1965.5 examples/sec; 0.065 sec/batch)
step 500, loss=1.62 (1778.1 examples/sec; 0.072 sec/batch)
step 510, loss=1.49 (1951.4 examples/sec; 0.066 sec/batch)
step 520, loss=1.41 (1848.1 examples/sec; 0.069 sec/batch)
step 530, loss=1.29 (1861.3 examples/sec; 0.069 sec/batch)
step 540, loss=1.49 (1864.5 examples/sec; 0.069 sec/batch)
step 550, loss=1.50 (1925.5 examples/sec; 0.066 sec/batch)
step 560, loss=1.59 (1965.4 examples/sec; 0.065 sec/batch)
step 570, loss=1.53 (1990.2 examples/sec; 0.064 sec/batch)
step 580, loss=1.39 (1908.6 examples/sec; 0.067 sec/batch)
step 590, loss=1.56 (1961.8 examples/sec; 0.065 sec/batch)
step 600, loss=1.53 (1902.4 examples/sec; 0.067 sec/batch)
step 610, loss=1.36 (1986.2 examples/sec; 0.064 sec/batch)
step 620, loss=1.41 (1975.8 examples/sec; 0.065 sec/batch)
step 630, loss=1.37 (1359.0 examples/sec; 0.094 sec/batch)
step 640, loss=1.42 (2006.3 examples/sec; 0.064 sec/batch)
step 650, loss=1.32 (1926.8 examples/sec; 0.066 sec/batch)
step 660, loss=1.38 (1856.8 examples/sec; 0.069 sec/batch)
step 670, loss=1.31 (1615.1 examples/sec; 0.079 sec/batch)
step 680, loss=1.42 (1897.6 examples/sec; 0.067 sec/batch)
step 690, loss=1.31 (1951.4 examples/sec; 0.066 sec/batch)
step 700, loss=1.45 (1058.2 examples/sec; 0.121 sec/batch)
step 710, loss=1.33 (1863.9 examples/sec; 0.069 sec/batch)
step 720, loss=1.41 (1924.8 examples/sec; 0.067 sec/batch)
step 730, loss=1.39 (1917.7 examples/sec; 0.067 sec/batch)
step 740, loss=1.60 (1858.7 examples/sec; 0.069 sec/batch)
step 750, loss=1.45 (1933.1 examples/sec; 0.066 sec/batch)
step 760, loss=1.30 (1843.2 examples/sec; 0.069 sec/batch)
step 770, loss=1.29 (1922.2 examples/sec; 0.067 sec/batch)
step 780, loss=1.39 (1924.9 examples/sec; 0.066 sec/batch)
step 790, loss=1.45 (1907.3 examples/sec; 0.067 sec/batch)
step 800, loss=1.30 (1913.0 examples/sec; 0.067 sec/batch)
step 810, loss=1.36 (1934.2 examples/sec; 0.066 sec/batch)
step 820, loss=1.34 (1886.5 examples/sec; 0.068 sec/batch)
step 830, loss=1.37 (1841.6 examples/sec; 0.070 sec/batch)
step 840, loss=1.35 (1912.6 examples/sec; 0.067 sec/batch)
step 850, loss=1.24 (1909.8 examples/sec; 0.067 sec/batch)
step 860, loss=1.41 (1917.0 examples/sec; 0.067 sec/batch)
step 870, loss=1.26 (1821.0 examples/sec; 0.070 sec/batch)
step 880, loss=1.29 (1823.2 examples/sec; 0.070 sec/batch)
step 890, loss=1.31 (1913.2 examples/sec; 0.067 sec/batch)
step 900, loss=1.44 (2066.2 examples/sec; 0.062 sec/batch)
step 910, loss=1.47 (1844.2 examples/sec; 0.069 sec/batch)
step 920, loss=1.38 (1914.7 examples/sec; 0.067 sec/batch)
step 930, loss=1.18 (1884.0 examples/sec; 0.068 sec/batch)
step 940, loss=1.33 (1877.6 examples/sec; 0.068 sec/batch)
step 950, loss=1.19 (1826.2 examples/sec; 0.070 sec/batch)
step 960, loss=1.38 (1940.5 examples/sec; 0.066 sec/batch)
step 970, loss=1.37 (1947.3 examples/sec; 0.066 sec/batch)
step 980, loss=1.26 (1921.8 examples/sec; 0.067 sec/batch)
step 990, loss=1.29 (1844.1 examples/sec; 0.069 sec/batch)
step 1000, loss=1.19 (2098.3 examples/sec; 0.061 sec/batch)
step 1010, loss=1.19 (1868.5 examples/sec; 0.069 sec/batch)
step 1020, loss=1.29 (1836.6 examples/sec; 0.070 sec/batch)
step 1030, loss=1.37 (1896.0 examples/sec; 0.068 sec/batch)
step 1040, loss=1.23 (1870.3 examples/sec; 0.068 sec/batch)
step 1050, loss=1.27 (1877.9 examples/sec; 0.068 sec/batch)
step 1060, loss=1.39 (1963.9 examples/sec; 0.065 sec/batch)
step 1070, loss=1.28 (1979.4 examples/sec; 0.065 sec/batch)
step 1080, loss=1.21 (1917.5 examples/sec; 0.067 sec/batch)
step 1090, loss=1.40 (1896.0 examples/sec; 0.068 sec/batch)
step 1100, loss=1.15 (1893.4 examples/sec; 0.068 sec/batch)
step 1110, loss=1.11 (1841.6 examples/sec; 0.070 sec/batch)
step 1120, loss=1.23 (1855.3 examples/sec; 0.069 sec/batch)
step 1130, loss=1.31 (1797.0 examples/sec; 0.071 sec/batch)
step 1140, loss=1.28 (1819.6 examples/sec; 0.070 sec/batch)
step 1150, loss=1.26 (1958.6 examples/sec; 0.065 sec/batch)
step 1160, loss=1.23 (1896.6 examples/sec; 0.067 sec/batch)
step 1170, loss=1.34 (1933.1 examples/sec; 0.066 sec/batch)
step 1180, loss=1.24 (1869.0 examples/sec; 0.068 sec/batch)
step 1190, loss=1.19 (1889.1 examples/sec; 0.068 sec/batch)
step 1200, loss=1.37 (1826.7 examples/sec; 0.070 sec/batch)
step 1210, loss=1.40 (1950.8 examples/sec; 0.066 sec/batch)
step 1220, loss=1.26 (1833.5 examples/sec; 0.070 sec/batch)
step 1230, loss=1.23 (1888.9 examples/sec; 0.068 sec/batch)
step 1240, loss=1.17 (1877.0 examples/sec; 0.068 sec/batch)
step 1250, loss=1.41 (1960.0 examples/sec; 0.065 sec/batch)
step 1260, loss=1.29 (1854.7 examples/sec; 0.069 sec/batch)
step 1270, loss=1.13 (2023.2 examples/sec; 0.063 sec/batch)
step 1280, loss=1.34 (1929.8 examples/sec; 0.066 sec/batch)
step 1290, loss=1.20 (1873.5 examples/sec; 0.068 sec/batch)
step 1300, loss=1.24 (1951.3 examples/sec; 0.066 sec/batch)
step 1310, loss=1.15 (1840.3 examples/sec; 0.070 sec/batch)
step 1320, loss=1.06 (1911.2 examples/sec; 0.067 sec/batch)
step 1330, loss=1.26 (1949.0 examples/sec; 0.066 sec/batch)
step 1340, loss=1.24 (1896.5 examples/sec; 0.067 sec/batch)
step 1350, loss=1.17 (1912.7 examples/sec; 0.067 sec/batch)
step 1360, loss=1.08 (1925.7 examples/sec; 0.066 sec/batch)
step 1370, loss=1.49 (1997.7 examples/sec; 0.064 sec/batch)
step 1380, loss=1.32 (1969.7 examples/sec; 0.065 sec/batch)
step 1390, loss=1.29 (1819.8 examples/sec; 0.070 sec/batch)
step 1400, loss=1.34 (2006.5 examples/sec; 0.064 sec/batch)
step 1410, loss=1.12 (1844.8 examples/sec; 0.069 sec/batch)
step 1420, loss=1.20 (1859.7 examples/sec; 0.069 sec/batch)
step 1430, loss=1.07 (1901.2 examples/sec; 0.067 sec/batch)
step 1440, loss=1.13 (1832.0 examples/sec; 0.070 sec/batch)
step 1450, loss=1.05 (1869.1 examples/sec; 0.068 sec/batch)
step 1460, loss=1.25 (1917.7 examples/sec; 0.067 sec/batch)
step 1470, loss=1.27 (1792.5 examples/sec; 0.071 sec/batch)
step 1480, loss=1.18 (1874.7 examples/sec; 0.068 sec/batch)
step 1490, loss=1.20 (1927.0 examples/sec; 0.066 sec/batch)
step 1500, loss=1.09 (1921.9 examples/sec; 0.067 sec/batch)
step 1510, loss=1.20 (1896.2 examples/sec; 0.068 sec/batch)
step 1520, loss=1.30 (1800.1 examples/sec; 0.071 sec/batch)
step 1530, loss=1.21 (1829.0 examples/sec; 0.070 sec/batch)
step 1540, loss=1.35 (1821.9 examples/sec; 0.070 sec/batch)
step 1550, loss=1.20 (1929.8 examples/sec; 0.066 sec/batch)
step 1560, loss=1.18 (1940.8 examples/sec; 0.066 sec/batch)
step 1570, loss=1.19 (1820.8 examples/sec; 0.070 sec/batch)
step 1580, loss=1.01 (1954.3 examples/sec; 0.065 sec/batch)
step 1590, loss=1.09 (1890.5 examples/sec; 0.068 sec/batch)
step 1600, loss=1.14 (1988.3 examples/sec; 0.064 sec/batch)
step 1610, loss=1.00 (1906.1 examples/sec; 0.067 sec/batch)
step 1620, loss=1.07 (1938.0 examples/sec; 0.066 sec/batch)
step 1630, loss=1.17 (1926.3 examples/sec; 0.066 sec/batch)
step 1640, loss=1.16 (1881.1 examples/sec; 0.068 sec/batch)
step 1650, loss=1.28 (1899.0 examples/sec; 0.067 sec/batch)
step 1660, loss=1.11 (1944.9 examples/sec; 0.066 sec/batch)
step 1670, loss=1.15 (1948.2 examples/sec; 0.066 sec/batch)
step 1680, loss=1.27 (1842.7 examples/sec; 0.069 sec/batch)
step 1690, loss=1.22 (1273.3 examples/sec; 0.101 sec/batch)
step 1700, loss=1.29 (1370.6 examples/sec; 0.093 sec/batch)
…………
…………
…………
…………
step 99850, loss=0.60 (1834.3 examples/sec; 0.070 sec/batch)
step 99860, loss=0.58 (1834.3 examples/sec; 0.070 sec/batch)
step 99870, loss=0.50 (2053.8 examples/sec; 0.062 sec/batch)
step 99880, loss=0.73 (1902.0 examples/sec; 0.067 sec/batch)
step 99890, loss=0.65 (1929.2 examples/sec; 0.066 sec/batch)
step 99900, loss=0.70 (1886.3 examples/sec; 0.068 sec/batch)
step 99910, loss=0.55 (1914.5 examples/sec; 0.067 sec/batch)
step 99920, loss=0.77 (1941.3 examples/sec; 0.066 sec/batch)
step 99930, loss=0.54 (1929.5 examples/sec; 0.066 sec/batch)
step 99940, loss=0.61 (1895.4 examples/sec; 0.068 sec/batch)
step 99950, loss=0.50 (1949.0 examples/sec; 0.066 sec/batch)
step 99960, loss=0.54 (1972.2 examples/sec; 0.065 sec/batch)
step 99970, loss=0.60 (1885.1 examples/sec; 0.068 sec/batch)
step 99980, loss=0.58 (1921.5 examples/sec; 0.067 sec/batch)
step 99990, loss=0.50 (1979.7 examples/sec; 0.065 sec/batch)

'''评测模型在测试集上的准确率,测试集10000个样本(1)使用固定的batch_size,然后一个batch一个batch输入测试数据(2)计算一共多少个batch才能将全部样本测试完毕(3)每一个step使用session的run方法获取images_test, labels_test的batch(4)在执行top_k_top计算模型在这个batch的top 1上预测正确的样本数,最后汇总所有正确的预测结果'''num_examples = 10000import mathnum_iter = int(math.ceil(num_examples / batch_size))true_count = 0total_sample_count = num_iter * batch_sizestep = 0while step < num_iter :    image_batch, label_batch = sess.run([images_test, labels_test])    predictions = sess.run([top_k_op], feed_dict={image_holder: image_batch, label_holder: label_batch})    true_count += np.sum(predictions)    step += 1print true_countprint total_sample_count

8051
9984

'''最后将准确率的评测结果计算并打印出来'''precision = float(true_count) / float(total_sample_count)print true_countprint total_sample_countprint  ("分类正确率是: %.3f" % precision)

8051
9984
分类正确率是: 0.806