TensorFlow－３－Basic Models

https://web.stanford.edu/class/cs20si/index.html
tensorflow分开定义计算过程

第1阶段：组装图

第2阶段：使用会话在图中执行操作。

常量值存储在图形定义中,会话分配内存以存储变量值。使用ｐｙｔｈｏｎ属性使得变量在首次调用的时候才加载（）

第1阶段：组装图

Step 1: Read in data

这里写代码片

Step 2: Create placeholders for inputs and labels

tf.placeholder(dtype, shape=None, name=None)

Step 3: Create weight and bias

tf.Variable(initial_value=None, trainable=True, collections=None, name=None, dtype=None, ...)

Step 4: Build model to predict Y

Y_predicted = X * w + b

Step 5: Specify loss function

tf.square(Y - Y_predicted, name="loss")

Step 6: Create optimizer

tf.train.GradientDescentOptimizer(learning_rate=0.001).minimize(loss)

第2阶段：使用会话在图中执行操作－训练模型。

Initialize variablesRun optimizer op(with data fed into placeholders for inputs and labels)

用TensorBoard查看模型See your model in TensorBoard

Step 1: writer = tf.summary.FileWriter('./my_graph/03/linear_reg',sess.graph)Step 2: $ tensorboard --logdir='./my_graph'

用matplotlib显示结果Plot the results with matplotlib

Step 1: Uncomment the plotting code at the end of your programStep 2: Run it again

If run into problem of matplotlib in virtual environment, go to
GitHub/setups and see the file possible setup problems

讨论两个问题
１　怎么知道模型是正确的？
２　怎么提高模型？
首先对于第一个问题，怎么知道模型是正确的？用ｌｏｓｓ函数来判断。loss是衡量预测值与真实值的偏差程度。

l1_loss，l2_loss
http://www.jianshu.com/p/ac26866e81bc

Huber loss
如果预测值与真实值相差很小，就取其平方，如果预测值与真实值相差很大，就去绝对值。

def huber_loss(labels, predictions, delta=1.0):　　　　residual = tf.abs(predictions - labels)　　　　condition = tf.less(residual, delta)　　　　small_res = 0.5 * tf.square(residual)　　　　large_res = delta * residual - 0.5 * tf.square(delta)　　　　return tf.select(condition, small_res, large_res)

例子
１　定义数据feed_dict with placeholder　or variables

""" Example to demonstrate the use of feed_dictAuthor: Chip HuyenPrepared for the class CS 20SI: "TensorFlow for Deep Learning Research"cs20si.stanford.edu"""import osos.environ['TF_CPP_MIN_LOG_LEVEL']='2'import tensorflow as tf# Example 1: feed_dict with placeholder# create a placeholder of type float 32-bit, value is a vector of 3 elementsa = tf.placeholder(tf.float32, shape=[3])# create a constant of type float 32-bit, value is a vector of 3 elementsb = tf.constant([5, 5, 5], tf.float32)# use the placeholder as you would a constantc = a + b  # short for tf.add(a, b)with tf.Session() as sess:    # print(sess.run(c)) # InvalidArgumentError because a doesn’t have any value    # feed [1, 2, 3] to placeholder a via the dict {a: [1, 2, 3]}    # fetch value of c    print(sess.run(c, {a: [1, 2, 3]})) # >> [6. 7. 8.]# Example 2: feed_dict with variablesa = tf.add(2, 5)b = tf.multiply(a, 3)with tf.Session() as sess:    # define a dictionary that says to replace the value of 'a' with 15    replace_dict = {a: 15}    # Run the session, passing in 'replace_dict' as the value to 'feed_dict'    print(sess.run(b, feed_dict=replace_dict)) # >> 45

2 延迟加载

""" Example to demonstrate how the graph definition getsbloated because of lazy loadingAuthor: Chip HuyenPrepared for the class CS 20SI: "TensorFlow for Deep Learning Research"cs20si.stanford.edu"""import osos.environ['TF_CPP_MIN_LOG_LEVEL']='2'import tensorflow as tf ######################################## ## NORMAL LOADING                     #### print out a graph with 1 Add node  ## ########################################x = tf.Variable(10, name='x')y = tf.Variable(20, name='y')z = tf.add(x, y)＃定义了一个ｎｏｄｅ占用内存但是没有计算with tf.Session() as sess:    sess.run(tf.global_variables_initializer())    writer = tf.summary.FileWriter('./graphs/l2', sess.graph)    for _ in range(10):        sess.run(z)＃调用的时候才计算    print(tf.get_default_graph().as_graph_def())    writer.close()######################################## ## LAZY LOADING                       #### print out a graph with 10 Add nodes## ########################################x = tf.Variable(10, name='x')y = tf.Variable(20, name='y')with tf.Session() as sess:    sess.run(tf.global_variables_initializer())    writer = tf.summary.FileWriter('./graphs/l2', sess.graph)    for _ in range(10):        sess.run(tf.add(x, y))＃直接在调用的时候定义节点，也就是用的时候再定义的意思吧    print(tf.get_default_graph().as_graph_def())     writer.close()

简单ＴＦ例子：

""" Some simple TensorFlow's opsAuthor: Chip HuyenPrepared for the class CS 20SI: "TensorFlow for Deep Learning Research"cs20si.stanford.edu"""import osos.environ['TF_CPP_MIN_LOG_LEVEL']='2'import numpy as npimport tensorflow as tfa = tf.constant(2)b = tf.constant(3)x = tf.add(a, b)with tf.Session() as sess:    writer = tf.summary.FileWriter('./graphs', sess.graph)     print(sess.run(x))writer.close() # close the writer when you’re done using ita = tf.constant([2, 2], name='a')b = tf.constant([[0, 1], [2, 3]], name='b')x = tf.multiply(a, b, name='dot_product')with tf.Session() as sess:    print(sess.run(x))# >> [[0 2]#    [4 6]]tf.zeros(shape, dtype=tf.float32, name=None)#creates a tensor of shape and all elements will be zeros (when ran in session)x = tf.zeros([2, 3], tf.int32) y = tf.zeros_like(x, optimize=True)print(y)print(tf.get_default_graph().as_graph_def())with tf.Session() as sess:    y = sess.run(y)with tf.Session() as sess:    print(sess.run(tf.linspace(10.0, 13.0, 4)))    print(sess.run(tf.range(5)))    for i in np.arange(5):        print(i)samples = tf.multinomial(tf.constant([[1., 3., 1]]), 5)with tf.Session() as sess:    for _ in range(10):        print(sess.run(samples))t_0 = 19 x = tf.zeros_like(t_0) # ==> 0y = tf.ones_like(t_0) # ==> 1with tf.Session() as sess:    print(sess.run([x, y]))t_1 = ['apple', 'peach', 'banana']x = tf.zeros_like(t_1) # ==> ['' '' '']y = tf.ones_like(t_1) # ==> TypeError: Expected string, got 1 of type 'int' instead.t_2 = [[True, False, False],       [False, False, True],       [False, True, False]] x = tf.zeros_like(t_2) # ==> 2x2 tensor, all elements are Falsey = tf.ones_like(t_2) # ==> 2x2 tensor, all elements are Truewith tf.Session() as sess:    print(sess.run([x, y]))with tf.variable_scope('meh') as scope:    a = tf.get_variable('a', [10])    b = tf.get_variable('b', [100])writer = tf.summary.FileWriter('test', tf.get_default_graph())x = tf.Variable(2.0)y = 2.0 * (x ** 3)z = 3.0 + y ** 2grad_z = tf.gradients(z, [x, y])with tf.Session() as sess:    sess.run(x.initializer)    print(sess.run(grad_z))

ＴＦ变量操作

"""Example to demonstrate the ops of tf.Variables()"""import osos.environ['TF_CPP_MIN_LOG_LEVEL']='2'import tensorflow as tf# Example 1: how to run assign opW = tf.Variable(10)assign_op = W.assign(100)with tf.Session() as sess:    sess.run(W.initializer)    print(W.eval()) # >> 10    print(sess.run(assign_op)) # >> 100# Example 2: tricky example# create a variable whose original value is 2my_var = tf.Variable(2, name="my_var") # assign 2 * my_var to my_var and run the op my_var_times_twomy_var_times_two = my_var.assign(2 * my_var)with tf.Session() as sess:    sess.run(tf.global_variables_initializer())    print(sess.run(my_var_times_two)) # >> 4    print(sess.run(my_var_times_two)) # >> 8    print(sess.run(my_var_times_two)) # >> 16# Example 3: each session maintains its own copy of variablesW = tf.Variable(10)sess1 = tf.Session()sess2 = tf.Session()# You have to initialize W at each sessionsess1.run(W.initializer)sess2.run(W.initializer)print(sess1.run(W.assign_add(10))) # >> 20print(sess2.run(W.assign_sub(2))) # >> 8print(sess1.run(W.assign_add(100))) # >> 120print(sess2.run(W.assign_sub(50))) # >> -42sess1.close()sess2.close()

线性回归

""" Simple linear regression example in TensorFlowThis program tries to predict the number of thefts from the number of fire in the city of ChicagoAuthor: Chip HuyenPrepared for the class CS 20SI: "TensorFlow for Deep Learning Research"cs20si.stanford.edu"""import osos.environ['TF_CPP_MIN_LOG_LEVEL']='2'import numpy as npimport matplotlib.pyplot as pltimport tensorflow as tfimport xlrdimport utilsDATA_FILE = 'data/fire_theft.xls'# Step 1: read in data from the .xls filebook = xlrd.open_workbook(DATA_FILE, encoding_override="utf-8")sheet = book.sheet_by_index(0)data = np.asarray([sheet.row_values(i) for i in range(1, sheet.nrows)])n_samples = sheet.nrows - 1# Step 2: create placeholders for input X (number of fire) and label Y (number of theft)X = tf.placeholder(tf.float32, name='X')Y = tf.placeholder(tf.float32, name='Y')# Step 3: create weight and bias, initialized to 0w = tf.Variable(0.0, name='weights')b = tf.Variable(0.0, name='bias')# Step 4: build model to predict YY_predicted = X * w + b # Step 5: use the square error as the loss functionloss = tf.square(Y - Y_predicted, name='loss')# loss = utils.huber_loss(Y, Y_predicted)# Step 6: using gradient descent with learning rate of 0.01 to minimize lossoptimizer = tf.train.GradientDescentOptimizer(learning_rate=0.001).minimize(loss)with tf.Session() as sess:    # Step 7: initialize the necessary variables, in this case, w and b    sess.run(tf.global_variables_initializer())     writer = tf.summary.FileWriter('./graphs/linear_reg', sess.graph)    # Step 8: train the model    for i in range(50): # train the model 100 epochs        total_loss = 0        for x, y in data:            # Session runs train_op and fetch values of loss            _, l = sess.run([optimizer, loss], feed_dict={X: x, Y:y})             total_loss += l        print('Epoch {0}: {1}'.format(i, total_loss/n_samples))    # close the writer when you're done using it    writer.close()     # Step 9: output the values of w and b    w, b = sess.run([w, b]) # plot the resultsX, Y = data.T[0], data.T[1]plt.plot(X, Y, 'bo', label='Real data')plt.plot(X, X * w + b, 'r', label='Predicted data')plt.legend()plt.show()

线性回归预测芝加哥的火灾

""" Simple linear regression example in TensorFlowThis program tries to predict the number of thefts from the number of fire in the city of ChicagoAuthor: Chip HuyenPrepared for the class CS 20SI: "TensorFlow for Deep Learning Research"cs20si.stanford.edu"""import osos.environ['TF_CPP_MIN_LOG_LEVEL']='2'import numpy as npimport matplotlib.pyplot as pltimport tensorflow as tfimport xlrdimport utilsDATA_FILE = 'data/fire_theft.xls'# Phase 1: Assemble the graph# Step 1: read in data from the .xls filebook = xlrd.open_workbook(DATA_FILE, encoding_override='utf-8')sheet = book.sheet_by_index(0)data = np.asarray([sheet.row_values(i) for i in range(1, sheet.nrows)])n_samples = sheet.nrows - 1# Step 2: create placeholders for input X (number of fire) and label Y (number of theft)# Both have the type float32# Step 3: create weight and bias, initialized to 0# name your variables w and b# Step 4: predict Y (number of theft) from the number of fire# name your variable Y_predicted# Step 5: use the square error as the loss function# name your variable loss# Step 6: using gradient descent with learning rate of 0.01 to minimize loss# Phase 2: Train our modelwith tf.Session() as sess:    # Step 7: initialize the necessary variables, in this case, w and b    # TO - DO       # Step 8: train the model    for i in range(50): # run 100 epochs        total_loss = 0        for x, y in data:            # Session runs optimizer to minimize loss and fetch the value of loss. Name the received value as l            # TO DO: write sess.run()            total_loss += l        print("Epoch {0}: {1}".format(i, total_loss/n_samples))# plot the results# X, Y = data.T[0], data.T[1]# plt.plot(X, Y, 'bo', label='Real data')# plt.plot(X, X * w + b, 'r', label='Predicted data')# plt.legend()# plt.show()

线性回归ＯＣＲ识别

""" Simple logistic regression model to solve OCR task with MNIST in TensorFlowMNIST dataset: yann.lecun.com/exdb/mnist/Author: Chip HuyenPrepared for the class CS 20SI: "TensorFlow for Deep Learning Research"cs20si.stanford.edu"""import osos.environ['TF_CPP_MIN_LOG_LEVEL']='2'import numpy as npimport tensorflow as tffrom tensorflow.examples.tutorials.mnist import input_dataimport time# Define paramaters for the modellearning_rate = 0.01batch_size = 128n_epochs = 30# Step 1: Read in data# using TF Learn's built in function to load MNIST data to the folder data/mnistmnist = input_data.read_data_sets('/data/mnist', one_hot=True) # Step 2: create placeholders for features and labels# each image in the MNIST data is of shape 28*28 = 784# therefore, each image is represented with a 1x784 tensor# there are 10 classes for each image, corresponding to digits 0 - 9. # each lable is one hot vector.X = tf.placeholder(tf.float32, [batch_size, 784], name='X_placeholder') Y = tf.placeholder(tf.int32, [batch_size, 10], name='Y_placeholder')# Step 3: create weights and bias# w is initialized to random variables with mean of 0, stddev of 0.01# b is initialized to 0# shape of w depends on the dimension of X and Y so that Y = tf.matmul(X, w)# shape of b depends on Yw = tf.Variable(tf.random_normal(shape=[784, 10], stddev=0.01), name='weights')b = tf.Variable(tf.zeros([1, 10]), name="bias")# Step 4: build model# the model that returns the logits.# this logits will be later passed through softmax layerlogits = tf.matmul(X, w) + b # Step 5: define loss function# use cross entropy of softmax of logits as the loss functionentropy = tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=Y, name='loss')loss = tf.reduce_mean(entropy) # computes the mean over all the examples in the batch# Step 6: define training op# using gradient descent with learning rate of 0.01 to minimize lossoptimizer = tf.train.AdamOptimizer(learning_rate).minimize(loss)with tf.Session() as sess:    # to visualize using TensorBoard    writer = tf.summary.FileWriter('./graphs/logistic_reg', sess.graph)    start_time = time.time()    sess.run(tf.global_variables_initializer())     n_batches = int(mnist.train.num_examples/batch_size)    for i in range(n_epochs): # train the model n_epochs times        total_loss = 0        for _ in range(n_batches):            X_batch, Y_batch = mnist.train.next_batch(batch_size)            _, loss_batch = sess.run([optimizer, loss], feed_dict={X: X_batch, Y:Y_batch})             total_loss += loss_batch        print('Average loss epoch {0}: {1}'.format(i, total_loss/n_batches))    print('Total time: {0} seconds'.format(time.time() - start_time))    print('Optimization Finished!') # should be around 0.35 after 25 epochs    # test the model    preds = tf.nn.softmax(logits)    correct_preds = tf.equal(tf.argmax(preds, 1), tf.argmax(Y, 1))    accuracy = tf.reduce_sum(tf.cast(correct_preds, tf.float32)) # need numpy.count_nonzero(boolarr) :(    n_batches = int(mnist.test.num_examples/batch_size)    total_correct_preds = 0    for i in range(n_batches):        X_batch, Y_batch = mnist.test.next_batch(batch_size)        accuracy_batch = sess.run([accuracy], feed_dict={X: X_batch, Y:Y_batch})         total_correct_preds += accuracy_batch       print('Accuracy {0}'.format(total_correct_preds/mnist.test.num_examples))    writer.close()