[深度学习] (2):实现简单的线性回归(使用TensorFlow优化)

摘要

在学习使用TensorFlow, 动手写了LR的一般实现和TF实现，用的都是GD优化，但是TF明显快啊

PS: TF绘图的时候还加入了动画效果23333 _(:з」∠)_

---

普通实现

import numpy as npimport matplotlib.pyplot as pltNUM_OF_POINTS = 100NOISE_SHIFT = 20N_ITER = 3000alpha = 0.001X = np.linspace(-10, 10, NUM_OF_POINTS)y = 2 * X + np.random.rand(NUM_OF_POINTS) * NOISE_SHIFTW = np.array([1])bias = 0cost_ = []for i in range(N_ITER):    error = X * W + bias - y    cost = np.sum(np.square(error)) / (2 * NUM_OF_POINTS)    cost_.append(cost)    W = W - alpha * np.sum(error * X) / NUM_OF_POINTS    bias = bias - alpha * np.sum(error) / NUM_OF_POINTS# plot the resultfig = plt.figure(figsize=(14,5))ax1 = fig.add_subplot(1,2,1)ax1.scatter(X, y, c='r')ax1.plot(X, X*W+bias, c='b')ax2 = fig.add_subplot(1,2,2)ax2.scatter(range(len(cost_)), cost_)plt.show()

结果如下

---

TensorFlow实现 (绘图的时候还加入了动画效果 : )

import tensorflow as tfimport numpy as npimport matplotlib.pyplot as plt# 演示一个简单的LR回归NUM_OF_POINTS = 100NOISE_SHIFT = 20N_ITER = 200ALPHA = tf.constant(0.0001)X_train = np.linspace(-10, 10, NUM_OF_POINTS)y_train = 2 * X_train + np.random.rand(NUM_OF_POINTS) * NOISE_SHIFT# Model parametersW = tf.Variable([.3], dtype=tf.float32)b = tf.Variable([-.3], dtype=tf.float32)# Model input and outputx = tf.placeholder(tf.float32)linear_model = W * x + by = tf.placeholder(tf.float32)# lossloss = tf.reduce_sum(tf.square(linear_model - y)) # sum of the squares# optimizeroptimizer = tf.train.GradientDescentOptimizer(ALPHA)train = optimizer.minimize(loss)# plot the training datafig = plt.figure(figsize=(14,5))ax = fig.add_subplot(1,2,1)ax.scatter(X_train, y_train)ax_r = fig.add_subplot(1,2,2)plt.ion()plt.show()with tf.Session() as sess:    # 对于Variable必须先init才可以使用    init = tf.global_variables_initializer()    sess.run(init)    loss_ = []    for i in range(N_ITER):        print(i)        # 去除红线        try:            ax.lines.remove(lines[0])        except Exception:            pass        sess.run(train, {x: X_train, y: y_train})        loss_.append(sess.run(loss, {x: X_train, y: y_train}))        prediction = sess.run(linear_model, feed_dict={x: X_train})        lines = ax.plot(X_train, prediction, 'r-', lw=5)        ax_r.scatter(i, sess.run(loss, {x: X_train, y: y_train}))        plt.pause(0.1)    print(sess.run([W, b]))# plt.close()# plt.scatter(range(len(loss_)), loss_)# plt.show()plt.pause(111)

---

总结

明显看到TF的GD比自己写的快了不少，在100次左右就几乎收敛了，而且步长还小，不知道底层是怎么优化的，直接解正规方程吗2333，还是用牛顿法优化2333，都不会思密达，继续学习，怀挺！