机器学习十大算法之线性回归

1. 线性回归的模型函数和损失函数

　　　　线性回归遇到的问题一般是这样的。我们有m个样本，每个样本对应于n维特征和一个结果输出，如下：

　　　　

　　　　我们的问题是，对于一个新的, 他所对应的是多少呢？ 如果这个问题里面的y是连续的，则是一个回归问题，否则是一个分类问题。

　　　　对于n维特征的样本数据，如果我们决定使用线性回归，那么对应的模型是这样的：

　　　　, 其中(i = 0,1,2... n)为模型参数， (i = 0,1,2... n)为每个样本的n个特征值。这个表示可以简化，我们增加一个特征 ，这样。

　　　　进一步用矩阵形式表达更加简洁如下：

　　　　

　　　　其中， 假设函数为mx1的向量,为nx1的向量，里面有n个代数法的模型参数。为mxn维的矩阵。m代表样本的个数，n代表样本的特征数。

　　　　得到了模型，我们需要求出需要的损失函数，一般线性回归我们用均方误差作为损失函数。损失函数的代数法表示如下：

　　　　

　　　　进一步用矩阵形式表达损失函数：

　　　　

　　　　由于矩阵法表达比较的简洁，后面我们将统一采用矩阵方式表达模型函数和损失函数。

2. 线性回归的算法

　　　　对于线性回归的损失函数J，我们常用的有两种方法来求损失函数最小化时候的参数：一种是梯度下降法，一种是最小二乘法。

　　　　如果采用梯度下降法，则θ的迭代公式是这样的：

　　　　

　　　　通过若干次迭代后，我们可以得到最终的θ的结果

　　　　如果采用最小二乘法，则的结果公式如下：

　　　　

当然线性回归，还有其他的常用算法，比如牛顿法和拟牛顿法，这里不详细描述。

3. 线性回归的推广：多项式回归

　　　　回到我们开始的线性模型，, 如果这里不仅仅是x的一次方，比如增加二次方，那么模型就变成了多项式回归。这里写一个只有两个特征的p次方多项式回归的模型：

　　　　

　　　　我们令

    我们就得到了下式：

　　　　

　　　　可以发现，我们又重新回到了线性回归，这是一个五元线性回归，可以用线性回归的方法来完成算法。对于每个二元样本特征,我们得到一个五元样本特征，通过这个改进的五元样本特征，我们重新把不是线性回归的函数变回线性回归。

4. 线性回归的推广：广义线性回归

　　　　在上一节的线性回归的推广中，我们对样本特征端做了推广，这里我们对于特征y做推广。比如我们的输出不满足和的线性关系，但是和满足线性关系，模型函数如下：

　　　　

　　　　这样对与每个样本的输入y，我们用 lny去对应， 从而仍然可以用线性回归的算法去处理这个问题。我们把 Iny一般化，假设这个函数是单调可微函数,则一般化的广义线性回归形式是：

　　　　或者 

　　　　这个函数我们称为联系函数。

5. 线性回归的正则化

　　　　为了防止模型的过拟合，我们在建立线性模型的时候经常需要加入正则化项。一般有L1正则化和L2正则化。

　　　　线性回归的L1正则化:Lasso回归,L1正则化的项有一个常数系数来调节损失函数的均方差项和正则化项的权重,如下：　　

　　　　

　　　　其中n为样本个数，为常数系数，需要进行调优。为L1范数。

 　　　　Lasso回归可以使得一些特征的系数变小，甚至还是一些绝对值较小的系数直接变为0。增强模型的泛化能力。Lasso回归的求解办法一般有坐标轴下降法（coordinate descent）和最小角回归法（ Least Angle Regression）

　　　　线性回归的L2正则化:Ridge回归，Ridge回归的正则化项是L2范数，如下：

　　　　

　　　　其中为常数系数，需要进行调优。为L2范数。

　　　　Ridge回归在不抛弃任何一个特征的情况下，缩小了回归系数，使得模型相对而言比较的稳定，但和Lasso回归比，这会使得模型的特征留的特别多，模型解释性差。

 　　　  Ridge回归的求解比较简单，一般用最小二乘法。这里给出用最小二乘法的矩阵推导形式，和普通线性回归类似。

　　　　令的导数为0，得到下式：

　　　　

　　　　整理即可得到最后的的结果：

　　　　

 　　　 其中E为单位矩阵。

练习代码：

#!/usr/bin/python # -*- coding: utf-8 -*- __author__ = "zhuxiaoxia"__version__ = "$Revision: 1.0 $" __date__ = "$Date: 2017/10/31 21:20:19 $"__copyright__ = "Copyright (c) 2017 zhuxiaoxia"__license__ = "Python"class LinearRegression:      def __init__(self):        self.__LEARNING_RATE = 0.1  #学习率lr        self.__MAX_FEATURE_CNT = 11   #样本数m        self.theta =[0,0,0,0,0,0,0,0,0,0,0]  #初始化权重delta        #训练样本         self.samples = [[1,1,0,1,0,0,1,0,0,1,1,4], \                        [1,0,1,0,0,1,0,0,3,0,0,0], \                        [1,0,0,2,2,0,1,0,0,0,0,0], \                        [1,2,0,1,0,1,1,0,0,0,0,2], \                        [1,0,1,0,0,0,0,0,3,1,0,2], \                        [1,0,1,0,0,1,1,1,1,0,0,1], \                        [1,0,1,0,0,1,0,1,0,1,1,4], \                        [1,2,0,1,1,0,0,0,0,0,1,3], \                        [1,0,1,0,0,2,0,0,0,2,0,4], \                        [1,0,0,1,0,1,1,0,0,1,1,3], \                        [1,0,1,0,0,0,2,0,0,1,1,3], \                        [1,1,1,1,1,0,0,1,0,0,0,2], \                        [1,0,0,0,0,0,0,2,1,1,1,5]]        #测试样本        self.test_cases = \            [[1,1,1,1,1,1,1,0,0,1,0,3], \            [1,1,0,1,0,1,0,1,0,0,1,3], \            [1,0,1,1,1,0,0,1,0,1,0,3], \            [1,0,1,0,0,0,1,1,0,2,0,5], \            [1,0,0,0,1,1,1,0,0,1,0,2], \            [1,1,1,0,0,0,0,1,0,1,1,5], \            [1,3,0,1,0,0,1,0,0,0,3,3], \            [1,0,1,0,0,0,1,1,1,0,0,1], \            [1,0,0,0,2,0,0,0,1,0,2,2], \            [1,0,1,0,0,3,1,0,0,0,0,0], \            [1,0,1,1,2,0,0,0,0,1,0,2]]    def __hypothesis(self,x):        h = 0        for idx in range(0,self.__MAX_FEATURE_CNT):            h+=x[idx]*self.theta[idx]        return h    def __update_theta(self,x,delta):        for idx in range(0,self.__MAX_FEATURE_CNT):            self.theta[idx]-=x[idx]*delta    def __train(self):        for x in self.samples:            h = self.__hypothesis(x[0:-1])            y = x[self.__MAX_FEATURE_CNT]            delta = (h-y)*self.__LEARNING_RATE            self.__update_theta(x,delta)    def __get_loss(self):        loss_sum = 0        for x in self.samples:            h = self.__hypothesis(x[0:-1])            y = x[self.__MAX_FEATURE_CNT]            loss_sum +=(h-y)*(h-y)/2            return loss_sum    def online_training(self):        for itr in range(0,100):            self.__train()  #更新theta            loss_sum = self.__get_loss()  #获取loss            for i in range(0,self.__MAX_FEATURE_CNT):                print "theta[%d] =%1f" %(i,self.theta[i])            print "第%d迭代的损失为%1f" %(itr,loss_sum)            if loss_sum<0.00001:                break    def test(self):        for t in self.test_cases:            h=self.__hypothesis(t[0:-1])            print "预测值H=%1f,真实值ANS=%d，误差loss=%f" %(h,t[self.__MAX_FEATURE_CNT],self.__get_loss())    def pt(self):        for i in range(0,self.__MAX_FEATURE_CNT):                print "theta[%d]=%1f" %(i,self.theta[i])if __name__=="__main__":    lr=LinearRegression()   #实例化线性回归类    lr.online_training()  #训练    print "-------train over!!!-----------"    lr.pt()    print "------------------------------"    lr.test()  #测试数据集

输出：

theta[0] =0.932768theta[1] =0.689504theta[2] =0.398498theta[3] =0.188364theta[4] =-0.238094theta[5] =0.357043theta[6] =0.178898theta[7] =0.599572theta[8] =0.554495theta[9] =1.090739theta[10] =0.844823第0迭代的损失为0.002805theta[0] =0.790254theta[1] =0.752568theta[2] =0.234935theta[3] =0.050874theta[4] =-0.339084theta[5] =0.049102theta[6] =-0.055743theta[7] =0.773089theta[8] =0.070681theta[9] =1.433472theta[10] =0.981377第1迭代的损失为0.001114theta[0] =0.739567theta[1] =0.762793theta[2] =0.155520theta[3] =0.022372theta[4] =-0.340424theta[5] =-0.076967theta[6] =-0.169710theta[7] =0.823717theta[8] =-0.088236theta[9] =1.596149theta[10] =1.012621第2迭代的损失为0.000655theta[0] =0.721642theta[1] =0.764553theta[2] =0.108136theta[3] =0.023634theta[4] =-0.325732theta[5] =-0.129802theta[6] =-0.218305theta[7] =0.834445theta[8] =-0.142559theta[9] =1.684289theta[10] =1.009551第3迭代的损失为0.000107theta[0] =0.715244theta[1] =0.767323theta[2] =0.075176theta[3] =0.029650theta[4] =-0.313209theta[5] =-0.153439theta[6] =-0.235075theta[7] =0.834937theta[8] =-0.160420theta[9] =1.735453theta[10] =0.995822第4迭代的损失为0.000035theta[0] =0.712808theta[1] =0.771849theta[2] =0.050200theta[3] =0.033593theta[4] =-0.304852theta[5] =-0.164903theta[6] =-0.237359theta[7] =0.834542theta[8] =-0.164998theta[9] =1.766492theta[10] =0.979475第5迭代的损失为0.000361theta[0] =0.711707theta[1] =0.777326theta[2] =0.030370theta[3] =0.034509theta[4] =-0.299348theta[5] =-0.170870theta[6] =-0.233504theta[7] =0.835425theta[8] =-0.164882theta[9] =1.786138theta[10] =0.963370第6迭代的损失为0.000782theta[0] =0.711062theta[1] =0.783034theta[2] =0.014156theta[3] =0.032983theta[4] =-0.295309theta[5] =-0.174087theta[6] =-0.227391theta[7] =0.837536theta[8] =-0.163344theta[9] =1.799183theta[10] =0.948549第7迭代的损失为0.001124theta[0] =0.710598theta[1] =0.788554theta[2] =0.000608theta[3] =0.029784theta[4] =-0.291837theta[5] =-0.175784theta[6] =-0.220767theta[7] =0.840347theta[8] =-0.161583theta[9] =1.808311theta[10] =0.935355第8迭代的损失为0.001343theta[0] =0.710233theta[1] =0.793688theta[2] =-0.010917theta[3] =0.025522theta[4] =-0.288453theta[5] =-0.176569theta[6] =-0.214378theta[7] =0.843385theta[8] =-0.159986theta[9] =1.815037theta[10] =0.923836第9迭代的损失为0.001455theta[0] =0.709946theta[1] =0.798371theta[2] =-0.020870theta[3] =0.020621theta[4] =-0.284943theta[5] =-0.176773theta[6] =-0.208506theta[7] =0.846336theta[8] =-0.158632theta[9] =1.820227theta[10] =0.913904第10迭代的损失为0.001488theta[0] =0.709730theta[1] =0.802601theta[2] =-0.029577theta[3] =0.015357theta[4] =-0.281238theta[5] =-0.176592theta[6] =-0.203224theta[7] =0.849029theta[8] =-0.157500theta[9] =1.824382theta[10] =0.905412第11迭代的损失为0.001472theta[0] =0.709578theta[1] =0.806409theta[2] =-0.037274theta[3] =0.009912theta[4] =-0.277338theta[5] =-0.176150theta[6] =-0.198514theta[7] =0.851391theta[8] =-0.156544theta[9] =1.827806theta[10] =0.898192第12迭代的损失为0.001425theta[0] =0.709483theta[1] =0.809838theta[2] =-0.044140theta[3] =0.004399theta[4] =-0.273280theta[5] =-0.175526theta[6] =-0.194325theta[7] =0.853407theta[8] =-0.155720theta[9] =1.830684theta[10] =0.892076第13迭代的损失为0.001360theta[0] =0.709437theta[1] =0.812929theta[2] =-0.050312theta[3] =-0.001104theta[4] =-0.269106theta[5] =-0.174775theta[6] =-0.190597theta[7] =0.855094theta[8] =-0.154995theta[9] =1.833141theta[10] =0.886908第14迭代的损失为0.001286theta[0] =0.709433theta[1] =0.815724theta[2] =-0.055896theta[3] =-0.006545theta[4] =-0.264864theta[5] =-0.173936theta[6] =-0.187268theta[7] =0.856485theta[8] =-0.154345theta[9] =1.835261theta[10] =0.882547第15迭代的损失为0.001208theta[0] =0.709463theta[1] =0.818261theta[2] =-0.060977theta[3] =-0.011892theta[4] =-0.260592theta[5] =-0.173037theta[6] =-0.184286theta[7] =0.857616theta[8] =-0.153754theta[9] =1.837104theta[10] =0.878869第16迭代的损失为0.001129theta[0] =0.709521theta[1] =0.820571theta[2] =-0.065623theta[3] =-0.017119theta[4] =-0.256326theta[5] =-0.172097theta[6] =-0.181603theta[7] =0.858525theta[8] =-0.153209theta[9] =1.838719theta[10] =0.875768第17迭代的损失为0.001051theta[0] =0.709602theta[1] =0.822683theta[2] =-0.069891theta[3] =-0.022211theta[4] =-0.252094theta[5] =-0.171134theta[6] =-0.179179theta[7] =0.859246theta[8] =-0.152703theta[9] =1.840140theta[10] =0.873152第18迭代的损失为0.000976theta[0] =0.709700theta[1] =0.824621theta[2] =-0.073828theta[3] =-0.027156theta[4] =-0.247921theta[5] =-0.170159theta[6] =-0.176979theta[7] =0.859809theta[8] =-0.152230theta[9] =1.841399theta[10] =0.870945第19迭代的损失为0.000904theta[0] =0.709811theta[1] =0.826405theta[2] =-0.077473theta[3] =-0.031948theta[4] =-0.243825theta[5] =-0.169183theta[6] =-0.174975theta[7] =0.860241theta[8] =-0.151785theta[9] =1.842519theta[10] =0.869080第20迭代的损失为0.000836theta[0] =0.709933theta[1] =0.828054theta[2] =-0.080857theta[3] =-0.036581theta[4] =-0.239819theta[5] =-0.168212theta[6] =-0.173141theta[7] =0.860564theta[8] =-0.151367theta[9] =1.843520theta[10] =0.867503第21迭代的损失为0.000772theta[0] =0.710061theta[1] =0.829582theta[2] =-0.084011theta[3] =-0.041055theta[4] =-0.235915theta[5] =-0.167252theta[6] =-0.171456theta[7] =0.860798theta[8] =-0.150971theta[9] =1.844418theta[10] =0.866167第22迭代的损失为0.000711theta[0] =0.710194theta[1] =0.831003theta[2] =-0.086957theta[3] =-0.045368theta[4] =-0.232121theta[5] =-0.166308theta[6] =-0.169903theta[7] =0.860959theta[8] =-0.150597theta[9] =1.845228theta[10] =0.865034第23迭代的损失为0.000655theta[0] =0.710330theta[1] =0.832329theta[2] =-0.089716theta[3] =-0.049521theta[4] =-0.228442theta[5] =-0.165385theta[6] =-0.168465theta[7] =0.861060theta[8] =-0.150241theta[9] =1.845962theta[10] =0.864072第24迭代的损失为0.000602theta[0] =0.710468theta[1] =0.833567theta[2] =-0.092305theta[3] =-0.053517theta[4] =-0.224881theta[5] =-0.164483theta[6] =-0.167132theta[7] =0.861112theta[8] =-0.149904theta[9] =1.846630theta[10] =0.863253第25迭代的损失为0.000553theta[0] =0.710605theta[1] =0.834728theta[2] =-0.094741theta[3] =-0.057357theta[4] =-0.221442theta[5] =-0.163606theta[6] =-0.165890theta[7] =0.861125theta[8] =-0.149584theta[9] =1.847239theta[10] =0.862554第26迭代的损失为0.000508theta[0] =0.710742theta[1] =0.835818theta[2] =-0.097036theta[3] =-0.061046theta[4] =-0.218123theta[5] =-0.162755theta[6] =-0.164730theta[7] =0.861107theta[8] =-0.149279theta[9] =1.847798theta[10] =0.861956第27迭代的损失为0.000466theta[0] =0.710877theta[1] =0.836844theta[2] =-0.099202theta[3] =-0.064586theta[4] =-0.214926theta[5] =-0.161931theta[6] =-0.163646theta[7] =0.861063theta[8] =-0.148989theta[9] =1.848313theta[10] =0.861444第28迭代的损失为0.000428theta[0] =0.711009theta[1] =0.837811theta[2] =-0.101249theta[3] =-0.067983theta[4] =-0.211849theta[5] =-0.161134theta[6] =-0.162628theta[7] =0.861000theta[8] =-0.148713theta[9] =1.848788theta[10] =0.861003第29迭代的损失为0.000392theta[0] =0.711139theta[1] =0.838723theta[2] =-0.103186theta[3] =-0.071240theta[4] =-0.208889theta[5] =-0.160364theta[6] =-0.161672theta[7] =0.860921theta[8] =-0.148450theta[9] =1.849228theta[10] =0.860623第30迭代的损失为0.000359theta[0] =0.711265theta[1] =0.839586theta[2] =-0.105022theta[3] =-0.074362theta[4] =-0.206046theta[5] =-0.159623theta[6] =-0.160771theta[7] =0.860831theta[8] =-0.148200theta[9] =1.849637theta[10] =0.860294第31迭代的损失为0.000329theta[0] =0.711388theta[1] =0.840403theta[2] =-0.106763theta[3] =-0.077353theta[4] =-0.203316theta[5] =-0.158909theta[6] =-0.159922theta[7] =0.860732theta[8] =-0.147961theta[9] =1.850019theta[10] =0.860008第32迭代的损失为0.000301theta[0] =0.711507theta[1] =0.841177theta[2] =-0.108416theta[3] =-0.080218theta[4] =-0.200696theta[5] =-0.158221theta[6] =-0.159119theta[7] =0.860628theta[8] =-0.147733theta[9] =1.850375theta[10] =0.859759第33迭代的损失为0.000276theta[0] =0.711622theta[1] =0.841911theta[2] =-0.109986theta[3] =-0.082961theta[4] =-0.198183theta[5] =-0.157561theta[6] =-0.158360theta[7] =0.860518theta[8] =-0.147515theta[9] =1.850709theta[10] =0.859541第34迭代的损失为0.000252theta[0] =0.711734theta[1] =0.842608theta[2] =-0.111480theta[3] =-0.085588theta[4] =-0.195774theta[5] =-0.156927theta[6] =-0.157641theta[7] =0.860407theta[8] =-0.147307theta[9] =1.851022theta[10] =0.859350第35迭代的损失为0.000231theta[0] =0.711842theta[1] =0.843270theta[2] =-0.112900theta[3] =-0.088102theta[4] =-0.193466theta[5] =-0.156318theta[6] =-0.156959theta[7] =0.860294theta[8] =-0.147109theta[9] =1.851317theta[10] =0.859181第36迭代的损失为0.000211theta[0] =0.711945theta[1] =0.843900theta[2] =-0.114253theta[3] =-0.090507theta[4] =-0.191254theta[5] =-0.155734theta[6] =-0.156312theta[7] =0.860181theta[8] =-0.146920theta[9] =1.851595theta[10] =0.859031第37迭代的损失为0.000193theta[0] =0.712045theta[1] =0.844499theta[2] =-0.115541theta[3] =-0.092809theta[4] =-0.189136theta[5] =-0.155174theta[6] =-0.155698theta[7] =0.860068theta[8] =-0.146739theta[9] =1.851857theta[10] =0.858898第38迭代的损失为0.000177theta[0] =0.712141theta[1] =0.845070theta[2] =-0.116768theta[3] =-0.095011theta[4] =-0.187108theta[5] =-0.154637theta[6] =-0.155114theta[7] =0.859957theta[8] =-0.146567theta[9] =1.852105theta[10] =0.858778第39迭代的损失为0.000161theta[0] =0.712233theta[1] =0.845613theta[2] =-0.117938theta[3] =-0.097118theta[4] =-0.185167theta[5] =-0.154123theta[6] =-0.154558theta[7] =0.859848theta[8] =-0.146402theta[9] =1.852340theta[10] =0.858671第40迭代的损失为0.000148theta[0] =0.712322theta[1] =0.846131theta[2] =-0.119053theta[3] =-0.099133theta[4] =-0.183309theta[5] =-0.153630theta[6] =-0.154029theta[7] =0.859742theta[8] =-0.146245theta[9] =1.852562theta[10] =0.858574第41迭代的损失为0.000135theta[0] =0.712407theta[1] =0.846625theta[2] =-0.120117theta[3] =-0.101060theta[4] =-0.181531theta[5] =-0.153158theta[6] =-0.153526theta[7] =0.859637theta[8] =-0.146095theta[9] =1.852773theta[10] =0.858487第42迭代的损失为0.000123theta[0] =0.712489theta[1] =0.847096theta[2] =-0.121132theta[3] =-0.102903theta[4] =-0.179830theta[5] =-0.152707theta[6] =-0.153046theta[7] =0.859536theta[8] =-0.145951theta[9] =1.852973theta[10] =0.858407第43迭代的损失为0.000113theta[0] =0.712567theta[1] =0.847544theta[2] =-0.122100theta[3] =-0.104665theta[4] =-0.178202theta[5] =-0.152274theta[6] =-0.152589theta[7] =0.859438theta[8] =-0.145814theta[9] =1.853163theta[10] =0.858334第44迭代的损失为0.000103theta[0] =0.712642theta[1] =0.847973theta[2] =-0.123025theta[3] =-0.106350theta[4] =-0.176646theta[5] =-0.151860theta[6] =-0.152153theta[7] =0.859343theta[8] =-0.145683theta[9] =1.853344theta[10] =0.858267第45迭代的损失为0.000094theta[0] =0.712714theta[1] =0.848382theta[2] =-0.123907theta[3] =-0.107961theta[4] =-0.175157theta[5] =-0.151465theta[6] =-0.151737theta[7] =0.859251theta[8] =-0.145558theta[9] =1.853516theta[10] =0.858205第46迭代的损失为0.000086theta[0] =0.712782theta[1] =0.848772theta[2] =-0.124750theta[3] =-0.109501theta[4] =-0.173733theta[5] =-0.151086theta[6] =-0.151341theta[7] =0.859162theta[8] =-0.145438theta[9] =1.853679theta[10] =0.858148第47迭代的损失为0.000079theta[0] =0.712848theta[1] =0.849144theta[2] =-0.125554theta[3] =-0.110974theta[4] =-0.172372theta[5] =-0.150724theta[6] =-0.150962theta[7] =0.859076theta[8] =-0.145324theta[9] =1.853835theta[10] =0.858096第48迭代的损失为0.000072theta[0] =0.712911theta[1] =0.849500theta[2] =-0.126322theta[3] =-0.112382theta[4] =-0.171070theta[5] =-0.150377theta[6] =-0.150601theta[7] =0.858994theta[8] =-0.145215theta[9] =1.853984theta[10] =0.858046第49迭代的损失为0.000066theta[0] =0.712972theta[1] =0.849839theta[2] =-0.127056theta[3] =-0.113728theta[4] =-0.169825theta[5] =-0.150046theta[6] =-0.150257theta[7] =0.858914theta[8] =-0.145110theta[9] =1.854126theta[10] =0.858001第50迭代的损失为0.000060theta[0] =0.713029theta[1] =0.850163theta[2] =-0.127757theta[3] =-0.115015theta[4] =-0.168635theta[5] =-0.149729theta[6] =-0.149928theta[7] =0.858838theta[8] =-0.145011theta[9] =1.854261theta[10] =0.857958第51迭代的损失为0.000055theta[0] =0.713085theta[1] =0.850473theta[2] =-0.128427theta[3] =-0.116245theta[4] =-0.167497theta[5] =-0.149426theta[6] =-0.149613theta[7] =0.858765theta[8] =-0.144915theta[9] =1.854390theta[10] =0.857918第52迭代的损失为0.000050theta[0] =0.713137theta[1] =0.850769theta[2] =-0.129066theta[3] =-0.117420theta[4] =-0.166409theta[5] =-0.149136theta[6] =-0.149313theta[7] =0.858695theta[8] =-0.144824theta[9] =1.854512theta[10] =0.857880第53迭代的损失为0.000046theta[0] =0.713188theta[1] =0.851052theta[2] =-0.129677theta[3] =-0.118544theta[4] =-0.165369theta[5] =-0.148859theta[6] =-0.149027theta[7] =0.858627theta[8] =-0.144737theta[9] =1.854630theta[10] =0.857845第54迭代的损失为0.000042theta[0] =0.713236theta[1] =0.851322theta[2] =-0.130261theta[3] =-0.119619theta[4] =-0.164375theta[5] =-0.148594theta[6] =-0.148753theta[7] =0.858563theta[8] =-0.144654theta[9] =1.854742theta[10] =0.857811第55迭代的损失为0.000038theta[0] =0.713283theta[1] =0.851580theta[2] =-0.130819theta[3] =-0.120646theta[4] =-0.163424theta[5] =-0.148341theta[6] =-0.148492theta[7] =0.858501theta[8] =-0.144575theta[9] =1.854848theta[10] =0.857780第56迭代的损失为0.000035theta[0] =0.713327theta[1] =0.851826theta[2] =-0.131352theta[3] =-0.121627theta[4] =-0.162515theta[5] =-0.148098theta[6] =-0.148242theta[7] =0.858442theta[8] =-0.144499theta[9] =1.854950theta[10] =0.857750第57迭代的损失为0.000032theta[0] =0.713369theta[1] =0.852061theta[2] =-0.131861theta[3] =-0.122566theta[4] =-0.161647theta[5] =-0.147867theta[6] =-0.148003theta[7] =0.858385theta[8] =-0.144426theta[9] =1.855048theta[10] =0.857722第58迭代的损失为0.000029theta[0] =0.713410theta[1] =0.852286theta[2] =-0.132348theta[3] =-0.123462theta[4] =-0.160817theta[5] =-0.147646theta[6] =-0.147775theta[7] =0.858330theta[8] =-0.144357theta[9] =1.855141theta[10] =0.857695第59迭代的损失为0.000027theta[0] =0.713448theta[1] =0.852501theta[2] =-0.132813theta[3] =-0.124320theta[4] =-0.160023theta[5] =-0.147434theta[6] =-0.147557theta[7] =0.858278theta[8] =-0.144290theta[9] =1.855230theta[10] =0.857670第60迭代的损失为0.000024theta[0] =0.713485theta[1] =0.852707theta[2] =-0.133258theta[3] =-0.125139theta[4] =-0.159264theta[5] =-0.147232theta[6] =-0.147349theta[7] =0.858228theta[8] =-0.144227theta[9] =1.855315theta[10] =0.857645第61迭代的损失为0.000022theta[0] =0.713521theta[1] =0.852903theta[2] =-0.133682theta[3] =-0.125922theta[4] =-0.158539theta[5] =-0.147039theta[6] =-0.147150theta[7] =0.858181theta[8] =-0.144166theta[9] =1.855396theta[10] =0.857623第62迭代的损失为0.000020theta[0] =0.713554theta[1] =0.853091theta[2] =-0.134088theta[3] =-0.126671theta[4] =-0.157846theta[5] =-0.146854theta[6] =-0.146960theta[7] =0.858135theta[8] =-0.144109theta[9] =1.855473theta[10] =0.857601第63迭代的损失为0.000019theta[0] =0.713587theta[1] =0.853270theta[2] =-0.134476theta[3] =-0.127386theta[4] =-0.157184theta[5] =-0.146677theta[6] =-0.146779theta[7] =0.858091theta[8] =-0.144053theta[9] =1.855547theta[10] =0.857580第64迭代的损失为0.000017theta[0] =0.713618theta[1] =0.853441theta[2] =-0.134847theta[3] =-0.128070theta[4] =-0.156550theta[5] =-0.146508theta[6] =-0.146605theta[7] =0.858050theta[8] =-0.144000theta[9] =1.855618theta[10] =0.857560第65迭代的损失为0.000015theta[0] =0.713647theta[1] =0.853605theta[2] =-0.135201theta[3] =-0.128724theta[4] =-0.155945theta[5] =-0.146347theta[6] =-0.146439theta[7] =0.858010theta[8] =-0.143950theta[9] =1.855685theta[10] =0.857542第66迭代的损失为0.000014theta[0] =0.713675theta[1] =0.853762theta[2] =-0.135540theta[3] =-0.129349theta[4] =-0.155366theta[5] =-0.146193theta[6] =-0.146281theta[7] =0.857972theta[8] =-0.143901theta[9] =1.855750theta[10] =0.857524第67迭代的损失为0.000013theta[0] =0.713702theta[1] =0.853911theta[2] =-0.135864theta[3] =-0.129946theta[4] =-0.154813theta[5] =-0.146045theta[6] =-0.146129theta[7] =0.857935theta[8] =-0.143855theta[9] =1.855812theta[10] =0.857507第68迭代的损失为0.000012theta[0] =0.713728theta[1] =0.854054theta[2] =-0.136173theta[3] =-0.130517theta[4] =-0.154285theta[5] =-0.145904theta[6] =-0.145985theta[7] =0.857900theta[8] =-0.143811theta[9] =1.855870theta[10] =0.857490第69迭代的损失为0.000011theta[0] =0.713753theta[1] =0.854191theta[2] =-0.136468theta[3] =-0.131062theta[4] =-0.153780theta[5] =-0.145770theta[6] =-0.145846theta[7] =0.857867theta[8] =-0.143769theta[9] =1.855927theta[10] =0.857475第70迭代的损失为0.000010theta[0] =0.713776theta[1] =0.854321theta[2] =-0.136751theta[3] =-0.131584theta[4] =-0.153297theta[5] =-0.145641theta[6] =-0.145714theta[7] =0.857835theta[8] =-0.143729theta[9] =1.855981theta[10] =0.857460第71迭代的损失为0.000009theta[0] =0.713799theta[1] =0.854446theta[2] =-0.137021theta[3] =-0.132082theta[4] =-0.152835theta[5] =-0.145518theta[6] =-0.145588theta[7] =0.857804theta[8] =-0.143690theta[9] =1.856032theta[10] =0.857446第72迭代的损失为0.000008theta[0] =0.713820theta[1] =0.854565theta[2] =-0.137279theta[3] =-0.132559theta[4] =-0.152394theta[5] =-0.145400theta[6] =-0.145467theta[7] =0.857775theta[8] =-0.143653theta[9] =1.856081theta[10] =0.857432第73迭代的损失为0.000008theta[0] =0.713841theta[1] =0.854679theta[2] =-0.137526theta[3] =-0.133014theta[4] =-0.151973theta[5] =-0.145288theta[6] =-0.145352theta[7] =0.857747theta[8] =-0.143618theta[9] =1.856128theta[10] =0.857419第74迭代的损失为0.000007theta[0] =0.713861theta[1] =0.854788theta[2] =-0.137762theta[3] =-0.133449theta[4] =-0.151570theta[5] =-0.145180theta[6] =-0.145241theta[7] =0.857720theta[8] =-0.143584theta[9] =1.856173theta[10] =0.857407第75迭代的损失为0.000006theta[0] =0.713879theta[1] =0.854892theta[2] =-0.137987theta[3] =-0.133865theta[4] =-0.151184theta[5] =-0.145078theta[6] =-0.145136theta[7] =0.857695theta[8] =-0.143552theta[9] =1.856216theta[10] =0.857395第76迭代的损失为0.000006theta[0] =0.713897theta[1] =0.854992theta[2] =-0.138202theta[3] =-0.134263theta[4] =-0.150816theta[5] =-0.144980theta[6] =-0.145035theta[7] =0.857670theta[8] =-0.143522theta[9] =1.856257theta[10] =0.857384第77迭代的损失为0.000005theta[0] =0.713914theta[1] =0.855087theta[2] =-0.138408theta[3] =-0.134643theta[4] =-0.150464theta[5] =-0.144886theta[6] =-0.144939theta[7] =0.857647theta[8] =-0.143492theta[9] =1.856296theta[10] =0.857374第78迭代的损失为0.000005theta[0] =0.713931theta[1] =0.855178theta[2] =-0.138605theta[3] =-0.135006theta[4] =-0.150128theta[5] =-0.144796theta[6] =-0.144847theta[7] =0.857625theta[8] =-0.143464theta[9] =1.856334theta[10] =0.857363第79迭代的损失为0.000004theta[0] =0.713947theta[1] =0.855265theta[2] =-0.138793theta[3] =-0.135353theta[4] =-0.149807theta[5] =-0.144710theta[6] =-0.144759theta[7] =0.857604theta[8] =-0.143437theta[9] =1.856369theta[10] =0.857354第80迭代的损失为0.000004theta[0] =0.713962theta[1] =0.855348theta[2] =-0.138972theta[3] =-0.135685theta[4] =-0.149499theta[5] =-0.144628theta[6] =-0.144675theta[7] =0.857583theta[8] =-0.143412theta[9] =1.856404theta[10] =0.857344第81迭代的损失为0.000004theta[0] =0.713976theta[1] =0.855427theta[2] =-0.139144theta[3] =-0.136002theta[4] =-0.149206theta[5] =-0.144550theta[6] =-0.144594theta[7] =0.857564theta[8] =-0.143387theta[9] =1.856436theta[10] =0.857335第82迭代的损失为0.000003theta[0] =0.713990theta[1] =0.855503theta[2] =-0.139308theta[3] =-0.136305theta[4] =-0.148925theta[5] =-0.144475theta[6] =-0.144517theta[7] =0.857545theta[8] =-0.143364theta[9] =1.856467theta[10] =0.857327第83迭代的损失为0.000003theta[0] =0.714003theta[1] =0.855576theta[2] =-0.139465theta[3] =-0.136595theta[4] =-0.148657theta[5] =-0.144404theta[6] =-0.144444theta[7] =0.857527theta[8] =-0.143341theta[9] =1.856497theta[10] =0.857319第84迭代的损失为0.000003theta[0] =0.714015theta[1] =0.855645theta[2] =-0.139615theta[3] =-0.136872theta[4] =-0.148400theta[5] =-0.144335theta[6] =-0.144374theta[7] =0.857510theta[8] =-0.143320theta[9] =1.856526theta[10] =0.857311第85迭代的损失为0.000003theta[0] =0.714027theta[1] =0.855711theta[2] =-0.139759theta[3] =-0.137136theta[4] =-0.148155theta[5] =-0.144270theta[6] =-0.144307theta[7] =0.857494theta[8] =-0.143299theta[9] =1.856553theta[10] =0.857303第86迭代的损失为0.000002theta[0] =0.714039theta[1] =0.855774theta[2] =-0.139896theta[3] =-0.137389theta[4] =-0.147921theta[5] =-0.144207theta[6] =-0.144243theta[7] =0.857479theta[8] =-0.143280theta[9] =1.856579theta[10] =0.857296第87迭代的损失为0.000002theta[0] =0.714050theta[1] =0.855835theta[2] =-0.140027theta[3] =-0.137631theta[4] =-0.147697theta[5] =-0.144148theta[6] =-0.144181theta[7] =0.857464theta[8] =-0.143261theta[9] =1.856604theta[10] =0.857289第88迭代的损失为0.000002theta[0] =0.714060theta[1] =0.855893theta[2] =-0.140152theta[3] =-0.137862theta[4] =-0.147483theta[5] =-0.144091theta[6] =-0.144123theta[7] =0.857450theta[8] =-0.143243theta[9] =1.856628theta[10] =0.857283第89迭代的损失为0.000002theta[0] =0.714070theta[1] =0.855948theta[2] =-0.140271theta[3] =-0.138083theta[4] =-0.147278theta[5] =-0.144036theta[6] =-0.144067theta[7] =0.857436theta[8] =-0.143226theta[9] =1.856651theta[10] =0.857277第90迭代的损失为0.000002theta[0] =0.714079theta[1] =0.856001theta[2] =-0.140386theta[3] =-0.138294theta[4] =-0.147083theta[5] =-0.143984theta[6] =-0.144013theta[7] =0.857423theta[8] =-0.143210theta[9] =1.856673theta[10] =0.857271第91迭代的损失为0.000001theta[0] =0.714089theta[1] =0.856051theta[2] =-0.140495theta[3] =-0.138496theta[4] =-0.146896theta[5] =-0.143934theta[6] =-0.143962theta[7] =0.857411theta[8] =-0.143194theta[9] =1.856693theta[10] =0.857265第92迭代的损失为0.000001theta[0] =0.714097theta[1] =0.856100theta[2] =-0.140599theta[3] =-0.138689theta[4] =-0.146718theta[5] =-0.143887theta[6] =-0.143913theta[7] =0.857399theta[8] =-0.143179theta[9] =1.856713theta[10] =0.857260第93迭代的损失为0.000001theta[0] =0.714106theta[1] =0.856146theta[2] =-0.140699theta[3] =-0.138873theta[4] =-0.146547theta[5] =-0.143841theta[6] =-0.143867theta[7] =0.857388theta[8] =-0.143165theta[9] =1.856732theta[10] =0.857255第94迭代的损失为0.000001theta[0] =0.714114theta[1] =0.856190theta[2] =-0.140795theta[3] =-0.139049theta[4] =-0.146384theta[5] =-0.143798theta[6] =-0.143822theta[7] =0.857377theta[8] =-0.143152theta[9] =1.856750theta[10] =0.857250第95迭代的损失为0.000001theta[0] =0.714121theta[1] =0.856232theta[2] =-0.140886theta[3] =-0.139217theta[4] =-0.146228theta[5] =-0.143756theta[6] =-0.143779theta[7] =0.857366theta[8] =-0.143139theta[9] =1.856768theta[10] =0.857245第96迭代的损失为0.000001theta[0] =0.714128theta[1] =0.856272theta[2] =-0.140973theta[3] =-0.139378theta[4] =-0.146079theta[5] =-0.143716theta[6] =-0.143739theta[7] =0.857356theta[8] =-0.143126theta[9] =1.856784theta[10] =0.857240第97迭代的损失为0.000001theta[0] =0.714135theta[1] =0.856311theta[2] =-0.141056theta[3] =-0.139532theta[4] =-0.145936theta[5] =-0.143678theta[6] =-0.143700theta[7] =0.857347theta[8] =-0.143114theta[9] =1.856800theta[10] =0.857236第98迭代的损失为0.000001theta[0] =0.714142theta[1] =0.856348theta[2] =-0.141136theta[3] =-0.139679theta[4] =-0.145800theta[5] =-0.143642theta[6] =-0.143662theta[7] =0.857338theta[8] =-0.143103theta[9] =1.856815theta[10] =0.857232第99迭代的损失为0.000001-------train over!!!-----------theta[0]=0.714142theta[1]=0.856348theta[2]=-0.141136theta[3]=-0.139679theta[4]=-0.145800theta[5]=-0.143642theta[6]=-0.143662theta[7]=0.857338theta[8]=-0.143103theta[9]=1.856815theta[10]=0.857232------------------------------预测值H=2.713385,真实值ANS=3，误差loss=0.000001预测值H=3.001739,真实值ANS=3，误差loss=0.000001预测值H=3.001680,真实值ANS=3，误差loss=0.000001预测值H=5.000312,真实值ANS=5，误差loss=0.000001预测值H=2.137853,真实值ANS=2，误差loss=0.000001预测值H=5.000739,真实值ANS=5，误差loss=0.000001预测值H=5.571539,真实值ANS=3，误差loss=0.000001预测值H=1.143579,真实值ANS=1，误差loss=0.000001预测值H=1.993903,真实值ANS=2，误差loss=0.000001预测值H=-0.001582,真实值ANS=0，误差loss=0.000001预测值H=1.998542,真实值ANS=2，误差loss=0.000001In [ ]:​