PCA降维

PCA简化数据

引言

不多说了，PCA就是用来降维操作的，将多维数据处理成维数比较少的数据，保留重要特征。

基本概念

PCA是主要成分分析（Principal component analysis， PCA）。 在PAC中数据从原来的坐标系转化到新的坐标系中，新的坐标系的选择是又数据本身决定的。第一个坐标轴的选取是原始数据中方差最大的方向，第二个坐标和第一个坐标正交且方差最大的方向，该过程是重复的。

过程原理

首先我们得到数据，第一个坐标轴的选取是原始数据中方差最大的方向，很明显是直线B，第二个坐标和第一个坐标正交且方差最大的方向，比较明显直线C符合要求，

再拿个例子，将下图二维降维到一维。

前面我们提到第一个主要的成分是数据中差异最大的方向选出来的，第二的成分则是数据差异性次大的方向，并且与第一个方向正交，通过数据的协方差的特征值和特征矩阵分析就可以得到这些数据。得到前N个重要的方向，然后在数据转到这前N个方向做的坐标上

伪代码：

去除均值    计算协方差矩阵    计算协方差矩阵的特征值和特征向量    将特征值降序    保留前N个特征向量    将数据转到N二哥特征向量建的新的空间中

重要的概率论和线性代数

首先来说一下协方差矩阵这个知识吧，对于方差这个知识应该比较熟悉吧，方差矩阵也不是什么难点。对于协方差了来说，百度给出的定义是协方差分析是建立在方差分析和回归分析基础之上的一种统计分析方法。 方差分析是从质量因子的角度探讨因素不同水平对实验指标影响的差异。

期望值分别为E[X]与E[Y]的两个实随机变量X与Y之间的协方差Cov(X,Y)定义为：

Cov(X,Y) = E[(X-E[X])(Y-E[Y])]

从直观上来看，协方差表示的是两个变量总体误差的期望。

说这些还是比较难理解协方差矩阵的意义，简洁的说吧，一个nxm的矩阵X，X的列协方差矩阵的是一个mxm的矩阵，对角线Kij(i = j)是这一列的方差。其他位置是对应的Kij是i列和j列的相关性，Kij为0时没有相关性。要得到方差最大又要得到最小相关度，那么最简单的就是对矩阵协方差做对角化，然后排序，最大的就是方差最大和相关度最小的。

其次重要的是线代中矩阵乘法的几何意义，这里我只给出结论：矩阵X和Y的乘法是将X在Y上的映射，即X以Y为基的映射，也可以是说X在Y上的投影。如果Y的维度要小，就达到了降维的目的。假如Z=X*Y那么，Z就是X在Y上的投影，也可以说Z是X经过Y变换来的。Z乘Y.T就把Z给变回来

再次重要的是线代中矩阵的特征值和特征向量，《线性代数》课本中是这么定义的：设A是n阶方阵，如果存在数λ和n维非零列向量x，使的成立，则称数λ是方阵A的特征值，非零向量x成为A的对应特征值λ的特征向量，λ值的大小表示A在x方向上的拉伸程度，我们通过特征值分解得到的前N个特征向量，那么就对应了这个矩阵最主要的N个变化方向。我们利用这前N个变化方向，就可以近似这个矩阵（变换）。也就是之前说的：提取这个矩阵最重要的特征。总结一下，特征值分解可以得到特征值与特征向量，特征值表示的是这个特征到底有多重要，而特征向量表示这个特征是什么，可以将每一个特征向量理解为一个线性的子空间，我们可以利用这些线性的子空间干很多的事情。详见链接的图文解释

代码详解

# coding=utf-8    import numpy as np    from numpy import *    import pylab as pl    def loadDataSet(fileName):        fr = open(fileName)        stringArr = [line.strip().split('\t') for line in fr.readlines()]        datArr = [map(float,line) for line in stringArr]        return mat(datArr)    def pca(dataMat, topNfeat = 9999999):        meanVals = mean(dataMat, axis = 0)        meanRemoved = dataMat - meanVals        print meanRemoved        #求出协方差矩阵        covMat = cov(meanRemoved, rowvar=0)        print 'covMat:', covMat        #求出协方差矩阵的特征值和特征向量        eigVal, eigVects = linalg.eig(mat(covMat))        print 'eigVal:', eigVal        print 'eigVects:', eigVects        #升序        eigValInd = argsort(eigVal)        print 'eigValInd:',eigValInd        #取出topNfeat个最大的        eigValInd = eigValInd[:-(topNfeat+1):-1]        print 'eigValInd:',eigValInd        #作为特征项        redEigVects = eigVects[:, eigValInd]        print redEigVects        #以原矩阵为基降维操作        lowDataMat = meanRemoved*redEigVects        #降维数据还原        reconMat = (lowDataMat * redEigVects.T) + meanVals        return  lowDataMat, reconMat    import matplotlib.pyplot as plt        fig = plt.figure()        ax = fig.add_subplot(111)        ax.scatter(dataMat[:,0].flatten().A[0], dataMat[:,1].flatten().A[0],marker='*',s=90, c='y')        ax.scatter(recondMat[:,0].flatten().A[0], recondMat[:,1].flatten().A[0],marker='o',s=50,c='g')        plt.show()    def main():        dataMat = loadDataSet(r'testSet.txt')        lowDMat , reconMat = pca(dataMat, 1)        print shape(lowDMat)        print reconMat        show(dataMat, reconMat)    if __name__ == '__main__':        main()

测试结果：

总结

PCA让我认识到线代和概率论的重要性，还要一些老师没有提到的几何意义

优点：降低数据的复杂性，识别最重要的多个特征

缺点：不一定需要，且可能损失有用的信息

PCA可以识别数据中的主要特征，通过数据沿最大方差方向旋转坐标轴实现的，具体过程还是要好好了解线代和概率论的知识，特别是这些几何意义。

数据（最下方没东西了）

10.235186   11.32199710.122339   11.8109939.190236    8.9049439.306371    9.8473948.330131    8.34035210.152785   10.12353210.408540   10.8219869.003615    10.0392069.534872    10.0969919.498181    10.8254469.875271    9.23342610.362276   9.37689210.191204   11.2508517.720499    6.4763009.334379    8.4712687.963186    6.7313338.244973    9.0137859.569196    10.5689498.854793    9.0765369.382171    7.2408628.179055    8.9445028.267896    8.7970179.047165    8.7250688.741043    7.9013857.190216    7.8045878.081227    9.3144318.047460    5.7207807.917584    7.5432548.676942    10.1022209.210251    9.4247177.732998    9.8402027.681754    8.6098977.925863    10.0791598.261509    8.2420808.514872    7.52756110.324450   10.8044817.856710    7.9315437.858608    7.9953409.196808    6.5775989.644415    10.9350819.579833    9.0850217.888484    5.9764289.072624    9.7033448.914184    9.2985157.822869    7.08666310.538554   11.0614648.280187    8.7090128.884223    8.6701059.359927    10.5750559.078611    9.7108337.935134    8.5861738.805945    10.5751459.584316    9.61407611.269714   11.7172549.120444    9.0197747.977520    8.3139238.104648    9.4561288.617126    7.3317239.033811    9.4697068.327680    5.1220928.532272    10.1009099.295434    8.9338249.905202    9.02755910.585764   10.91273310.427584   11.5325789.072767    9.9601449.164275    8.6451219.746058    10.7170809.286072    9.3400248.188233    7.4324157.948598    8.4454197.563350    5.6561788.972405    8.8018699.980868    8.7889967.753490    7.7142487.431143    9.0328198.943403    8.35935410.481890   9.9889699.150454    10.2787608.123894    9.0603518.626164    8.4693427.354185    7.63125211.323046   11.0150328.190008    6.8607928.412598    7.6613589.258404    8.58038211.007915   11.4438818.279403    8.3470038.931149    10.10522110.239245   10.0774738.129346    7.0968778.485823    9.37356110.703640   11.6516189.500728    8.1502289.712414    9.9104459.333374    9.4075578.787865    10.1680219.238180    10.2534789.577388    8.89515010.447753   10.3182279.303944    9.2231369.883268    11.6629459.471921    10.44379210.007753   9.5799128.110298    7.1062636.964069    6.58504010.413499   9.6493098.032629    7.0532548.015549    9.16675310.462924   8.6566129.530788    10.1341309.202658    9.31422210.103241   10.2351597.849264    6.6248569.059071    7.99255510.172889   10.7247899.528439    6.4209907.190422    6.7897929.085716    9.8463289.452887    8.7353867.417322    7.3485948.468639    8.7150868.303642    9.4632319.939052    10.0267718.701989    7.5169789.737541    10.5872818.280233    7.85244410.648386   10.2592039.173893    10.5203729.135397    10.7514067.594580    8.4888338.587520    8.4634068.581887    7.8886449.448768    8.7074227.882664    7.77203010.050635   9.8597209.012078    9.5338998.770020    8.8829969.428804    9.4463068.504209    8.3196939.800003    10.9646678.069660    7.68309910.012217   10.3206448.704677    8.9181468.198722    7.2977869.868322    9.9016579.426997    11.4803539.228767    9.2629768.952359    9.5284718.186847    8.6005879.026371    8.7051439.483364    9.8070797.826587    7.97540111.197846   10.9592987.632421    8.7697458.761605    8.3093659.353670    8.7287586.466637    6.0389968.370634    9.17883010.337451   11.0756008.917679    8.2883679.076621    8.4876267.278948    4.63409710.153017   11.2191837.132603    5.8531189.338644    9.8059409.878602    9.18700010.009505   10.9245059.384438    10.6918607.535322    8.1604816.808732    8.2684698.302965    8.0750098.345379    8.3053569.517530    8.2498399.267825    9.99910910.291511   11.0326648.605909    8.7052078.331145    7.8122958.632412    10.5742878.766397    8.7121079.407070    9.7327569.709495    9.72956910.422201   11.0703606.831495    6.4667638.187122    8.4059298.523093    9.0418447.952394    6.80122010.490780   10.00146810.813791   9.8024947.861113    7.5414758.800399    8.7389747.542152    6.6128389.446981    9.3786598.281684    7.3585728.473801    8.20834311.736767   11.0220298.379578    8.7143488.313718    8.8323819.342589    10.4166597.560710    6.8896489.295344    9.7390409.176612    9.7187818.614385    10.1505219.079373    8.83979410.333289   10.9212559.453502    7.33513410.174590   10.2925009.693713    9.7936367.474925    7.75139110.107905   10.1569979.257241    7.85426610.209794   11.4101577.248050    6.43367610.150091   9.28859710.077713   10.3215008.191122    8.9315198.791469    10.2872169.229434    9.0951938.682571    8.5460057.524099    7.7097518.442410    8.3260379.364851    9.0959899.061222    7.5578997.989999    8.5553638.801275    8.86873210.351932   9.49779610.230710   10.4961519.783163    9.89140810.651481   9.4316178.387393    6.4005079.003921    7.0500038.483723    8.3148869.020501    7.5457719.329105    11.0956619.583687    9.2719298.908705    8.4075298.835406    8.0835179.736362    8.29673510.030302   9.7371788.287142    6.9934609.173211    9.3063359.026355    9.6965319.128391    9.92124711.486346   12.91077711.519458   11.4721119.027707    10.2639749.351935    8.5422009.421701    11.4032019.005687    8.1009697.015279    6.6142788.213607    8.3409488.226646    8.7189978.144753    8.36687710.133642   12.79016910.763481   10.84701610.003622   10.3377169.007955    9.7924828.670506    10.78293110.386414   9.95616210.104761   10.1230448.079502    8.3040759.945424    11.8554098.642497    9.9980669.349722    8.6903289.034991    8.8264908.738746    7.5184648.919532    9.7403129.464136    10.44458810.710057   12.66685710.042007   10.5320918.447996    7.4263639.509351    9.03051611.946359   10.5530759.981617    9.9126519.853876    9.63296710.560648   11.8817148.370952    9.9894918.323209    10.1025299.828359    11.7024628.515623    8.4267549.004363    9.62803610.529847   10.45803110.028765   10.6248809.448114    9.3132278.332617    7.3822958.323006    8.2766087.740771    8.7997508.379615    8.1461928.340764    9.1844589.863614    8.2546949.969563    9.4051349.164394    9.18212710.622098   9.7225929.592072    10.0294468.212027    7.4773669.080225    8.2444488.555774    7.8423259.958046    9.6962218.972573    9.7971289.213223    7.1284378.737239    9.38513810.333907   10.9948568.797511    8.64307511.044848   9.6231608.539260    9.09711311.582163   11.8843337.863848    7.1761996.218103    5.2835629.120602    7.2501909.001166    9.6352038.081476    8.8442249.369802    8.2309118.768925    8.6669879.841098    8.54389610.451522   9.5495119.755402    9.1175227.988961    6.8698548.872507    9.78711810.363980   10.7166086.315671    5.7659539.638879    9.2023558.588126    8.0379668.947408    9.1443869.051130    7.1951329.321709    8.38066810.146531   9.7547459.843373    8.8914379.213148    11.7006327.630078    7.2947538.093088    7.9675907.488915    6.0906528.126036    8.5864728.760350    7.26898710.201347   9.1410137.838208    7.3077006.155653    5.5639977.767841    6.2545288.425656    8.61583210.362168   10.88681510.180024   10.3789349.794665    10.0478129.970394    9.6682797.030217    7.0604719.275414    9.09573810.314911   10.4565399.259774    8.20485110.023919   9.5583078.887540    9.8667049.851608    9.4109898.710882    7.2680129.017007    10.2176737.976369    9.0009798.738332    8.6647348.344510    8.9776008.959613    12.3242409.169982    8.6246357.487451    8.1548598.706316    7.7194559.564832    8.9404038.327775    9.0445099.734032    10.1952558.021343    6.4450929.081048    11.0243977.626651    6.54926310.725858   8.5753748.731381    8.30778810.394237   10.5968747.029311    7.6588329.517907    7.50990410.394064   10.06089810.752500   9.4316019.692431    10.3321309.651897    7.8768628.592329    10.09683710.212801   10.8274969.045043    9.2655248.901643    8.03611510.794525   9.31883011.040915   12.0217468.390836    9.6724699.840166    11.22656810.806810   12.2056338.924285    10.9340568.411251    8.2896727.808891    9.6632909.733437    8.4869588.300026    7.4773748.221756    10.2783089.096867    9.6196779.410116    9.28918810.097176   9.7684709.387954    8.8448559.376134    7.7046308.231599    9.1012039.910738    10.6948558.645689    7.7645898.090245    7.1095969.253483    9.8136729.331546    8.0393869.843256    10.2087929.713131    9.2476659.259369    10.70462210.243948   9.6958836.396262    6.4563908.936289    8.7038718.750846    9.3472736.497155    4.1302519.516552    10.1648489.125766    8.8587758.374387    7.3001148.132816    7.62110710.099505   9.1591349.356477    6.8699998.112934    7.5875477.265396    6.98703111.950505   13.71510910.745959   10.8221718.893270    7.8873326.003473    4.9602197.498851    6.45133410.162072   9.9359548.732617    9.1776799.300827    9.95236011.908436   12.2568019.371215    9.1886459.943640    9.2450377.386450    7.0468198.410374    8.2932187.830419    6.4402538.263140    8.27944611.448164   12.1923638.216533    9.1866289.316128    10.0466978.156927    6.8347929.951421    11.2405989.059607    8.45844610.476339   10.5604617.548200    7.2271279.432204    7.2367059.402750    9.12641311.188095   13.8534269.520201    11.0281318.884154    9.7640718.961105    8.8331178.549663    8.86576510.111708   10.5154629.024761    9.1693687.904149    8.0487569.240995    7.7961428.126538    6.1161257.442148    7.9313359.486821    10.0913599.834289    11.6947209.009714    11.5991709.761314    11.3440836.993941    6.5629888.659524    8.4101077.685363    8.0972977.793217    6.5191098.883454    9.2573478.781821    9.2319807.946281    7.6589788.523959    10.6464809.031525    8.6496488.317140    7.7589789.192417    11.1512188.408486    8.28218210.327702   11.4590488.389687    8.5487278.642250    7.0568708.833447    9.2676388.805261    8.3202819.726211    9.0959978.477631    9.5075309.738838    9.6521108.272108    7.5826969.258089    8.4959318.334144    8.8107668.150904    6.4860327.259669    7.27015611.034180   11.51995410.705432   10.6425278.388814    7.1591378.559369    7.8462847.187988    6.5193138.811453    7.7659008.492762    7.9929418.739752    8.50290910.150752   10.4202957.062378    5.3652898.448195    7.48000010.224333   11.5927509.533795    9.2128459.519492    7.6905019.661847    10.3761897.963877    8.59719310.184486   9.1367098.505234    9.1592108.187646    8.5186909.167590    9.4059178.612162    8.51875510.970868   10.3922299.603649    9.1410959.704263    8.8301789.657506    8.1324499.337882    11.0453069.521722    9.5377648.954197    8.7281798.635658    10.3526628.910816    9.0203179.900933    9.39200210.247105   8.2896499.571690    8.1712377.388627    7.6680718.354008    10.0745909.775598    8.8356968.768913    7.9836048.330199    8.4740988.169356    9.36117210.346522   10.0864347.976144    9.2667028.429648    7.86582411.261674   11.78858710.051066   10.1124258.954626    9.7893438.382220    8.1210129.820642    9.4264418.125950    9.6950878.646465    7.2918088.190202    8.0037378.773887    7.3061758.731000    10.3004369.163098    7.8167699.456346    9.2239229.645180    9.3240538.835060    8.9669159.325950    10.9432489.941912    9.5485359.282799    10.1194889.567591    9.4621648.529019    9.7680019.314824    10.1537278.264439    8.2738608.307262    8.2140369.122041    8.6578618.404258    8.3893657.828355    8.4194339.803180    10.1082868.662439    8.5819538.883265    8.9783778.012330    8.2624519.420258    8.9748787.015415    6.3659409.888832    11.1630369.677549    10.3464318.410158    7.9128999.464147    10.7629007.067227    7.0357179.320923    10.5830899.056917    8.7712418.110004    8.38778910.310021   10.9700148.211185    8.8096278.942883    8.8407469.479958    8.3287008.973982    8.7022918.519257    8.7648559.424556    8.9569117.222919    8.1777878.257007    9.7006199.778795    9.2961348.028806    8.5759749.886464    9.9650769.090552    6.9789309.605548    10.2567519.959004    9.6102298.308701    9.5091247.748293    9.6859338.311108    9.4281149.697068    10.2179569.582991    9.4787739.167265    10.19841210.329753   10.4066028.908819    7.42878910.072908   10.3932947.992905    9.2266298.907696    7.2693668.421948    9.3429687.481399    7.22503310.358408   10.1661308.786556    10.2799439.658701    11.37936710.167807   9.4175528.653449    8.6566818.020304    8.6712708.364348    10.0040689.119183    9.7881998.405504    9.74058011.020930   11.9043509.755232    9.51571310.059542   9.5897488.727131    9.7779987.666182    6.0286428.870733    8.3675019.340446    7.7072699.919283    10.7968137.905837    8.32603410.181187   10.0898658.797328    8.9819888.466272    7.76503210.335914   12.6205399.365003    8.6091158.011017    7.24948910.923993   13.9015137.074631    7.5587209.824598    8.8512978.861026    8.37085710.127296   10.86153510.548377   10.8556958.880470    7.9487618.901619    9.6747057.813710    9.24691210.128808   10.56066811.096699   10.9116448.551471    6.8715148.907241    8.67781510.571647   10.2948388.815314    8.8107258.453396    8.3392969.594819    11.48758010.714211   9.6289087.428788    7.71286910.892119   12.7477529.024071    11.1126927.803375    7.8470388.521558    8.8818489.742818    11.5202039.832836    9.1803968.703132    10.0284989.905029    11.34760610.037536   8.8826888.629995    8.3928639.583497    9.2196638.781687    9.6505989.344119    9.53702410.407510   9.2239297.244488    6.55902110.643616   10.2883838.757557    6.94790110.784590   11.23335010.028427   11.3300337.968361    6.8303088.925954    8.5391137.738692    7.1149878.192398    8.35201610.412017   12.4311228.208801    5.7776787.820077    7.7907209.542754    11.5425416.817938    7.4292297.365218    7.9567979.274391    7.9327009.546475    8.8034127.471734    6.7978708.016969    7.8480708.852701    8.4581148.215012    8.4683306.975507    6.8469809.435134    10.6097009.228075    9.3426228.388410    7.6378567.111456    9.2891639.403508    8.4826549.133894    8.34357510.670801   9.7508219.983542    10.07453710.012865   8.5370178.929895    8.9519097.666951    7.4736159.493839    7.8217838.894081    7.0594139.593382    9.8597329.126847    8.3957009.532945    9.8506969.459384    9.3842138.982743    8.21706210.107798   8.79077210.563574   9.0448908.278963    9.5187908.734960    10.4941299.597940    9.53089510.025478   9.50827010.335922   10.9740638.404390    8.1467487.108699    6.0384698.873951    7.4742278.731459    8.1544558.795146    7.5346876.407165    6.8103529.979312    10.2874308.786715    8.39673610.753339   10.36056710.508031   10.32197610.636925   10.19379710.614322   11.2154208.916411    8.9652868.112756    8.30476910.833109   10.4975428.319758    9.7276919.945336    11.82009710.150461   9.91471510.185024   10.3887229.793569    9.07995510.590128   11.8115968.505584    6.88428210.461428   10.7454398.755781    9.4184277.488249    7.17207210.238905   10.4286599.887827    10.4278218.529971    8.8382178.375208    10.2428378.901724    8.3983048.607694    9.1731988.691369    9.9642619.584578    9.64154610.265792   11.4050787.592968    6.6833558.692791    9.3890317.589852    6.00579310.550386   11.7365848.578351    7.2270557.526931    6.8751348.577081    9.8771159.272136    11.05092810.300809   10.6530598.642013    9.0066819.720491    10.2652029.029005    9.6469288.736201    7.9756038.672886    9.0707598.370633    8.4121709.483776    9.1833416.790842    7.5949929.842146    10.1568109.563336    7.9625328.724669    9.8707329.012145    9.1713269.116948    9.7911676.219094    7.9884209.468422    8.3599758.825231    8.4752089.572224    9.6964289.609128    8.4881759.428590    10.4689988.293266    8.6177019.423584    10.3556889.240796    9.51722810.915423   13.02625210.854684   11.1308669.226816    9.3917969.580264    10.3592357.289907    6.8982089.338857    10.3740259.523176    11.33219010.162233   10.3573968.873930    9.2073988.607259    7.7948048.852325    8.2157978.077272    6.5010428.169273    8.2696136.806421    7.5444238.793151    9.69154911.640981   11.3657029.544082    11.5765459.009266    9.6055969.726552    9.4267199.495888    10.6266248.683982    9.3378648.322105    8.6310998.887895    8.6449318.662659    11.3730259.263321    7.5360167.802624    7.1716258.773183    8.5615658.730443    10.1975968.942915    7.7583838.057618    8.7749968.112081    8.20234910.378884   12.1037559.248876    8.6372499.739599    9.7085768.126345    8.2784878.894788    7.9661179.683165    9.01922110.886957   12.0538439.668852    10.9021327.486692    6.4711388.794850    9.1736098.835915    8.2967279.443984    11.3753448.696621    6.4345809.645560    9.2337229.623857    7.91559010.840632   12.6202687.298135    7.3561419.639644    8.9023899.849802    7.68262410.609964   10.2596159.768229    11.3828117.646351    7.57184910.230300   9.4708598.224402    8.4968666.879671    8.3936487.976247    8.6672219.183268    8.69455011.471853   12.78628010.428349   10.6157268.090828    5.9025049.738627    8.4857928.139709    8.3963339.508055    8.9905298.857260    8.4977328.902558    7.0144339.660607    11.0408338.772221    10.51215011.020038   9.3541347.918527    7.7420627.630835    7.75626011.043272   11.0416139.299376    8.6741579.795087    8.4318379.415683    8.3121017.942037    6.9429139.724790    11.76649610.222032   11.5508768.894163    8.3060208.394309    8.0704209.012776    6.8805489.661093    10.1389219.896472    9.7623729.135628    8.7599288.762656    10.3060288.602473    8.86195610.085297   10.46477410.644983   10.9457679.034571    8.3916688.602920    8.5019448.224766    7.4027588.755050    9.4310859.669937    8.64104910.693530   10.2871249.462806    7.6111539.287707    10.08236310.941260   10.7837289.263080    7.91332810.167111   10.2253388.783830    9.4653458.958624    8.6621369.841649    9.9267817.205691    6.7906388.629089    9.1354617.469440    8.4504428.179133    7.7904348.083984    7.8755209.271300    8.1353598.652349    8.2543977.983920    6.6096847.836860    9.7852387.418535    7.0112568.458288    10.0953649.387605    9.7269118.663951    8.20670510.146507   11.6985778.937103    10.99092411.218687   11.1419458.363142    9.1069367.877643    7.1229229.620978    9.9056899.509649    10.7732096.748743    6.7053859.300919    8.0850299.332257    9.8187917.898610    8.3666439.841914    9.4806756.920484    8.9595018.544713    9.5631368.162266    6.7152778.659552    9.28200810.673398   13.1748249.024000    10.3792388.183292    6.64757210.544919   10.6496027.201266    6.5296059.557407    11.0968218.304605    6.9409299.742855    9.92089710.024587   9.64522210.002296   9.9989408.965876    8.6654197.823136    6.9495728.125088    7.6540656.569589    6.04686310.195497   8.68912911.730011   10.3742218.739105    7.4575719.820059    10.2785269.547456    10.3981988.375072    8.4163028.889533    8.3089298.861201    9.29040812.677687   12.7884639.100735    8.6205377.728350    6.3282197.955373    8.3550288.733352    8.64541410.257527   11.1918139.246413    9.4970149.745302    9.6420357.785652    8.1476217.431673    8.5663998.654384    8.4667018.475392    6.7446779.968440    10.76519210.163616   10.80696310.238135   10.0366369.902889    10.7467309.523850    8.7497089.214363    9.1491789.266040    10.8415028.494292    7.77094210.821158   10.4101928.645888    7.9703089.885204    10.0980809.084990    10.8863499.277874    8.8714498.135131    7.1370647.917379    9.0805229.685586    8.8228508.558141    7.8481129.502917    10.0612556.409004    5.16477410.149235   10.5799517.847304    8.4113518.846930    6.8199398.675153    9.4111479.476276    9.06150811.099184   10.6442638.792411    10.3794058.400418    7.0727068.555713    7.9238058.024763    8.4269938.642696    10.4534127.906117    7.9204088.793393    9.7228788.280364    7.6698549.387766    9.7062459.626853    10.76249910.163631   10.9190079.375543    11.5135249.309440    8.57569910.055329   10.2972558.706241    9.09717210.032934   11.95189710.812974   11.31143510.352603   10.8198658.276870    9.0554038.397389    7.9444349.371741    10.39579010.825710   10.1440999.158483    11.38538210.658639   11.3898568.091762    6.63103910.734892   10.05459811.535880   11.6049129.799077    11.3716778.478725    9.0784559.399902    8.9477447.305377    8.1449737.613377    6.66879810.681308   10.8308459.973855    10.0041339.369918    7.8554338.838223    7.4290339.521831    10.6239309.724419    10.4474528.890224    9.2759239.932763    11.58995310.839337   9.0512508.497708    7.5217018.440236    8.7056709.063566    9.7557448.449647    8.9294858.554576    8.06323110.348606   10.5507185.985254    5.1868449.931937    10.1755829.854922    9.2013939.114580    9.13421510.334899   8.543604