Fisher准则线性分类器的Python实现

来源：互联网发布：创业知乎编辑：程序博客网时间：2024/06/06 07:24

Fisher准则线性分类器的Python实现

Fisher准则线性分类器的Python实现
- 选取的训练集与测试集
- 分类决策与分类器
- 代码
- 测试集上的结果

本节内容：本节内容是根据上学期所上的模式识别课程的作业整理而来，第二道题目是线性分类器设计，数据集是Iris(鸢尾花的数据集)，根据前一题的Kmeans聚类得出的结果，分成训练集与测试集，进行比较。

选取的训练集与测试集

训练集：(选取上一题中的第一种结果:每一类大小都是l=0.67*len(dataset[i])，前l个数据)
第一类（33个）：[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21,
22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32],
第二类（25个）：[52, 77, 100, 102, 103, 104, 105, 107, 108, 109, 110, 111, 112, 115, 116, 117, 118, 120, 122, 124, 125, 128, 129, 130, 131],
第三类（41个）：[50, 51, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92]]
测试集
第一类(17个)：[[33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49],
第二类(13个）：[132, 134, 135, 136, 137, 139, 140, 141, 143, 144, 145, 147, 148],
第三类(21个)： [93, 94, 95, 96, 97, 98, 99, 101, 106, 113, 114, 119, 121, 123, 126, 127, 133, 138, 142, 146, 149]]

分类决策与分类器

分类决策：一对一
分类器：Fisher线性分类器
g12(x)=WT12x−W012,g13(x)=WT13x−W013,g23(x)=WT23x−W023,g21(x)=−g12(x)，g31(x)=−g13(x)，g32(x)=−g23(x)
相关参数
类间离散度矩阵：S12=(m1−m2)(m1−m2)T
类内离散度矩阵：Sw=∑xi∈ωi(xi−m1)(xi−m1)T
最优投影方向：W∗=S−1w(m1−m2)
判别函数
第一类：g12(x)>0 and g13(x)>0
第二类：g12(x)<0 and g23(x)>0
第三类：g13(x)<0 and g23(x)<0
拒识情况：其他
在本题中，以上分类器分别为：

g12(x) =(W12.T)*(X.T)-W012，W12=la.inv(sw12)*((u01-u02).T)W012=(l1*(W12.T)*(u01.T)+l2*(W12.T)*(u02.T))/(l1+l2)g13(x) =(W13.T)*(X.T)-W013，  W13=la.inv(sw13)*((u01-u03).T)W013=(l1*(W13.T)*(u01.T)+l3*(W13.T)*(u03.T))/(l1+l3)g23(x)= (W23.T)*(X.T)-W023，W23=la.inv(sw23)*(u02-u03).TW023=(l2*(W23.T)*(u02.T)+l3*(W23.T)*(u03.T))/(l3+l2)

代码

# coding=gbk#python edition: Python3.4.1,2014,10,17import numpy as npfrom numpy import linalg as ladef read_points():    dataset=[]    with open('Iris.txt','r') as file:        for line in file:            if line =='\n':                continue            dataset.append(list(map(float,line.split(' '))))        file.close()        return  datasetdef generate_traineddata():    arr=[[] for i in range(3)]    with open('setbase.txt','r') as file:        index=0        for line in file:            if line=='\n' :                continue            elif line[0]=='C':                index=int(line[-2])-1                continue            arr[index].append(int(line))        file.close()    train=[[] for i in range(3)]    test=[[] for i in range(3)]    for i in range(len(arr)):        tr=int(0.67* len(arr[i]))        train[i]=arr[i][:tr]        test[i]=arr[i][tr:]    f1=open('trained.txt','w')    f2=open('tested.txt','w')    print(train,end='\n')    print(test,end='\n')    for i in range(3):        for j in train[i]:            f1.write("%d\n"%j)        f1.write('\n')        for k in test[i]:            f2.write("%d\n"%k)        f2.write('\n')    f1.close()    f2.close()    return train,testdef createMatrix(train,test,dataset):    trainmat=[[] for i in range(3)]    testmat=[[] for i in range(3)]    for i in range(3):        for j in train[i]:            trainmat[i].append(dataset[j])        for k in test[i]:            testmat[i].append(dataset[k])    return   trainmat,testmatdef classify(trainmat,testmat,test):    #求三类训练集的均值向量    tr1,tr2,tr3=np.mat(trainmat[0]),np.mat(trainmat[1]),np.mat(trainmat[2])    te=[[] for i in range(3)]    te[0],te[1],te[2]=np.mat(testmat[0]),np.mat(testmat[1]),np.mat(testmat[2])    u01=np.mean(tr1,axis=0)    u02=np.mean(tr2,axis=0)    u03=np.mean(tr3,axis=0)    #获得矩阵长度    l1,l2,l3=len(trainmat[0]),len(trainmat[1]),len(trainmat[2])    #求三类训练集的类内离散度矩阵    s1,s2,s3=0,0,0    for i in range(l1):        s1=s1+(tr1[i]-u01).T*(tr1[i]-u01)    for i in range(l2):        s2=s2+ (tr2[i]-u02).T*(tr2[i]-u02)    for i in range(l3):        s3=s3+ (tr3[i]-u03).T*(tr3[i]-u03)    #总类内离散度矩阵    sw12,sw13,sw23=s1+s2,s1+s3, s2+s3    #求向量W*与边界    W12=la.inv(sw12)*((u01-u02).T)    W012=(l1*(W12.T)*(u01.T)+l2*(W12.T)*(u02.T))/(l1+l2)    W13=la.inv(sw13)*((u01-u03).T)    W013=(l1*(W13.T)*(u01.T)+l3*(W13.T)*(u03.T))/(l1+l3)    W23=la.inv(sw23)*(u02-u03).T    W023=(l2*(W23.T)*(u02.T)+l3*(W23.T)*(u03.T))/(l3+l2)    result=[[] for i in range(4)]    for i in range(3):        testset=te[i]        count=0        for X in testset:            if ((W12.T)*(X.T)-W012>0) and   ((W13.T)*(X.T)-W013>0):                result[0].append(test[i][count])            elif  ((W12.T)*(X.T)-W012<0) and   ((W23.T)*(X.T)-W023>0):                result[1].append(test[i][count])            elif    ((W13.T)*(X.T)-W013<0) and   ((W23.T)*(X.T)-W023<0):                result[2].append(test[i][count])            else:                result[3].append(test[i][count])            count=count+1    str1='类内离散度矩阵：'    str2='投影方向：'    str3='边界点：'    str4='Fisher得出的分类结果：'    fisher=open('fisher.txt','w')    print(str1,'s1:',end='\n',file=fisher)    print(s1,end='\n',file=fisher)    print(str1,'s2:',end='\n',file=fisher)    print(s2,end='\n',file=fisher)    print(str1,'s3:',end='\n',file=fisher)    print(s3,end='\n',file=fisher)    print(str2,'\n','W12:\n',W12,end='\n',file=fisher)    print('W13:\n',W13,end='\n',file=fisher)    print('W23:\n',W23,end='\n',file=fisher)    print(str3,'\n','W012:',W012,end='\n' ,file=fisher)    print('W013:',W013,end='\n' ,file=fisher)    print('W023:',W023,end='\n' ,file=fisher)    print(str4,end='\n' ,file=fisher)    for i in range(4):        print('第%d类'%(i+1),result[i],end='\n' ,file=fisher)    fisher.close()    return    resultdef main():    dataset=read_points()    train,test= generate_traineddata()    trainmat,testmat=createMatrix(train,test,dataset)    result= classify(trainmat,testmat,test)    print(result)if __name__=='__main__':    main()

测试集上的结果

实验中测得的参数：
类内离散度矩阵： s1:
[[ 4.23636364 3.10090909 0.50545455 0.50545455]
[ 3.10090909 4.10060606 0.02030303 0.47363636]
[ 0.50545455 0.02030303 1.01515152 0.16181818]
[ 0.50545455 0.47363636 0.16181818 0.34181818]]
类内离散度矩阵： s2:
[[ 6.12 0.948 5.132 0.046 ]
[ 0.948 2.5144 0.4916 0.6928]
[ 5.132 0.4916 7.0024 1.2592]
[ 0.046 0.6928 1.2592 1.6136]]
类内离散度矩阵： s3:
[[ 9.60097561 3.05414634 5.14512195 1.73707317]
[ 3.05414634 4.27512195 2.67926829 1.69756098]
[ 5.14512195 2.67926829 7.34439024 2.53463415]
[ 1.73707317 1.69756098 2.53463415 1.56878049]]
投影方向：
W12: [[ 0.00467303], [ 0.21663412], [-0.43497883], [-0.71989691]]T
W13: [[-0.01511481], [ 0.3200074 ], [-0.25327497], [-0.55700484]]T
W23: [[ 0.02024091], [-0.05645396], [ 0.0580067 ], [ 0.17693648]]T
边界点：
W012: [[-1.44922042]], W013: [[-0.34557374]], W023: [[ 0.53145747]]
分类结果（无拒识情况，错误的为斜黑体）
第1类 [33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]
第2类 [132, 134, 135, 136, 137, 139, 140, 141, 143, 144, 145, 147, 148, 101, 113, 114, 119, 121, 123, 126, 127, 138, 142, 146, 149]
本身为第三类，却分到了第二类
第3类 [93, 94, 95, 96, 97, 98, 99, 106, 133]
正确率
39/51*100%=76.47%

0 0