imageclassification 第二节 KNN

一.算法思想

KNN算法又称之为k近邻算法。此算法思路非常简单，就图像分类器来说，就是通过比较输入图像与训练图像数据的距离来判断输入图像的类别，其大致思路如下：

 1.计算输入图像与训练样本图像的欧式距离

 2.对上述距离进行排序

 3.选择距离最少的前k个

 4.对k中类别进行投票，得票最高的那个类别，就是分类器预测的类别

二.实践

1.数据的转换

图像的分类问题，首要的任务就是要将图像转化成向量（矩阵）。代码如下

def img2vector(filename):rows=32   #图像的行大小，根据图像的大小设置cols=32<span style="white-space: pre;">  #图像的列大小，数值根据实际来设置imgVector=zeros((1,rows*cols))    #创建的1行raw*cols列大小的零矩阵fileIn=open(filename)          #打开图像for row in xrange(rows):lineStr=fileIn.readline()    #以字符串形式读取一行数据for col in xrange(cols):imgVector[0,row*32+col]=int(lineStr[col])  #按列赋值  int()表示强制转换return imgVector

2.加载训练数据和测试数据

测试数据和训练数据下载地址：在这 。这个数据库包括数字0-9的手写体。每个数字大约有200个样本。每个样本保持在一个txt文件中。手写体图像本身的大小是32x32的二值图，转换到txt文件保存后，内容也是32x32个数字，0或者1，如下：

0000000000000111100000000000000000000000000011111110000000000000000000000011111111110000000000000000000111111111111110000000000000000001111111011111100000000000000000111111100000111100000000000000001111111000000011100000000000000011111110000000111100000000000000111111100000000111000000000000001111111000000001110000000000000011111100000000011110000000000000111111000000000011100000000000001111110000000000111000000000000001111110000000000111000000000000011111100000000001110000000000000111111000000000011100000000000001111110000000000111000000000000111111100000000011110000000000001111011000000000111100000000000011110000000000011110000000000000011110000000000011110000000000000111100000000001111100000000000001111000000000111110000000000000011110000000011111000000000000000011100000011111100000000000000000111100011111110000000000000000001111111111111100000000000000000001111111111111000000000000000000011111111111100000000000000000000011111111100000000000000000000000011111000000000000000000000000000011000000000000000000

代码如下：

def loadDataSet():print '-----Getting training set------'dataSetDir='D:/Downloads/digits/'   #digits文件夹路径trainingFileList=os.listdir(dataSetDir+'trainingDigits') #训练数据文件列表numSamples=len(trainingFileList)     #训练样例数目train_x=zeros((numSamples,1024))      #建立相应大小的矩阵train_y=[]                      #标签for i in xrange(numSamples):filename=trainingFileList[i]     #获取文件名 如0_1.txttrain_x[i,:]=img2vector(dataSetDir+'trainingDigits/%s' %filename) #将图像转化成矩阵label=int(filename.split('_')[0])   #以'_'为分割号，对文件名进行分割,产生['0','1.txt'],取[0]train_y.append(label)     #加入标签print '---------Getting testing set----'      #测试数据 也如上testingFileList=os.listdir(dataSetDir+'testDigits')numSamples=len(testingFileList)test_x=zeros((numSamples,1024))test_y=[]for i in xrange(numSamples):filename=testingFileList[i]test_x[i,:]=img2vector(dataSetDir+'testDigits/%s' %filename)label=int(filename.split('_')[0])test_y.append(label)return train_x,train_y,test_x,test_y

3.knn分类

def kNNClassify(newInput,dataSet,labels,k):numSamples=dataSet.shape[0]  #获取训数据数目diff=tile(newInput,(numSamples,1))-dataSet   #求差值squaredDiff=diff**2        #平方squaredDist=sum(squaredDiff,axis=1)  #求和distance=squaredDist**0.5     #求根号sortedDistIndices=argsort(distance)  #排序classCount={}for i in xrange(k):                     #投票voteLabel=labels[sortedDistIndices[i]]     classCount[voteLabel]=classCount.get(voteLabel,0)+1maxCount=0for key,value in classCount.items():      #求最值if value>maxCount: maxCount=valuemaxIndex=keyreturn maxIndex

1.其中tile函数，只要功能是对矩阵进行“复制”。

若a=[1,2,3],则 

》tile(a,(3,1))

[[1,2,3],

 [1,2,3]

 [1,2,3]]

2.对词典的操作理解

classCount[voteLabel]=classCount.get(voteLabel,0)+1

若votelabel存在，则get得到的是votelabel的对应value。若不存在，则得到0.

全部代码如下：

from numpy import *import operatorimport osdef kNNClassify(newInput,dataSet,labels,k):numSamples=dataSet.shape[0]diff=tile(newInput,(numSamples,1))-dataSetsquaredDiff=diff**2squaredDist=sum(squaredDiff,axis=1)distance=squaredDist**0.5sortedDistIndices=argsort(distance)classCount={}for i in xrange(k):voteLabel=labels[sortedDistIndices[i]]classCount[voteLabel]=classCount.get(voteLabel,0)+1maxCount=0for key,value in classCount.items():if value>maxCount:maxCount=valuemaxIndex=keyreturn maxIndexdef img2vector(filename):rows=32cols=32imgVector=zeros((1,rows*cols))fileIn=open(filename)for row in xrange(rows):lineStr=fileIn.readline()for col in xrange(cols):imgVector[0,row*32+col]=int(lineStr[col])return imgVectordef loadDataSet():print '-----Getting training set------'dataSetDir='D:/Downloads/digits/'trainingFileList=os.listdir(dataSetDir+'trainingDigits')numSamples=len(trainingFileList)train_x=zeros((numSamples,1024))train_y=[]for i in xrange(numSamples):filename=trainingFileList[i]train_x[i,:]=img2vector(dataSetDir+'trainingDigits/%s' %filename)label=int(filename.split('_')[0])train_y.append(label)print '---------Getting testing set----'testingFileList=os.listdir(dataSetDir+'testDigits')numSamples=len(testingFileList)test_x=zeros((numSamples,1024))test_y=[]for i in xrange(numSamples):filename=testingFileList[i]test_x[i,:]=img2vector(dataSetDir+'testDigits/%s' %filename)label=int(filename.split('_')[0])test_y.append(label)return train_x,train_y,test_x,test_ydef testhandWritingClass():print 'step 1: load data...'train_x,train_y,test_x,test_y=loadDataSet()print 'step 2: training...'passprint 'step 3: testing...'matchCount=0numTestSamples=test_x.shape[0]print 'step 4: show the result...'for i in xrange(numTestSamples):predict=kNNClassify(test_x[i],train_x,train_y,1)print 'Your input is: %d and classified to class: %d'%(test_y[i],predict)if predict==test_y[i]:matchCount+=1accuracy=float(matchCount)/numTestSamplesprint 'The classify accuracy is: %.2f%%' %(accuracy*100)if __name__=='__main__':testhandWritingClass()

四.结果

step 1: load data...-----Getting training set---------------Getting testing set----step 2: training...step 3: testing...step 4: show the result...Your input is: 0 and classified to class: 0Your input is: 0 and classified to class: 0Your input is: 0 and classified to class: 0...............Your input is: 9 and classified to class: 9Your input is: 9 and classified to class: 9Your input is: 9 and classified to class: 9Your input is: 9 and classified to class: 9Your input is: 9 and classified to class: 9Your input is: 9 and classified to class: 9Your input is: 9 and classified to class: 9The classify accuracy is: 98.63%

五 .补充

对于这个程序，若我想使用自己手写的图片来分类，而不是上面那种“图片”，又该如何？如下图

这里可以参考上述数据库形式，将图片转化成数字，及把图中黑色部分灰度复制为0，白色为1，就可以了。

测试函数如下：

def testhandWritingClass2():from PIL import Imagefrom scipy.misc import imread, imresizeim=imread('0_1.bmp')im1=imresize(im,(32,32))print im1.shapeimgVector=zeros((1,1024))for i in range(32):for j in range(32):if im1[i,j]>0:imgVector[0,32*i+j]=1label=0print 'step 1: load data...'train_x,train_y,test_x,test_y=loadDataSet()print 'step 2: training...'passprint 'step 3: testing...'matchCount=0print 'step 4: show the result...'predict=kNNClassify(imgVector,train_x,train_y,3)print 'Your input is: %d and classified to class: %d'%(label,predict)

结果：

step 1: load data...-----Getting training set---------------Getting testing set----step 2: training...step 3: testing...step 4: show the result...Your input is: 0 and classified to class: 0[Finished in 2.5s]

参考

1.http://blog.csdn.net/zouxy09/article/details/16955347

2.机器学习实战