感知器算法

感知器算法是一种用于二进制分类的监督学习算法，可以预测数字向量所表示的输入是否属于特定的类。
在机器学习的术语中，分类被认为是监督学习的实例，即，其中可观测得到正确识别的训练集，可将之用于训练学习。 相应的无监督过程被称为聚类或聚类分析，并且涉及基于固有相似性（例如，被视为多维向量空间中的向量的实例之间的距离）的某种度量将数据分组到类别中。 （维基百科）
在人工神经网络领域中，感知机也被指为单层的人工神经网络，以区别于较复杂的多层感知机（Multilayer Perceptron）。作为一种线性分类器，（单层）感知机可说是最简单的前向人工神经网络形式。尽管结构简单，感知机能够学习并解决相当复杂的问题。感知机主要的本质缺陷是它不能处理线性不可分问题。（ibid）
感知器学习算法是在1950年代后期由Frank Rosenblatt提出的。感知器算法是一种用于二进制分类的监督学习算法，可以预测数字向量所表示的输入是否属于特定的类（其将种类假定为标记+1和-1），分类器（classfier）试图通过线性分离器（sepereator）来划分这两个类。

在感知器模型中，Figure.1有三个输入，X1,X2,X3,X4,X5可以使用这些变量，并对应于二进制分类。 例如，如果匹配训练集中的正集，则否则，在这种情况下，权重允许您使用感知器来设置用于决策的数学模型，并且其指示相应输入对于输出的重要性。 例如，如果你去旅行，但是你讨厌恶劣的天气，这个时候假设决定是X1，权重分配给6，如果阈值是5，那么X1天气是最重要的因素。 其他因素对决策没有影响。 所以可以假设好天气输出1，恶劣天气输出0（Nielsen, M. A. 2015)。

Figure. 1 model of perceptron（Bollegala, D，2017）
在该神经网络模型中a是该神经网络模型中a是activation，a = w1x1+w2x2+w3x3+w4x4+w5x5。如果a大于预定阈值θ，则神经元激发。产生输出为1或0。可得如下：
if a > θ then
output = 1
else
output = 0
其中w和x对应于权重和输入向量。 第二个变化是将阈值移动到不等式的另一侧，并将其替换为感知器的偏差b≡-threshold。 使用偏差而不是阈值，感知器的规则可以重写为以下公式：

通过引入始终为ON的特征（即，x0 = 1对于所有实例），我们可以挤压偏差通过设置w0 = b将项b代入权重向量

在这里我们可以将激活写为权重向量和特征向量之间的内积。 然而，我们应该记住，偏差项仍然出现在模型中。
以下是该算法的伪代码：

以下是python的实现代码：

import timeimport randomimport numpydef process(userChoice):    bias = 0    train_errors = []    test_errors = []    # Read input of iterations wanted    maxIter = int(raw_input("\nHow many iterations would you like to run? "))    # Get all features from files and determine the number of features    featspace = get_feat_space("/Users/liuminghao/Desktop/Dataminding/data/train.positive")    featspace = featspace.union(get_feat_space("/Users/liuminghao/Desktop/Dataminding/data/train.negative"))    featspace = featspace.union(get_feat_space("/Users/liuminghao/Desktop/Dataminding/data/test.positive"))    featspace = featspace.union(get_feat_space("/Users/liuminghao/Desktop/Dataminding/data/test.negative"))    D = len(featspace)    print "Dimensionality of the feature space: %i\n" % D    # Convert feature set to a list    featspace = list(featspace)    ''' Create a feature index that stores each unique feature as {Key: Value}        where 'Key' is a feature e.g. 'atmosphere' and 'Val' is an id (0,1,...,D)        Start the feat_index at 1 since bias will be 0    '''    feat_index, weights = {}, {}    for (feat_id, feat_val) in enumerate(featspace):        feat_index[feat_val] = feat_id        weights[feat_val] = 0    # Generate the train and test data    train_data = get_feat_vects("./data/train.positive", D, 1)    train_data.extend(get_feat_vects("./data/train.negative", D, -1))    test_data = get_feat_vects("./data/test.positive", D, 1)    test_data.extend(get_feat_vects("./data/test.negative", D, -1))    ''' Run different train/test methods depending on users choice.        Choice 1 enters the if block and runs all training iterations before        running a single test iteration.        Choice 2 enters the else if block and runs a test iteration after every        training iteration using the results to plot a graph of error rate.    '''    if userChoice == 1:        startTime = time.clock()        print "--------------------TRAINING-------------------"        for i in range(maxIter):            # Shuffle the 'train_data' and train the perceptron            random.shuffle(train_data)            weights, bias, train_errors = percept_train(i, D, train_data, weights, bias, train_errors)        print "Overall Time taken: %.2f second(s)" % (time.clock() - startTime)        print "--------------------TESTING--------------------"        # Test the perceptron using the 'test_data'        test_errors = percept_test(D, test_data, weights, bias, test_errors)    elif userChoice == 2:        for i in range(maxIter):            print "- RUN %i - - - - - - - - - - - - - - - - - - - -" % (i+1)            # Shuffle the 'train_data' and train the perceptron            random.shuffle(train_data)            weights, bias, train_errors = percept_train(i, D, train_data, weights, bias, train_errors)            # Shuffle the 'test_data' and test the perceptron            random.shuffle(test_data)            test_errors = percept_test(D, test_data, weights, bias, test_errors)        # Plot a graph of error rate vs number of iterations        plot_graph(train_errors, test_errors, maxIter)# Method to determine the feature space from some input filesdef get_feat_space(fname):    feats = set()    with open(fname) as feat_file:        for line in feat_file:            for w in line.strip().split():                feats.add(w)    return feats# Method to determine a feature vector for each input filedef get_feat_vects(fname, D, label):    feat_vects = []    with open(fname) as feat_file:        for line in feat_file:            feats = {}            for w in line.strip().split():                feats[w] = 1            feat_vects.append((feats,label))    return feat_vects# Method to train the perceptrondef percept_train(i, D, train_data, weights, bias, train_errors):    startTime = time.clock()    numErrors = 0    for review in train_data:        feat_set, expected = review[0], review[1]        activation = sum(weights[feat]*feat_set[feat] for feat in feat_set) # calculate the activation        activation += bias # adding the bias        # If output was incorrect, update the weights        if ((expected*activation) <= 0):            numErrors += 1            for feat in feat_set:                weights[feat] = weights[feat] + expected*feat_set[feat]            bias += expected # updating the bias    # Add error data to an array to be used later    train_error = (float(numErrors)/float(len(train_data)))*100    train_errors.append(train_error)    print "Training iteration %i. Time taken: %.2f second(s)" % ((i+1), (time.clock() - startTime))    return (weights, bias, train_errors)# Method to test the perceptrondef percept_test(D, test_data, weights, bias, test_errors):    totalCorrect = 0    numErrors = 0    for review in test_data:        feat_set, expected = review[0], review[1]        activation = sum(weights[feat]*feat_set[feat] for feat in feat_set) # calculate the activation        activation += bias # adding the bias        # Store the number of correct vs incorrect outputs        if (numpy.sign(activation) == expected):            totalCorrect += 1        else:            numErrors += 1    # Add error data to an array to be used later    test_error = (float(numErrors)/float(len(test_data)))*100    test_errors.append(test_error)    correctPercent = (float(totalCorrect)/len(test_data))*100    print "Correctly classified instances: %i/%i (%.1f%%)" % (totalCorrect, len(test_data), correctPercent)    print "Error rate in classification: %.1f%%" % (100-correctPercent)    print "-----------------------------------------------\n"    return test_errors# Method to plot a graph of error rate vs number of iterationsdef plot_graph(train_errors, test_errors, maxIter):    from matplotlib import pyplot as plt    plt.plot(range(1, maxIter+1), train_errors, '-b', label='Training Errors')    plt.plot(range(1, maxIter+1), test_errors, '-r', label='Testing Errors')    plt.axis([1,maxIter,0,(max(max(train_errors), max(test_errors))+5)])    plt.xlabel('Iteration Number')    plt.ylabel('Error Rate %')    plt.legend(loc='upper right')    plt.show(block=True)#Executeif __name__ == "__main__":    print "\nPlease choose one of the following options:"    print "1. Repeat training multiple times and then perform single test"    print "2. Test after each iteration of training and output plot of results (requires matplotlib)"    userChoice = int(raw_input())    process(userChoice) # execute the main method