机器学习PLA&Pocket algorithm实现（python）

来源：互联网发布：数控加工编程代码例题编辑：程序博客网时间：2024/06/07 07:01

1. PLA&Pocket algorithm

在机器学习中，感知机（Perceptron）是二分类的线性分类模型，属于监督学习算法。对于线性可分数据集可以直接使用PLA进行分类，且最终一定可以找到一个最优分类超平面将数据完全分开；有关感知机收敛性证明可以参考李航老师的《统计学习方法》或者这篇博客，感知机的python实现可以参考这里。下两幅图来自台大林轩田老师的机器学习基石课程，分别为PLA算法和PLA算法的实施步骤。

本篇博客主要研究利用pocket algorithm对线性不可分数据集进行简单分类：对于线性不可分数据集，PLA算法将会一直运行下去（在不设置迭代次数等其他停止条件下），这时我们可以使用Pocket algorithm.（一种改进的PLA 算法），算法流程如下：

2. python实现

2.1 数据集

数据集：http://pan.baidu.com/s/1bVDV3K

2.2 代码实现

【注】以下代码可以实现PLA和Pocket algprithm

#!/usr/bin/python3# -*- coding: utf-8 -*-# @Time    : 2017/11/18 20:47# @Author  : Z.C.Wang# @Email   : iwangzhengchao@gmail.com# @File    : pocket_algorithm.py# @Software: PyCharm Community Edition"""Description :    pocket algorithm"""import timeimport numpy as npimport matplotlib.pyplot as plt# load data from DataSet.txtdata_set = []data_label = []file = open('DataSet.txt')# file = open('DataSet_linear_separable.txt')for line in file:    line = line.split('\t')    for i in range(len(line)):        line[i] = float(line[i])    data_set.append(line[0:2])    data_label.append(int(line[-1]))file.close()data = np.array(data_set)for i in range(len(data_label)):    if data_label[i] != 1:        data_label[i] = -1label = np.array(data_label)# Initialize w, b, alphaw = np.array([0.5, 1])b = 0alpha = 0.4trainLoss = []# Calculate train_lossf = (np.dot(data, w.T) + b) * labelidx = np.where(f <= 0)train_loss = -np.sum((np.dot(data[idx], w.T) + b) * label[idx]) / (np.sqrt(w[0]**2+w[1]**2))trainLoss.append(train_loss)# iterationmax_iter = 500iteration = 1start = time.time()while iteration <= max_iter:    if f[idx].size == 0:        break    for sample in data[idx]:        i = 0        w += alpha * sample * label[idx[0][i]]        b += alpha * label[idx[0][i]]        i += 1    print('Iteration:%d  w:%s  b:%s' % (iteration, w, b))    f = (np.dot(data, w.T) + b) * label    idx = np.where(f <= 0)    train_loss = -np.sum((np.dot(data[idx], w.T) + b) * label[idx]) / (np.sqrt(w[0] ** 2 + w[1] ** 2))    trainLoss.append(train_loss)    iteration = iteration + 1if f[idx].size == 0:    accuracy = 100else:    accuracy = len(f[idx]) / len(label) * 100end = time.time()print('Pocket learning algorithm is over')print('train time is %f s.' % (end - start))print('-'*50)print('min trainLoss: %f' % np.min(trainLoss))print('Classification accuracy: %.2f%%' % accuracy)# drawx1 = np.arange(-4, 5, 0.1)x2 = (w[0] * x1 + b) / (-w[1])idx_p = np.where(label == 1)idx_n = np.where(label != 1)data_p = data[idx_p]data_n = data[idx_n]plt.figure()plt.scatter(data_p[:, 0], data_p[:, 1], color='b')plt.scatter(data_n[:, 0], data_n[:, 1], color='r')plt.plot(x1, x2)plt.show()plt.figure()plt.plot(trainLoss)plt.ylabel('trainLoss')plt.xlabel('Iteration')plt.show()

2.3 运行结果

Iteration:498  w:[ 79.2537332 -64.189894 ]  b:508.4Iteration:499  w:[ 79.0104068 -80.1653084]  b:506.4Iteration:500  w:[ 78.4516652 -68.5393232]  b:508.0Pocket learning algorithm is overtrain time is 0.177006 s.--------------------------------------------------min trainLoss: 2.323198Classification accuracy: 4.00%

阅读全文

0 0