Introduction to One-class Support Vector Machines
来源:互联网 发布:单片机的堆栈 编辑:程序博客网 时间:2024/05/04 06:12
Traditionally, many classification problems try to solve the two or multi-class situation. The goal of the machine learning application is to distinguish test data between a number of classes, using training data. But what if you only have data of one class and the goal is to test new data and found out whether it is alike or not like the training data? A method for this task, which gained much popularity the last two decades, is the One-Class Support Vector Machine. This (quite lengthly) blog post will give an introduction to this technique and will show the two main approaches.
Just one class?
First look at our problem situation; we would like to determine whether (new) test data is member of a specific class, determined by our training data, or is not. Why would we want this? Imagine a factory type of setting; heavy machinery under constant surveillance of some advanced system. The task of the controlling system is to determine when something goes wrong; the products are below quality, the machine produces strange vibrations or something like a temperature that rises. It is relatively easy to gather training data of situations that are OK; it is just the normal production situation. But on the other side, collection example data of a faulty system state can be rather expensive, or just impossible. If a faulty system state could be simulated, there is no way to guarantee that all the faulty states are simulated and thus recognized in a traditional two-class problem.
To cope with this problem, one-class classification problems (and solutions) are introduced. By just providing the normal training data, an algorithm creates a (representational) model of this data. If newly encountered data is too different, according to some measurement, from this model, it is labeled as out-of-class. We will look in the application of Support Vector Machines to this one-class problem.
Basic concepts of Support Vector Machines
Let us first take a look at the traditional two-class support vector machine. Consider a data set
A very nice property of SVMs is that it can create a non-linear decision boundary by projecting the data through a non-linear function
The hyperplane is represented with the equation
When this minimization problem (with quadratic programming) is solved using Lagrange multipliers, it gets really interesting. The decision function (classification) rule for a data point
Here
Kernel Function
The function
Popular choices for the kernel function are linear, polynomial, sigmoidal but mostly the Gaussian Radial Base Function:
With this set of formulas and concepts we are able to classify a set of data point into two classes with a non-linear decision function. But, we are interested in the case of a single class of data. Roughly there are two different approaches, which we will discuss in the next two sections.
One-Class SVM according to Schölkopf
The Support Vector Method For Novelty Detection by Schölkopf et al. basically separates all the data points from the origin (in feature space
The quadratic programming minimization function is slightly different from the original stated above, but the similarity is still clear:
In the previous formulation the parameter
- it sets an upper bound on the fraction of outliers (training examples regarded out-of-class) and,
- it is a lower bound on the number of training examples used as Support Vector.
Due to the importance of this parameter, this approach is often referred to as
Again by using Lagrange techniques and using a kernel function for the dot-product calculations, the decision function becomes:
This method thus creates a hyperplane characterized by
One-Class SVM according to Tax and Duin
The method of Support Vector Data Description by Tax and Duin (SVDD) takes a spherical, instead of planar, approach. The algorithm obtains a spherical boundary, in feature space, around the data. The volume of this hypersphere is minimized, to minimize the effect of incorporating outliers in the solution.
The resulting hypersphere is characterized by a center
After solving this by introduction Lagrange multipliers
You can see the similarity with the traditional two-class method, the algorithm by Schölkopf and Tax and Duin. So far the theoretical fundamentals of Support Vector Machines. Lets take a very quick look to some applications of this method.
Applications (in Matlab)
A very good and much used library for SVM-classification is LibSVM, which can be used for Matlab. Out of the box it supports one-class SVM following the method of Schölkopf. Also available in the LibSVM tools is the method for SVDD, following the algorithm of Tax and Duin.
To give a nice visual clearification of how the kernel mapping (to feature space
Application to change detection
As a conclusion to this post I will give a look at the perspective from which I am using one-class SVMs in my current research for my master thesis (which is performed at the Dutch research company Dobots). My goal is to detect change points in a time series data; also known as novelty detection. One-class SVMs have already been applied to novelty detection for time series data. I will apply it specifically to accelerometer data, collection by smartphone sensors. My theory is that when the change points in the time series are explicitly discovered, representing changes in the activity performed by the user, the classification algorithms should perform better. Probably in a next post I will take a further look at an algorithm for novelty detection using one-class Support Vector Machines.
Update: GitHub repository
Currently I am using the SVDD method by Tax and Duin to implement change detection and temporal segmentation for accelerometer data. I am using the Matlab dd_tools package, created by Tax, for the incremental version of SVDD. You can use my implementation and fork it from the oc_svm github repository. Most of the functions are documented, but it is under heavy development and thus the precise workings differ from time to time. I am planning to write a good readme, but if you are interested I advise you to look at the apply_inc_svdd.m
file, which creates the SVM classifier and extracts properties from the constructed model.
- Introduction to One-class Support Vector Machines
- Introduction to Support Vector Machines
- Support Vector Machines
- Support Vector Machines (SVM)
- Support Vector Machines 1
- Support Vector Machines
- Support Vector Machines(SVM)
- 04 Support Vector Machines
- Support vector Machines(SVM)
- Unsupervised and Semi-supervised Multi-class Support Vector Machines
- Stanford ex6:Support Vector Machines
- Introduction to Restricted Boltzmann Machines
- Introduction to Restricted Boltzmann Machines
- One Class SVM, SVDD(Support Vector Domain Description)
- One-class Classification(introduction)
- Is it OK to mix categorical and continuous data for SVM (Support Vector Machines)?
- Support Vector Machines: Theory and Applications
- Support Vector Machines vs Artificial Neural Networks
- 3.2.1信号
- 项目从intellij idea迁移到eclipse中问题总结
- FastCGI进程意外退出,http错误404.3-NotFound 由于扩展配置问题 dll找不到找定的模块等 dz问题总结
- 【TensorFlow】tf.nn.local_response_normalization详解,lrn正则法如何计算?
- Android中关于enum的随笔
- Introduction to One-class Support Vector Machines
- nodejs实现restful API
- C#-WinForm登录窗体实现记住密码的功能(仿QQ实现)
- (Uva)Buy or Build(B)
- ExecutorService生命周期,,shutdownNow方法:这个方法会强制关闭ExecutorService,它将取消所有
- 朱松纯:初探计算机视觉三个源头兼谈人工智能
- jquery checkbox checked 第一次有效果 第二次以后就不好用了
- 从零开始搭建架构实施Android项目
- 控制DateTimePicker日期或时间的显示格式