为什么高斯核函数映射到无穷维度？

来源：互联网发布：苹果电脑装windows 编辑：程序博客网时间：2024/04/27 21:16

Consider the polynomial kernel of degree 2 defined by, $k(x, y) = (x^Ty)^2$ where $x, y \in \mathbb{R}^2$ and $x = (x_1, x_2), y = (y_1, y_2)$ .

Thereby, the kernel function can be written as,
$k(x, y) = (x_1y_1 + x_2y_2)^2$
$= x_{1}^2y_{1}^2 + 2x_1x_2y_1y_2 + x_{2}^2y_{2}^2$ .
Now, let us try to come up with a feature map $\Phi$ such that the kernel function can be written as $k(x, y) = \Phi(x)^T\Phi(y)$ .

Consider the following feature map, $\Phi(x) = (x_1^2, \sqrt{2}x_1x_2, x_2^2)$ . Basically, this feature map is mapping the points in $\mathbb{R}^2$ to points in $\mathbb{R}^3$ . Also, notice that, $\Phi(x)^T\Phi(y) = x_1^2y_1^2 + 2x_1x_2y_1y_2 + x_2^2y_2^2$ which is essentially our kernel function.

This means that our kernel function is actually computing the inner/dot product of points in $\mathbb{R}^3$ . That is, it is implicitly mapping our points from $\mathbb{R}^2$ to $\mathbb{R}^3$ .

NOTE：我们要做的是将线性不可分的sample映射到高维空间（在这个空间中线性可分），然后在这个高维空间中衡量两个sample之间的相似性，即做点积。在上述的多项式核函数例子中，阐述了通过核函数同样可以达到我们的目的，而且省略了显示映射的环节。

Now, coming to RBF.

Let us consider the RBF kernel again for points in $\mathbb{R}^2$ . Then, the kernel can be written as
$k(x, y) = \exp(-\|x - y\|^2)$
$= \exp(- (x_1 - y_1)^2 - (x_2 - y_2)^2)$
$= \exp(- x_1^2 + 2x_1y_1 - y_1^2 - x_2^2 + 2x_2y_2 - y_2^2)$
$= \exp(-\|x\|^2) \exp(-\|y\|^2) \exp(2x^Ty)$
(assuming gamma = 1). Using the taylor series you can write this as,
$k(x, y) = \exp(-\|x\|^2) \exp(-\|y\|^2) \sum_{n = 0}^{\infty} \frac{(2x^Ty)^n}{n!}$

Now, if we were to come up with a feature map $\Phi$ just like we did for the polynomial kernel, you would realize that the feature map would map every point in our $\mathbb{R}^2$ to an infinite vector. Thus, RBF implicitly maps every point to an infinite dimensional space.

NOTE：上面是RBF核函数（也可以理解为高斯核函数），在上面的转换过程中核函数最终通过傅里叶展开为无穷项，而每项都是一个多项式核函数，从这里可以看出高斯核函数映射之后为什么是无穷维的。

另外需要注意一点就是，无穷维的向量并不意味着对两个无穷维的向量相似性的度量。点积是衡量两个无穷维向量相似值最终收敛到的值！

原文链接：http://www.quora.com/Why-does-the-RBF-radial-basis-function-kernel-map-into-infinite-dimensional-space

1 0