Loss Function 整理

来源：互联网发布：网吧服务器优化编辑：程序博客网时间：2024/05/18 00:11

usepackage{txfonts}

机器学习常用Loss Function 整理

J (θ) = 1 2 m \sum i = 1 m (h θ (x (i)) - y (i)) 2

J (θ) = 1 2 m \sum i = 1 m | h θ (x (i)) - y (i) |

J (θ) = - 1 m [\sum i = 1 m (y (i) l o g h θ (x (i)) + (1 - y (i)) l o g (1 - h θ (x (i))))]

J (θ) = - 1 m ⎡ ⎣ \sum i = 1 m \sum j = 1 k 1 {y (i) = j} l o g e θ T j x ( i ) \sum k l = 1 e θ T l x ( i ) ⎤ ⎦ = - 1 m ⎡ ⎣ \sum i = 1 m \sum j = 1 k 1 {y (i) = j} log p (y (i) = j | x (i); θ) ⎤ ⎦

如果将下式中的∑kj=1替换成∑1j=0，即可得到 log loss

J (w ⃗) = [1 m \sum i = 1 m m a x (0, 1 - y (i) (w ⃗ \cdot x (i) \to + b))] + λ ∥ w ⃗ ∥ 2

或者可以写成：

J (w ⃗) = 1 m \sum i = 1 m ζ (i) + λ ∥ w ⃗ ∥ 2 (1) s u b j e c t t o y (i) ((w ⃗ \cdot x (i) \to + b) \geq 1 - ζ (i) a n d ζ (i) \geq 0, f o r a l l i

J (w) = ⎡ ⎣ 1 m \sum i = 1 m \sum j \neq y (i) m a x (0, 1 + w T j \cdot x (i) \to - w T y (i) \cdot x (i) \to) ⎤ ⎦ + λ ∥ w ∥ 2

或者可以写成：

J (w) = 1 m \sum i = 1 m ζ (i) + λ ∥ w ∥ 2 (2) s u b j e c t t o (i) ((w T y (i) \cdot x (i) \to - w T j \cdot x (i) \to) \geq e j i - ζ (i) i = 1, . . ., m, j = 1, . . ., k, y (i) \in {1, . . ., k}

where

e j i = {01 if y (i) = j if y (i) \neq j

将式(1), (2)改写成

J (w ⃗) = 1 m \sum i = 1 m ζ (i) 2 + λ ∥ w ⃗ ∥ 2 s u b j e c t t o y (i) ((w ⃗ \cdot x (i) \to + b) \geq 1 - ζ (i) a n d ζ (i) \geq 0, f o r a l l i

以及

J (w) = 1 m \sum i = 1 m ζ (i) 2 + λ ∥ w ∥ 2 s u b j e c t t o (i) ((w T y (i) \cdot x (i) \to - w T j \cdot x (i) \to) \geq e j i - ζ (i) i = 1, . . ., m, j = 1, . . ., k, y (i) \in {1, . . ., k}

J (θ) = \sum i = 1 m e - y (i) h θ (x (i)) + λ ∥ θ ∥

离散:

K L (p ∥ q) = \sum x p (x) l o g p ( x ) ( q ( x ) )

连续:

K L (p ∥ q) = \int p (x) l o g p ( x ) ( q ( x ) ) d x

阅读全文

0 0