机器学习2 梯度下降

来源：互联网发布：男生衣服品牌知乎编辑：程序博客网时间：2024/06/05 07:02

θ i : = θ i - α \partial \partial θ i J (θ) \partial \partial θ i J (θ) = \partial \partial θ i 1 2 (h θ (x) - y) 2 = (h θ (x) - y) \partial \partial θ i (θ 0 x 0 + . . . + θ i x i + θ n x n - y) = (h θ (x) - y) x i θ i : = θ i - α (h θ (x) - y) x i

θ i : = θ i - α \sum j = 1 n (h θ (x (j)) - y (j)) x i (j)

repeat j = 1 to m {
for all i {

θ i : = θ i - α (h θ (x (j)) - y (j)) x i (j)

}
}

\nabla A f (A) = ⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ \partial f \partial A 11 . . . \partial f \partial A m 1 . . . . . . . . . \partial f \partial A 1 n . . . \partial f \partial A m n ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥

t r A = \sum i = 1 n A i i \Rightarrow t r A B = t r B A t r A = t r A T t r (A + B) = t r A + t r B t r α A = α t r A

\nabla A t r A B = B T \nabla A T f (A) = (\nabla A f (A)) T \nabla A t r A B A T C = C A B + C T A B T \nabla A | A | = | A | (A - 1) T

X = ⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ (x (1)) T (x (2)) T . . . (x (m)) T ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ y \to = ⎡ ⎣ ⎢ ⎢ ⎢ ⎢ y (1) y (2) . . . y (m) ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ X θ = ⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ (x (1)) T (x (2)) T . . . (x (m)) T ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ θ = ⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ (x (1)) T θ (x (2)) T θ . . . (x (m)) T θ ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ = = ⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ h θ (x (1)) h θ (x (2)) . . . h θ (x (m)) ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ J (θ) = 1 2 \sum i = 1 m (h θ (x (i)) - y (i))) 2 = 1 2 (X θ - y \to) T (X θ - y \to) \Rightarrow \nabla θ J (θ) = \nabla θ 1 2 (X θ - y \to) T (X θ - y \to) = 1 2 \nabla θ t r (θ T X T X θ - θ T X T y \to - y \to T X θ + y \to T y \to) = 1 2 \nabla θ (t r θ T X T X θ - 2 t r y \to T X θ) = 1 2 \nabla θ (t r θ T X T X θ - 2 t r y \to T X θ) = 1 2 \nabla θ (t r θ θ T X T X - 2 t r θ y \to T X) = 1 2 (X T X θ + X T X θ - 2 X T y \to) = X T X θ - X T y \to \Rightarrow θ = (X T X) - 1 X T y \to

1 0