Machine Learning Notes - PLA

来源：互联网发布：怎样查询电信网络密码编辑：程序博客网时间：2024/05/16 17:34

$h(\overrightarrow{w};\overrightarrow{x})=sign(\overrightarrow{w}^T\cdot \overrightarrow{x}).$

$\overrightarrow{w}=\{w_d,w_{d-1},...,w_1,w_0\},\overrightarrow{x}=\{x_d,x_{d-1},...,x_1,1\}.$

$\text{For a mistake}: (\overrightarrow{x_{n(t)}},y_{n(t)}),(\text{here}, y_{n(t)}=\pm 1).$

$\text{Namely},h(\overrightarrow{w_t};\overrightarrow{x_{n(t)}})=sign(\overrightarrow{w_t}^T\cdot \overrightarrow{x_{n(t)}})\neq y_{n(t)}.$

$\text{Correct the mistake by}: \overrightarrow{w_{t+1}}\leftarrow \overrightarrow{w_t}+y_{n(t)}\cdot \overrightarrow{x_{n(t)}}, \text{until no mistake can be found}.$

$y_{n(t)}\overrightarrow{w_f}^T\overrightarrow{x_{n(t)}}\geq \underset{n}{\text{min}}~y_n\overrightarrow{w_f}^T\overrightarrow{x_n}>0.$

$\overrightarrow{w_f}^T\overrightarrow{w_{t+1}}=\overrightarrow{w_f}^T(\overrightarrow{w_t}+y_{n(t)}\overrightarrow{x_{n(t)}})$

$\geq \overrightarrow{w_f}^T\overrightarrow{w_t}+\underset{n}{\text{min}}~y_n\overrightarrow{w_f}^T\overrightarrow{x_n}>\overrightarrow{w_f}^T\overrightarrow{w_t}.$

$||\overrightarrow{w_{t+1}}||^2=||\overrightarrow{w_t}+y_{n(t)}\overrightarrow{x_{n(t)}}||^2\leq ||\overrightarrow{w_t}||^2+||y_{n(t)}\overrightarrow{x_{n(t)}}||^2\leq ||\overrightarrow{w_t}||^2+\underset{n}{\text{max}}~||y_n\overrightarrow{x_n}||^2.$

$||\overrightarrow{w_T}||^2 \leq ||\overrightarrow{w_0}||^2 + T\cdot \underset{n}{\text{max}}||\overrightarrow{x_n}||^2=T\cdot {\text{max}}||\overrightarrow{x_n}||^2=T\cdot R^2.$

$\frac{\overrightarrow{w_f}^T\overrightarrow{w_T}}{||\overrightarrow{w_f}||}\geq \frac{\overrightarrow{w_f}^T\overrightarrow{w_{T-1}}}{||\overrightarrow{w_f}||}+\underset{n}{\text{min}}~y_n\frac{\overrightarrow{w_f}^T}{||\overrightarrow{w_f}||}\overrightarrow{x_n}$

$\geq \frac{\overrightarrow{w_f}^T\overrightarrow{w_0}}{||\overrightarrow{w_f}||}+T\cdot \underset{n}{\text{min}}~y_n\frac{\overrightarrow{w_f}^T}{||\overrightarrow{w_f}||}\overrightarrow{x_n}=T\cdot \underset{n}{\text{min}}~y_n\frac{\overrightarrow{w_f}^T}{||\overrightarrow{w_f}||}\overrightarrow{x_n}.$

$\frac{\overrightarrow{w_f}^T\overrightarrow{w_T}}{||\overrightarrow{w_f}||\cdot ||\overrightarrow{w_T}||}\geq \sqrt{T}\cdot \frac{\underset{n}{\text{min}}~y_n\frac{\overrightarrow{w_f}^T}{||\overrightarrow{w_f}||}\overrightarrow{x_n}}{R}=\sqrt{T}\cdot \frac{\rho}{R}.$

$\sqrt{T}\cdot \frac{\rho}{R}\leq 1\Rightarrow T\leq \frac{R ^2}{\rho ^2}.$

0 0