PCA, Factor Analysis, regression related

参考

http://www.cnblogs.com/jerrylead/archive/2011/05/11/2043317.html

http://blog.csdn.net/puqutogether/article/details/43055633

http://blog.csdn.net/puqutogether/article/details/40889719

regression problem

• m<<n. the number of samples or instances is greater than the number of features dimension, leading to the X is singular.
  
  • using factor analysis to solve.
  • pca dimension
• overfitting (using pca or regularizer to avoid this issue)
• underfitting. the training data set is so small. using the local weighted linear regression.

single variable linear regression

multivariate linear regression

polynomial non-linear regression

logistic regression (non-linear)

http://blog.csdn.net/puqutogether/article/details/43191099

bayesian regression

bayesian estimation—MAP

introduce the prior for the parameters, so that the overfitting problem can be relieved. e.g. l2 norm prior

the prior can also be any extra knowledge, which is also called Occam’s razor.