csn231SettingUpTheDataAndTheModel

#####################
setting up the data and the model
Data Preprocessing
Weight Initilization
Batch Normalization
Regularization(L2/L1/Maxnorm/Dropout)
#######################
Loss functions
#######################
summary
#######################
Data Preprocessing
Method1
Mean subtraction
Normalization
Method2
PCA and Whitening
Common pitfall:
the mean must be computed only over the training data 
and then subtracted equally from all splits (train/val/test).
#######################
Weight Initialization
1.Small random numbers
2.Calibrating the variance with 1/sqt(n)
3.Sparse initialization
Initizaling the biases
Batch Normalization is in practice
#######################
How to prevent over-fitting
Regularization
L2 regularization
L1 regularization
Max norm constraints
Dropout
Theme of noise in forward pass
Bias regularization
Pre-layer regularization
########################
Loss Functions
when large number of classes 
Hiearchical sofmax
Attribute classification
Regression