ImageNet Evolution论文笔记（1）

Imagenet classification with deep convolutional neural networks

1，The Dataset

ImageNet：we down-sampled the images to a fixed resolution of 256 × 256. Given arectangular image, we first rescaled the image such that the shorter side was of length 256, and then cropped out the central 256×256 patch from the resulting image。subtracting the mean activity over the training set from each pixel

2，The Architecture

five convolutional layers, some of which are followed by max-pooling layers,and three fully-connected layers with a final 1000-way softmax


Local Response Normalization

3，Reducing Overfitting

Data Augmentation
1，extracting random 224 × 224 patches (and their horizontal reflections) from the256×256 images and training our network on these extracted patches；随机地从256x256的原始图像中截取224x224大小的区域(以及水平翻转的镜像)，相当于增加了(256-224)^2x2=2048倍的数据量。
2，At test time, the network makes a prediction by extracting five 224 × 224 patches (the four corner patches and the center patch) as well as their horizontal reflections (hence ten patches in all), and averaging the predictions made by the network’s softmax layer on the ten patches。进行预测时，则是取图片的四个角加中间共5个位置，并进行左右翻转，一共获得10张图片，对他们进行预测并对10次结果求均值。
3， perform PCA on the set of RGB pixel values throughout the ImageNet training set.对图像的RGB数据进行PCA处理，并对主成分做一个标准差为0.1的高斯扰动，增加一些噪声。

Dropou
1，At test time, we use all the neurons but multiply their outputs by 0.5
,2，use dropout in the first two fully-connected layers

4，Details of learning

1，a batch size of 128 examples, momentum of 0.9, and weight decay of 0.0005

2，We initialized the weights in each layer from a zero-mean Gaussian distribution with standard deviation 0.01. We initialized the neuron biases in the second, fourth, and fifth convolutional layers,as well as in the fully-connected hidden layers, with the constant 1. This initialization accelerates the early stages of learning by providing the ReLUs with positive inputs. We initialized the neuron biases in the remaining layers with the constant 0.

结构代码：