用TensorFlow构建基础的神经网络(三)：AlexNet

参考文献：ImageNet Classification with Deep Convolutional NeuralNetworks
中文翻译：AlexNet论文翻译 (翻译作者：liumaolincycle)
作者：XJTU_Ironboy
时间：2017年8月

三、AlexNet

1.介绍

  AlexNet 可以说是具有历史意义的一个网络结构，可以说在AlexNet之前，深度学习已经沉寂了很久。AlexNet是在2012年被发表的一个经典之作，并在当年取得了ImageNet最好成绩，它在当年的ImageNet图像分类竞赛中，top-5错误率比上一年的冠军下降了十个百分点，而且远远超过当年的第二名。同时该网络结构也是开启ImageNet数据集上更大，更深CNN的开山之作，之后在Imagenet上取得更好结果的ZF-net，SPP-net，VGG等网络，都是在其基础上修改得到。

2.网络结构

(1) 废话少说，我们直接上图。

  从上图我们可以看出，**AlexNet**是一个由**5层卷积层和3个全连接层**构成的总共8层的神经网络。(2) **构成细节** Layers Cell Num Input Size Step1 Step2 Step3 Step4 Output Size Conv1 96 227*227*3 Conv Size:11*11 Strides:4 Padding:VALID ReLU Pool    Size:3*3 Strides:2 Padding:VALID LRN Size:5 27*27*96 Conv2 256 27*27*96 Conv    Size:5*5 Strides:1 Padding:SAME ReLU Pool    Size:3*3 Strides:2 Padding:VALID LRN Size:5 13*13*256 Conv3 384 13*13*256 Conv    Size:3*3 Strides:1 Padding:SAME ReLU 13*13*384 Conv4 384 13*13*384 Conv    Size:3*3 Strides:1 Padding:SAME ReLU 13*13*384 Conv5 256 13*13*384 Conv    Size:3*3 Strides:1 Padding:SAME ReLU 9216(6*6*256) FC6 4096 9216(6*6*256) Full Connect 4096 FC7 4096 4096 Full Connect 4096 FC8 1000 4096 Full Connect 1000

3.TensorFlow上实现

  提前声明：由于是放暑假在家，身边并没有合适的硬件设备(主要指：NVIDIA TITAN X GPUs)来跑这种ImageNet上的大型程序(原论文的作者用GPU都跑了5-6天)，所以这里只介绍如何构建AlexNet，并没有运行结果，望读者见谅！
① 开始定义AlexNet的整体结构

# 定义AlexNet的所有层的结构in_units =154587    # 227x227x3=154587C1_units = 96C2_units = 256C3_units = 384C4_units = 384C5_units = 256F6_units = 4096F7_units = 4096output_units = 1000

② 输入层

# 定义输入x = tf.placeholder(tf.float32,[None,in_units],name = "x_input")y_= tf.placeholder(tf.float32,[None,output_units],name = "y_input")keep_prob = tf.placeholder(tf.float32)input_data = tf.reshape(x,[-1,227,227,3])

③ 卷积层

# 定义可配置卷积层函数def conv(layer_name,input_x,Filter_size,activation_function = None):    with tf.name_scope("conv_%s" % layer_name):        with tf.name_scope("Filter"):            Filter = tf.Variable(tf.random_normal(Filter_size,stddev = 0.1),dtype = tf.float32,name = "Filter")            tf.summary.histogram('Filter_%s'%layer_name,Filter)        with tf.name_scope("bias_filter"):            bias_filter = tf.Variable(tf.random_normal([Filter_size[3]],stddev = 0.1),dtype = tf.float32,name = "bias_filter")            tf.summary.histogram('bias_filter_%s'%layer_name,bias_filter)        with tf.name_scope("conv"):            conv =  tf.nn.conv2d(input_x,Filter,strides = [1,1,1,1],padding = "SAME")            tf.summary.histogram('conv_%s'%layer_name,conv)        temp_y = tf.add(conv,bias_filter)        with tf.name_scope("output_y"):            if activation_function is None:                output_y = temp_y            else:                output_y = activation_function(temp_y)            tf.summary.histogram('output_y_%s'%layer_name,output_y)        return output_y

④ 池化层

# 定义可配置最大池化层函数def max_pool(layer_name,input_x,pool_size):    with tf.name_scope("max_pool_%s" % layer_name):        with tf.name_scope("output_y"):            output_y = tf.nn.max_pool(input_x,pool_size,strides = [1,2,2,1],padding = "SAME")            tf.summary.histogram('output_y_%s'%layer_name,output_y)        return output_y

⑤ 全连接层

# 定义可配置全连接层函数def full_connect(layer_name,input_x,output_num,activation_function = None):    with tf.name_scope("full_connect_%s" % layer_name):        with tf.name_scope("Weights"):            Weights = tf.Variable(tf.random_normal([input_x.shape.as_list()[1],output_num],stddev = 0.1),dtype = tf.float32,name = "weight")            tf.summary.histogram('Weights_%s'%layer_name,Weights)        with tf.name_scope("biases"):            biases = tf.Variable(tf.random_normal([output_num],stddev = 0.1),dtype = tf.float32,name = "biases")            tf.summary.histogram('biases_%s'%layer_name,biases)    output_temp = tf.add(tf.matmul(input_x,Weights) , biases)    with tf.name_scope("output_y"):        if activation_function is None:            output_y = output_temp        else:            output_y = activation_function(output_temp)        tf.summary.histogram('output_y_%s'%layer_name,output_y)    return output_y

⑥ 构建整体结构

# 第一层卷积层layer1_Conv = conv("layer1",input_data,[11,11,3,96],4,activation_function = tf.nn.relu)layer1_Pool = max_pool("layer1",layer1_Conv,[1,3,3,1])layer1_lrn = tf.nn.local_response_normalization(layer1_Pool,5,bias = 1.0,alpha = 0.001/9,beta = 0.75,name = "lrn")# 第二层卷积层layer2_Conv = conv("layer2",layer1_lrn,[5,5,96，256],1,activation_function = tf.nn.relu)layer2_Pool = max_pool("layer2",layer2_Conv,[1,3,3,1])layer2_lrn = tf.nn.local_response_normalization(layer2_Pool,5,bias = 1.0,alpha = 0.001/9,beta = 0.75,name = "lrn")# 第三层卷积层layer3_Conv = conv("layer3",layer2_lrn,[3,3,256,384],1,activation_function = tf.nn.relu)# 第四层卷积层layer4_Conv = conv("layer4",layer3_Conv,[3,3,384,384],1,activation_function = tf.nn.relu)# 第五层卷积层layer5_Conv = conv("layer5",layer4_Conv,[3,3,384,256],1,activation_function = tf.nn.relu)# 将特征图reshape为向量layer5_vector = tf.reshape(layer5_Conv,[-1,tf.cast(layer5_Conv.shape[1]*layer5_Conv.shape[2]*layer5_Conv.shape[3],tf.int32)])# 第六层全连接层layer6_FC = full_connect("layer6",layer5_vector,4096,activation_function = tf.nn.relu)# 第七层全连接层layer7_FC = full_connect("layer7",layer6_FC,4096,activation_function = tf.nn.relu)# 第八层全连接层输出with tf.name_scope("output_y"):    y = full_connect("output_layer",layer7_FC,1000,activation_function = tf.nn.relu)

4.总结

  AlexNet 之所以能够成功，深度学习之所以能够重回历史舞台，原因在于：
  1. 非线性激活函数：ReLU
  2. 防止过拟合的方法：Dropout，Data augmentation
  3. 大数据训练：百万级ImageNet图像数据
  4. 其他：GPU实现，LRN归一化层的使用