Training Very Deep Networks
来源:互联网 发布:开开贷网络借钱 编辑:程序博客网 时间:2024/06/07 06:25
这篇文章提出一种训练深层网络的训练结构-highway,主要的来源于LSTM中的阀门开关的思想。highway的提出使得可以使用梯度下降可以直接训练较深的卷积神经网络。
plain网络前向过程为:
其中H为非线性激活函数。对于highway网络来说在上述的基础上又引入了两个非线性变换T和C,则highway为:
其中T为transform gate,C为carry gate,T和C实际上是定义了最终的输出分别是由多少变换和输入构成。为了简单,可以另C=1-T:
使用就卷积进行表达为:
特殊情况为:
需要注意的是highway的定义方式要求H,T,x,y具有相同的维度。
plain网络于highway结构比较如下:
实验结果说明:plain网络要比highway难优化。
0 0
- Training Very Deep Networks
- Training Very Deep Networks--Highway Networks
- Training Very Deep Networks公式推导
- Training Very Deep Networks--Highway Networks 论文笔记
- Efficient Training of Very Deep Neural Networks for Supervised Hashing
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Very Deep Convolutional Networks for Large-Scale Image Recognition(精读)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- very deep convolutional networks for large-scale image recognition---vggnet
- VERY DEEP CONVOLUTIONAL NETWORKS FOR LARGE-SCALE IMAGE RECOGNITION
- 论文Very Deep Convolutional Networks for Large-Scale Image Recognition
- Very Deep Convolutional Networks for Large-Scale Image Classification
- Very deep convolutional networks for large-scale image recognition
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Accurate Image Super-Resolution Using Very Deep Convolutional Networks
- Very deep convolutional networks for larage-scale image recognition
- DeepID3:Face Recognition with Very Deep Neural Networks
- xcode7免证书真机调试
- hdoj2034人见人爱A-B
- kafka文档(3)---- 配置选项翻译
- Spring MVC文件上传
- Kafka文档(1)---- 基本介绍(Getting Start)
- Training Very Deep Networks
- OpenCV【6】---使用均值漂移(Mean Shift)算法实时跟踪视频流中的物体
- 查找
- Java学习笔记之一个蛋疼的小细节
- kafka文档(2)----kafka API(java版本)
- ProtocolBuffer for Objective-C 运行环境配置及使用
- 老男孩Python培训第八期
- 单元测试的好处
- android消息队列机制(Handler)