Training Very Deep Networks公式推导
来源:互联网 发布:计算机与网络期刊等级 编辑:程序博客网 时间:2024/05/19 20:44
原论文:Training Very Deep Networks
作者:Rupesh Kumar Srivastava Klaus Greff Jurgen Schmidhuber
时间:22 Jul 2015
本文的大部分观点来自于这篇论文,并且加入了一些自己的理解。该博客纯属读书笔记。
假设一个简单的由L层隐层构成的网络,我们令它的参数去拟合函数H,为了推导方便,我们假设输入输出的维度相同。那么我们可以得到输出y:
1
通过实验证明求解器很难优化这个函数,所以我们将这个函数做适当的变形。我们加了两个新的函数T和C:
2
在这里我们可以简单的将函数T理解为实现了对原函数H,也就是输入输出的映射函数的一个放大或缩小的功能。函数C是对输入C的直接传送,使输入不经过函数H的映射直接加到输出y上。they express how much of the output is produced by transforming the input and carrying it 。
为了让式子更简洁,我们让C=1-T,得到:
3
我们发现:
4
当T=0,输入和输出变成了恒等映射关系,当T=1,输入和输出变成了原来的y=H(x)。
而当T=0.5时我们发现,y=0.5*H(x)+0.5x,再用F(x)代替H(x)*2,我们发现y=x+F(x),所以残差函数实际上是函数3的一个特例。
文章除了实验以外,并没有给出足够的理论依据。
0 0
- Training Very Deep Networks公式推导
- Training Very Deep Networks
- Training Very Deep Networks--Highway Networks
- Training Very Deep Networks--Highway Networks 论文笔记
- Efficient Training of Very Deep Neural Networks for Supervised Hashing
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Very Deep Convolutional Networks for Large-Scale Image Recognition(精读)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- very deep convolutional networks for large-scale image recognition---vggnet
- VERY DEEP CONVOLUTIONAL NETWORKS FOR LARGE-SCALE IMAGE RECOGNITION
- 论文Very Deep Convolutional Networks for Large-Scale Image Recognition
- Very Deep Convolutional Networks for Large-Scale Image Classification
- Very deep convolutional networks for large-scale image recognition
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Accurate Image Super-Resolution Using Very Deep Convolutional Networks
- Very deep convolutional networks for larage-scale image recognition
- DeepID3:Face Recognition with Very Deep Neural Networks
- 重载运算符之友元函数和成员函数
- python dict学习:模拟用户登录行为
- Android 圆形头像 类似QQ
- Android 开源框架Universal-Image-Loader完全解析(一)--- 基本介绍及使用
- 揭秘Android 6.0 之 OkHttp
- Training Very Deep Networks公式推导
- js原生态ajax异步处理
- C语言控制台窗口图形界面编程
- Android里的SVG
- 汇编输入字符串并求空格个数
- linux下的java开发环境
- 机器学习之线性分类
- C语言操作符
- ps学习心得