【论文笔记】residual neural network-kaiming he
来源:互联网 发布:淘宝官方网站登录网站 编辑:程序博客网 时间:2024/06/04 20:11
http://arxiv.org/abs/1512.03385
The stack of layer will cause degradation(same train, more layer, less accuracy, which is not caused by overfit or derivation vanishing).
H(x) is expected result, F(x) is residual error, H(x)=F(x)+x. Assume F(x) is easier be approached by CNN than H(x). Experiment support this assumption.
Bottle-neck architecture of ResNets is more economical.
ReLu
Activation function. Its derivative is logistic function. y = 0 when x < 0, it reduce the number of active neuron in network. Therefore there are only about half of parameters should be modified when BP, increase the training speed.
derivation vanishing
Activation function has output range (-1,1) or (0,1), cause the decrease of derivation in back propagation, making shallow layer parameters can not be modified effectively, called vanish.
0 0
- 【论文笔记】residual neural network-kaiming he
- 论文笔记《Single Image Haze Removal Using Dark Channel Prior--Kaiming He》
- 论文笔记:Residual Attention Network for Image Classification
- 【论文笔记】Recurrent Neural Network Regularization
- 论文笔记之:Progressive Neural Network
- [论文笔记] CRNN(Convolutional Recurrent Neural Network)_2015 学习笔记
- 【论文笔记】Image Classification with Deep Convolutional Neural Network
- 论文《A Convolutional Neural Network Cascade for Face Detection》笔记
- 【论文笔记】二值化神经网络(Binarized Neural Network)
- 论文笔记 UberNet : Training a ‘Universal’ Convolutional Neural Network
- 【论文笔记】A Convolutional Neural Network Cascade for Face Detection
- 论文《A Convolutional Neural Network Cascade for Face Detection》笔记
- 论文《A Convolutional Neural Network Cascade for Face Detection》笔记
- 【论文笔记】二值化神经网络(Binarized Neural Network)
- 论文笔记 UberNet : Training a ‘Universal’ Convolutional Neural Network
- 论文笔记 | Wide Residual Networks
- Neural Network 笔记
- 神经网络笔记(Neural Network)
- 微信公众平台java开发详解(工程代码+解析)
- Java NIO
- C# 实验三 找出字符串中的有用信息
- Java基本数据类型
- 华为2015年校园招聘机试题(2)
- 【论文笔记】residual neural network-kaiming he
- Caused by: java.lang.NoSuchMethodError: ....SchedulerFactoryBean.getObject()Lorg/quartz/Scheduler
- SSH 在中出现的could not initialize proxy - no Session 问题
- Android 递归栈溢出问题解决
- 郑轻OJ zzulioj 1068: 二进制数
- android studio改变主题字体大小
- 奖券数目
- 前端优化建议
- Java 异常处理