Deep Learning重要论文整理
来源:互联网 发布:千牛什么时候出mac版 编辑:程序博客网 时间:2024/06/06 01:09
非线性单元:
Maxout
Ian J Goodfellow, David Warde-Farley, Mehdi Mirza, Aaron Courville, and Yoshua Bengio. Maxout networks. arXiv preprint arXiv:1302.4389, 2013.
dropout
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1): 1929–1958, 2014.
LReLU
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. arXiv preprint arXiv:1502.01852, 2015.
目前非线性单元一般不破坏ReLU的结构而用非线性的运算方法接入网络层与层之间来产生非线性表达能力。
增加模型深度:
NIN
Min Lin, Qiang Chen, and Shuicheng Yan. Network in network. 12 2013. URL http://arxiv.org/abs/1312.4400.
Inception/GoogLeNet
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. arXiv preprint arXiv:1409.4842, 2014.
这里指的“深度”不光指层数,仅靠增加层数会带来训练困难。这里是指在有限层增加网络复杂程度。也可以说是非线性单元的一种变体。
训练过程中的效率:
LReLU
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. arXiv preprint arXiv:1502.01852, 2015.
BatchNorm
Sergey Ioffe, Christian Szegedy,. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift.
前者为参数提供了更加容易收敛初始值,后者防止训练过程中的梯度发散,两者都是解决同类问题,vanishing gradients(前)和exploding gradients(后)。
Detection
RCNN
Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation.
CV proposal + CNN feature extraction + SVM classifier
Fast RCNN
Ross Girshick. Fast R-CNN.
CV proposal + CNN feature extraction + Regression Network Prediction
Faster RCNN
Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.
CNN Regression Network Proposal + CNN feature extraction + Regression Network Prediction
- Deep Learning重要论文整理
- [deep learning] 部分论文
- Deep Learning论文选读
- deep learning资料整理
- Deep-Learning paper整理
- Deep Learning资源整理
- Deep Learning 资料整理
- Deep Residual Learning 论文解析
- Machine Learning & Deep Learning 论文阅读笔记
- 最近看过的部分Deep Learning论文
- [deep learning] 最近看过的部分论文
- Tag deep-learning 一大堆深度学习论文
- Tag deep-learning 一大堆深度学习论文
- 【论文笔记】Learning Deep Face Representation
- Deep&Wide Learning论文阅读笔记
- 论文-Deep Residual Learning for Image Recognition
- 论文笔记-deep learning-detection (2017)
- 论文阅读——Wide & Deep Learning
- 接口封装之暴露内部过多
- 我的Java之路三:登录程序,自己动手会有一点点成就感!
- 数据库SQL优化大总结之 百万级数据库优化方案
- Hadoop学习笔记—3.Hadoop RPC机制的使用
- 常见定理与方法(一)
- Deep Learning重要论文整理
- qt中获取文件路径和文件名
- tcpdump
- Linux服务管理
- babel 编译es6 connot find preset 解决方案
- MySQL性能优化的最佳21条经验
- Unity+Vuforia SDKAR开发系列教程--2.1.3 Vuforia许可证购买
- HDU 5883 The Best Path (一笔画 / 欧拉通路)
- C#中的静态与非静态方法比较