[paper note] Densely Connected Convolutional Networks
来源:互联网 发布:命名实体识别算法 编辑:程序博客网 时间:2024/06/01 23:14
- Code
- paper
Intuition
- Current trend of CNN architecture: create short paths from early layers to later layers.
- ResNet
- Highway network: The first network with more than 100 layers, bypassing paths
- Stochastic depth: Improves the training of deep residual networks by dropping layers randomly during training, which manages to train a 1202-layer ResNet
- FractalNets
- Wide filter is helpful.
- Connect all layers with each other.
- Combine features by concatenating them (ResNet combines by summation).
- DenseNet layers are very narrow (12 feature-maps per layer), resulting in less parameters
Model
- Dense connectivity: concatenate all the preceding layers:
xl=Hl([x0,x1,…,xl−1]) - Composite function:
H_l
is defined asBN + ReLU + 3x3 Conv
- Pooling and dense block
- See figure above
- Transition layer between dense blocks, consist of
BN + 1x1 Conv + 2x2 AveragePooing
- Growth rate
k
:- The number of output feature-maps.
- The
l
-th layer will havek x (l-1) + k_0
input feature-maps (k_0
:input image channels)
- Bottleneck layers
- Introduce 1x1 Conv before 3x3 Conv to reduce number of feature-maps will improve computation efficiency.
H_l
is changed toBN + ReLU + 1x1 Conv + BN + ReLU + 3x3 Conv
1x1 Conv
reduce the input to4k
feature-maps in the experiment.
- Compression
- Reduce feature-maps number in transition layer by factor
θ
- Reduce feature-maps number in transition layer by factor
Experiment
- Datasets
- CIFAR-10/100, 32x32
- Zero-padded with 4 pixels on each side
- Randomly cropped to again produce 32×32 images
- Half of the images are then horizontally mirrored
- SVHN (Street View House Numbers), 32x32
- ImageNet, 224x224, 1.2m for training, 50000 for validation, 1000 classes
- CIFAR-10/100, 32x32
- Settings: weight decay 10e-4, Nesterov momentum of 0.9 w\o dampening, learning rate 0.1 with decay scheme, dropout when no data augmentation
- Accuracy result:
- 3.46% on CIFAR-10, L=190, k=40
- 17.18% on CIFAR-100, L=190, k=40
- 1.59% on SVHN, L=100, k=24
- Capacity: the performance continues improving when L, k increase, showing the DenseNet is less prone to overfitting (???)
- Parameter efficiency.
0 0
- [paper note] Densely Connected Convolutional Networks
- READING NOTE: Densely Connected Convolutional Networks
- Densely Connected Convolutional Networks
- 【文献阅读】Densely Connected Convolutional Networks-best paper-CVPR-2017
- Densely Connected Convolutional Networks(DenseNet)
- DenseNet--Densely Connected Convolutional Networks
- Densely Connected Convolutional Networks翻译
- 论文阅读:DENSELY CONNECTED CONVOLUTIONAL NETWORKS
- 文献《Densely Connected Convolutional Networks》阅读笔记
- 论文阅读-《Densely Connected Convolutional Networks》
- 网络模型--Densely Connected Convolutional Networks
- 网络模型--Densely Connected Convolutional Networks
- Densely Connected Convolutional Networks-2017 CVPR 优秀论文
- 论文阅读 《Densely Connected Convolutional Networks》
- Densely Connected Convolutional Networks论文解析
- 论文笔记 | Densely Connected Convolutional Networks
- 论文笔记:Densely Connected Convolutional Networks(DenseNet模型详解)
- CVPR2017 最佳论文——Densely Connected Convolutional Networks
- Android CountDownTimer 简单实例
- 第15周OJ实践10 字符串比较
- 深度信念网络(DBN)
- [Android进阶]学习AccessibilityService实现微信抢红包插件
- 第3章Cucumber场景与附注实例---------------Cucumber简单操作实例
- [paper note] Densely Connected Convolutional Networks
- Google Guava官方教程(中文版)
- 关于C语言中 || 和 &&的用法
- Hive函数编程-关系运算
- ios 静态库(.a)文件相关流程及-ObjC, -all_load, -force_load
- iOS开发之懒加载2
- OPENCV边缘提取
- 用筛选法求n以内的素数
- Linux下不同运维人员共用root 账户权限审计