Squeeze-and-Excitation Networks
来源:互联网 发布:linux配置javahome 编辑:程序博客网 时间:2024/06/06 03:10
SENet的思想是对于不同通道的feature map,我们认为他的重要性是不同的,SENet通过添加一个Squeeze-and-Excitation(SE)模块,学习不同feature map的权重,例如我们输入feature map为C个通道,那么通过SE模块学习到一个长度为C的权重向量w,向量的第i个值,代表第i个通道的feature map的权重,将权重向量w与输入feature map相乘,则可以起到抑制不重要feature map,而专注于重要的feature map的作用.Squeeze-and-Excitation模块网络结构如下:
对于输入 X,其特征通道树为
Squeeze-and-Excitation(SE)模块在Inception,ResNet中的应用如下:
图中,SE模块为一个global pooling,两个全连接层FC,以及一个激活函数sigmoid.关于这麽设计的思想,可以参考:http://www.sohu.com/a/161633191_465975
使用 global average pooling 作为 Squeeze 操作。紧接着两个 Fully Connected 层组成一个 Bottleneck 结构去建模通道间的相关性,并输出和输入特征同样数目的权重。首先将特征维度降低到输入的 1/16,然后经过 ReLu 激活后再通过一个 Fully Connected 层升回到原来的维度。这样做比直接用一个 Fully Connected 层的好处在于:1)具有更多的非线性,可以更好地拟合通道间复杂的相关性;
2)极大地减少了参数量和计算量。然后通过一个 Sigmoid 的门获得 0~1 之间归一化的权重,最后通过一个 Scale 的操作来将归一化后的权重加权到每个通道的特征上。
SE tensorflow代码实现:
def bottleneck_se(l, ch_out, stride, preact): l, shortcut = apply_preactivation(l, preact) l = Conv2D('conv1', l, ch_out, 1, nl=BNReLU) l = Conv2D('conv2', l, ch_out, 3, stride=stride, nl=BNReLU) l = Conv2D('conv3', l, ch_out * 4, 1) squeeze = GlobalAvgPooling('gap', l) squeeze = FullyConnected('fc1', squeeze, ch_out // 4, nl=tf.nn.relu) squeeze = FullyConnected('fc2', squeeze, ch_out * 4, nl=tf.nn.sigmoid) l = l * tf.reshape(squeeze, [-1, ch_out * 4, 1, 1]) return l + resnet_shortcut(shortcut, ch_out * 4, stride)
完整代码可见:
tensorflow代码:https://github.com/ppwwyyxx/tensorpack/blob/master/examples/ResNet/imagenet-resnet-se.py
caffe代码:https://github.com/hujie-frank/SENet
- Squeeze-and-Excitation Networks
- 网络模型--Squeeze-and-Excitation Networks
- Squeeze-and-Excitation Networks个人理解
- SENet(Squeeze-and-Excitation Networks)算法笔记
- Squeeze-and-Excitation Networks论文翻译——中文版
- Squeeze-and-Excitation Networks论文翻译——中英文对照
- Squeeze & Excitation(SE)
- ansoft maxwell报错ansoft maxwell报错, 激励与激励相互重叠(excitation and excitation overlap
- Computer and Communication Networks
- Cryptograhpy and Networks notes
- tensorflow function笔记: tf.expand_dims and tf.squeeze
- ZigBee Wireless Networks and Transceivers
- 电子书 computer and communication networks
- Detecting and Using LTE Networks
- Visualizing and Understanding convolutional networks
- Neural Networks and Deep Learning
- Neural Networks and Backpropagation Algorithm
- Software-Defined Networks and OpenFlow
- PostgreSQL基础5--聚合与排序
- systemctl命令介绍
- ListView多条目加载
- Linux搭建本地TFTP服务器
- ClassLoad类加载器读取ClassPath路径下的配置文件并完成创建JDBC链接(二)
- Squeeze-and-Excitation Networks
- leetcode---triangle---dp
- POJ3363
- Ubuntu常用操作汇总与问题解决汇总
- 乙胺卮园劳挖诼倩部难乩乓写劳乩
- 基于Tensorflow+Python实现降噪自动编码器(DAE)
- jQuery学习-对象与选择器(锋利的jQuery)
- [bzoj-2049][Sdoi2008]Cave 洞穴勘测 题解
- 点击按钮显示多个框