Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
来源:互联网 发布:什么牌子的网络电视好 编辑:程序博客网 时间:2024/05/16 15:17
Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
(Submitted on 6 Jun 2016 (v1), last revised 24 Sep 2016 (this version, v3))
Modeling textual or visual information with vector representations trained from large language or visual datasets has been successfully explored in recent years. However, tasks such as visual question answering require combining these vector representations with each other. Approaches to multimodal pooling include element-wise product or sum, as well as concatenation of the visual and textual representations. We hypothesize that these methods are not as expressive as an outer product of the visual and textual vectors. As the outer product is typically infeasible due to its high dimensionality, we instead propose utilizing Multimodal Compact Bilinear pooling (MCB) to efficiently and expressively combine multimodal features. We extensively evaluate MCB on the visual question answering and grounding tasks. We consistently show the benefit of MCB over ablations without MCB. For visual question answering, we present an architecture which uses MCB twice, once for predicting attention over spatial features and again to combine the attended representation with the question representation. This model outperforms the state-of-the-art on the Visual7W dataset and the VQA challenge.
Submission history
From: Marcus Rohrbach [view email][v1] Mon, 6 Jun 2016 17:59:56 GMT (2194kb,D)
[v2] Thu, 23 Jun 2016 19:52:41 GMT (3358kb,D)
[v3] Sat, 24 Sep 2016 01:58:59 GMT (3443kb,D)
阅读全文
0 0
- 论文笔记 :Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
- 阅读笔记(Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding)
- Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
- Multimodal Compact Bilinear Pooling for Multimodal Neural Machine Translation
- VQA 之 Multimodal Compact Bilinear Pooling
- VQA 之 Multimodal Compact Bilinear Pooling
- Paper Reading - Snap and ask: Answering Multimodal Question by Naming Visual Instance
- Dynamic Memory Networks for Visual and Textual Question Answering
- Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering
- Hierarchical Question-Image Co-Attention for Visual Question Answering
- Hierarchical Question-Image Co-Attention for Visual Question Answering
- 论文笔记:Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
- 论文笔记: Hierarchical Question-Image Co-Attention for Visual Question Answering
- (reading)Revisiting Visual Question Answering Baselines
- 论文笔记: Compact Bilinear Pooling
- ABC-CNN: An Attention Based Convolutional Neural Network for Visual Question Answering
- 视觉问答(Visual Question Answering)论文初步整理
- 论文笔记: (compact) Bilinear Pooling, Confusion
- webstorm 2017激活
- Apache中的poi包中的XSSFWorkbook与HSSFWorkbook的区别
- 23.按字节数截取字符串
- imgproc模块—腐蚀和膨胀
- Python机器学习探索(一)——计算机学习数据的能力
- Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
- 系统分析与设计学习笔记(四)领域模型
- Multimodal Compact Bilinear Pooling for Multimodal Neural Machine Translation
- 图形分析之Nsight的使用
- 棘修CADfix 7.1+Grapyuoft Crchicbd 2017
- 协程resume()传入的参数
- 20170611 用JS while循环实现斐波那契数列求兔子总数
- 第8章 不相交集类
- angularJs中$sce服务安全显示html文本