Attention Mechanism in Deep Learninig

Collections and Understanding

- What is attention mechanism?

- What’s the difference between Soft attention and Hard attention?

---

Reference:

• Deep Learning: Progress in Theory and Attention Mechanisms
• Attention and Memory in Deep Learning and NLP

Further Reading

• MeteMind DMN_Dynamic Memory Nets
• NVIDIA Introduction to Neural Machine Translation with GPUs PART1
• NVIDIA Introduction to Neural Machine Translation with GPUs PART2
• NVIDIA Introduction to Neural Machine Translation with GPUs PART3
• HEURITECH – LE BLOG ATTENTION MECHANISM

研究Attention或者图像内容显著性研究的机构、人员(non-DL)：

• mingming chen图像内容的显著性与相似性研究(清华大学博士论文)
• xiaodi hou_Saliency Detection: A Spectral Residual Approach
  (5 lines matlab code)
• Koch
• vgg oxford
• ilab_Laurent Itti_University of Southern California