point-generator network and Coverage mechanism for summary
来源:互联网 发布:08 经济危机 知乎 编辑:程序博客网 时间:2024/06/14 19:29
论文来自: Get To The Point: Summarization with Pointer-Generator Networks
传统的神经方法不能处理OOV以及解码时的重复问题。这篇文章提出point网络用来解决OOV问题,coverage用来解决重复问题。
1. point-generator network
解码时用到的vocabulary是这个batch里的词组成,且是频率从高到低的N-Size个词,这个batch里的其他词不放入vocabulary里,比如超低频词。
注意这里P代表的是一个soft switch 而不是一个hard switch。由公式(7)可知,解码生成某个词的概率由两部分组成,第一个是生成模式生成这个词的概率;第二个是从源句子里抽取某个词的概率。公式七里有几个要解释的点:
ati 表示源句子中每个位置的attention scorei:wi=w 表示源句子中和词w相同的位置
如果w是个OOV词,则Pvoc(w)就等于0;如果w没有出现在源文档中,则公式7左边的第二个components就等于0
2. Coverage mechanism
This ensures that the attention mechanisms current decision (choosing where to attend next) is informed by a reminder of its previous decisions。This should make it easier for the attention mechanism to avoid repeatedly attending to the same locations, and thus avoid generating repetitive text.
两个向量相应位置处的元素取最小值,然后在向量上求和。上面公式里的a(i,t)代表解码第i个词时对应的attention soce,用他来和C做比较。the final loss function(下面公式单个step的loss):
- point-generator network and Coverage mechanism for summary
- gstreamer network buffering mechanism
- distinct between point and line (for svm)
- Deep Convolutional Network Cascade for Facial Point Detection(泛读)
- Deep Convolutional Network Cascade for Facial Point Detection阅读笔记
- Deep Convolutional Network Cascade for Facial Point Detection阅读笔记
- Deep Convolutional Network Cascade for Facial Point Detection
- Deep Convolutional Network Cascade for Facial Point Detection阅读笔记
- Deep Convolutional Network Cascade for Facial Point Detection阅读笔记
- Deep Convolutional Network Cascade for Facial Point Detection实践总结
- C. Watto and Mechanism
- Watto and Mechanism
- Watto and Mechanism CodeForces
- 802.11 network standards summary
- Code Coverage for C++
- End Point Performance Milestone Summary
- leetcode 4Sum Summary for unique and duplicates
- fs-cache and cachefs for network filesystem
- addLayer
- Unity AssetImporter.GetAtPath 用法
- Python中的yield关键字
- 翻转问题(对于线性结构)
- 分段数据
- point-generator network and Coverage mechanism for summary
- myeclipse下添加外部tomcat同步显示两个的问题
- 数据拟合教程
- 屌丝码农从业10年,今天我要改行了
- 最大子段和
- webview加载html文件,如何调用原生态的代码的
- eclipse新浪云sae插件安装
- Shell 间接变量问题
- 实施CMMI3有感:CMMI绝对是治疗软件公司特大毒瘤的特效药!