【Deep Learning】genCNN: A Convolutional Architecture for Word Sequence Prediction
来源:互联网 发布:不满意淘宝投诉结果 编辑:程序博客网 时间:2024/05/03 19:09
作者:Mingxuan Wang,李航,刘群
单位:华为、中科院
时间:2015
发表于:acl 2015
文章下载:http://pan.baidu.com/s/1bnBBVuJ
主要内容:
用deep learning设计了一种语言模型,能够根据之前“所有”的历史来预测当前词的条件概率。用语言模型迷惑度衡量、用机器翻译衡量,该模型都比baseline(5-gram、RNN、等)好
具体内容:
之前用deep learning在语言模型上的进展是:RNN和LSTM
参考的工具包:
RNN – http://rnnlm.org/
LSTM – https://github.com/lisa-groundhog/GroundHog本文作者的实现方式:
(1)用alpha-cnn来模拟当前词比较近的历史,约之前30个词;用beta-cnn来递归的模拟所有之前的历史。beta-cnn的输出是其他beta-cnn以及alpha-cnn的输入。网络结构如下:
(2)用了word2vec作为词语的输入,两层隐含层,用gate代替max pooling,最后输出层是softmax层
(3)同标准cnn不同的是:标准cnn在局部共享权重,本文既有共享的权重,也有不共享的权重
(4)训练方式是最大化训练语料中句子的概率实验结果(困惑度)
5-gram KN smoothing: 270
RNN:223
LSTM:206
本文方法:180
另外,训练时间比较长,1M句子,用了GPU还训练了2天。
0 0
- 【Deep Learning】genCNN: A Convolutional Architecture for Word Sequence Prediction
- 【SegNet】SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image
- SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
- Learning a Deep Convolutional Network for Image Super-Resolution(泛读)
- Learning a Deep Convolutional Network for Image Super-Resolution(泛读)
- Shallow and Deep Convolutional Networks for Saliency Prediction
- 【论文翻译】SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
- 语义分割-- SegNet:A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
- SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation 视频语义分割demo跑通
- 论文笔记 | SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
- [论文笔记]SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
- Learning a Deep Convolutional Network for Image Super-Resolution—Chao Dong_ECCV2014
- 【Deep Learning学习笔记】A Unified Architecture for Natural Language Processing_ICML2008
- Semi-supervised Deep Learning for Fully Convolutional Networks文章解读
- Convolutional Sequence to Sequence Learning
- CVPR2015:An Improved Deep Learning Architecture for Person Re-Identificaton
- 人群分析--ResnetCrowd: A Residual Deep Learning Architecture
- DeCAF: A deep convolutional activation feature for generic visual recognition
- 二义性 消除左递归
- c 循环左移
- 黑马程序员---java多线程的一些常见问题
- 我应该直接学 Swift,还是 Objective-C?
- js的时分插件(无日期)
- 【Deep Learning】genCNN: A Convolutional Architecture for Word Sequence Prediction
- nmap,端口扫描,获取ssh服务器的ip地址
- c 转二进制
- 第一次的感悟
- 常见MIME类型
- grideview只显示一行的问题
- iOS-学习笔记-UI-第十九天
- 4.17~4.22
- leetcode[112]:Path Sum