litlas
ログインアカウント作成

Effective Approaches to Attention-based Neural Machine Translation

著者: Thang Luong, Hieu Pham, Christopher D. Manning - Conference on Empirical Methods in Natural Language Processing, EMNLP 2015 被引用: 8,582

An attentional mechanism has lately been used to improve neural machine translation (NMT) by selectively focusing on parts of the source sentence during translation. However, there has been little work exploring useful architectures for attention-based NMT. This paper examines two simple and effective classes of attentional mechanism: a global approach which always attends to all source words and a local one that only looks at a subset of source words at a time. We demonstrate the effectiveness of both approaches over the WMT translation tasks between English and German in both directions. With local attention, we achieve a significant gain of 5.0 BLEU points over non-attentional systems which already incorporate known techniques such as dropout. Our ensemble model using different attention architectures has established a new state-of-the-art result in the WMT'15 English to German translation task with 25.9 BLEU points, an improvement of 1.0 BLEU points over the existing best system backed by NMT and an n-gram reranker.

Bleu: a Method forAutomatic Evaluation of…Bleu: a Method for Automatic Evaluation of Machine TranslationStatistical Phrase-BasedTranslationStatistical Phrase-Based TranslationRecurrent ContinuousTranslation ModelsRecurrent Continuous Translation ModelsSequence to SequenceLearning with Neural…Sequence to Sequence Learning with Neural NetworksLearning PhraseRepresentations using…Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine TranslationRecurrent Neural NetworkRegularizationRecurrent Neural Network RegularizationEnd-to-end ContinuousSpeech Recognition usin…End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First ResultsRecurrent Models ofVisual AttentionRecurrent Models of Visual AttentionNeural MachineTranslation by Jointly…Neural Machine Translation by Jointly Learning to Align and TranslateShow, Attend and Tell:Neural Image Caption…Show, Attend and Tell: Neural Image Caption Generation with Visual AttentionOn Using Very LargeTarget Vocabulary for…On Using Very Large Target Vocabulary for Neural Machine TranslationAddressing the Rare WordProblem in Neural…Addressing the Rare Word Problem in Neural Machine TranslationAbstractive SentenceSummarization with…Abstractive Sentence Summarization with Attentive Recurrent Neural NetworksConversational QueryUnderstanding Using…Conversational Query Understanding Using Sequence to Sequence ModelingRetainVis: VisualAnalytics with…RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical RecordsA Sequence-to-SequenceAir Quality Predictor…A Sequence-to-Sequence Air Quality Predictor Based on the n-Step Recurrent PredictionMemristive LSTM Networkfor Sentiment AnalysisMemristive LSTM Network for Sentiment AnalysisNeural MachineTranslation With…Neural Machine Translation With Sentence-Level Topic ContextSarcasm Detection UsingSoft Attention-Based…Sarcasm Detection Using Soft Attention-Based Bidirectional Long Short-Term Memory Model With Convolution NetworkRecurrent NeuralNetworks for Time Serie…Recurrent Neural Networks for Time Series Forecasting: Current Status and Future DirectionsSign LanguageTransformers: Joint…Sign Language Transformers: Joint End-to-End Sign Language Recognition and TranslationAn Adaptive Anti-NoiseNeural Network for…An Adaptive Anti-Noise Neural Network for Bearing Fault Diagnosis Under Noise and Varying Load ConditionsA Survey on theExplainability of…A Survey on the Explainability of Supervised Machine LearningA Reverse PositionalEncoding Multi-Head…A Reverse Positional Encoding Multi-Head Attention-Based Neural Machine Translation Model for Arabic DialectsEffective Approaches toAttention-based Neural…Effective Approaches to Attention-based Neural Machine Translation過去の参考文献中心の論文この論文を引用する論文古い新しい

ノードをクリックするとフォーカスを固定、空白をクリックすると本論文に戻ります。ホバーで一時的にプレビューできます。各ノードのページはタイトルから開けます。