RDL Research Database - Academic Research Discovery Platform

Abstract

Neural machine translation (NMT) models typically operate with a fixed vocabulary, but translation is an open-vocabulary problem.Previous work addresses the translation of out-of-vocabulary words by backing off to a dictionary.In this paper, we introduce a simpler and more effective approach, making the NMT model capable of open-vocabulary translation by encoding rare and unknown words as sequences of subword units.This is based on the intuition that various word classes are translatable via smaller units than words, for instance names (via character copying or transliteration), compounds (via compositional translation), and cognates and loanwords (via phonological and morphological transformations).We discuss the suitability of different word segmentation techniques, including simple character ngram models and a segmentation based on the byte pair encoding compression algorithm, and empirically show that subword models improve over a back-off dictionary baseline for the WMT 15 translation tasks English→German and English→Russian by up to 1.1 and 1.3 BLEU, respectively.

Keywords

Machine translationComputer scienceTranslation (biology)Natural language processingArtificial intelligenceTransfer-based machine translationExample-based machine translationSpeech recognitionChemistry

Affiliated Institutions

University of Edinburgh GB

Related Publications

Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

Yonghui Wu , Mike Schuster , Zhifeng Chen +28 more

Neural Machine Translation (NMT) is an end-to-end learning approach for automated translation, with the potential to overcome many of the weaknesses of conventional phrase-based...

2016 arXiv (Cornell University) 5624 citations

Letter N-Gram-based Input Encoding for Continuous Space Language Models

Henning Sperr , Jan Niehues , Alex Waibel

We present a letter-based encoding for words in continuous space language models. We represent the words completely by letter n-grams instead of using the word index. This way, ...

2013 22 citations

Deep Recurrent Models with Fast-Forward Connections for Neural Machine Translation

Jie Zhou , Ying Cao , Xuguang Wang +2 more

Neural machine translation (NMT) aims at solving machine translation (MT) problems using neural networks and has exhibited promising results in recent years. However, most of th...

2016 Transactions of the Association for C... 223 citations

Sequence to Sequence Learning with Neural Networks

Ilya Sutskever , Oriol Vinyals , Quoc V. Le

Deep Neural Networks (DNNs) are powerful models that have achieved excellent performance on difficult learning tasks. Although DNNs work well whenever large labeled training set...

2014 arXiv (Cornell University) 13295 citations

Publication Info

Year: N/A
Type: article
Citations: 6994
Access: Closed

External Links

Citation Metrics

6994

OpenAlex

Cite This

APA Style

                            
                                    Rico Sennrich, 
                                
                                    Barry Haddow, 
                                
                                    Alexandra Birch
                                
                            (n.d.). 
                            . 
                            Edinburgh Research Explorer (University of Edinburgh)
                            
                            .