BLEU | RDL Research Database

Abstract

Human evaluations of machine translation are extensive but expensive. Human evaluations can take months to finish and involve human labor that can not be reused. We propose a method of automatic machine translation evaluation that is quick, inexpensive, and language-independent, that correlates highly with human evaluation, and that has little marginal cost per run. We present this method as an automated understudy to skilled human judges which substitutes for them when there is need for quick or frequent evaluations.

Keywords

Computer scienceMachine translationBLEUArtificial intelligenceTranslation (biology)Natural language processingHuman–machine systemMachine learning

Related Publications

Automatic evaluation of summaries using N-gram co-occurrence statistics

Chin-Yew Lin , Eduard Hovy

Following the recent adoption by the machine translation community of automatic evaluation using the BLEU/NIST scoring process, we conduct an in-depth study of a similar idea fo...

2003 1573 citations

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

Mike Lewis , Yinhan Liu , Naman Goyal +5 more

We present BART, a denoising autoencoder for pretraining sequence-to-sequence models. BART is trained by (1) corrupting text with an arbitrary noising function, and (2) learning...

2020 1222 citations

BLEU deconstructed: Designing a Better MT Evaluation Metric

Xingyi Song , Trevor Cohn , Lucia Specia

BLEU is the de facto standard automatic evaluation met-ric in machine translation. While BLEU is undeniably useful, it has a number of limitations. Although it works well for la...

2021 OPAL (Open@LaTrobe) (La Trobe Univers... 27 citations

Cross-lingual Language Model Pretraining

Guillaume Lample , Alexis Conneau

Recent studies have demonstrated the efficiency of generative pretraining for English natural language understanding. In this work, we extend this approach to multiple languages...

2019 arXiv (Cornell University) 1617 citations

Show and tell: A neural image caption generator

Oriol Vinyals , Alexander Toshev , Samy Bengio +1 more

Automatically describing the content of an image is a fundamental problem in artificial intelligence that connects computer vision and natural language processing. In this paper...

2015 6136 citations

Publication Info

Year: 2001
Type: article
Pages: 311-311
Citations: 20359
Access: Closed

External Links

View on DOI.org

Social Impact

Altmetric

BLEU

PlumX Metrics

Social media, news, blog, policy document mentions

Citation Metrics

20359

OpenAlex

Cite This

APA Style

                            
                                    Kishore Papineni, 
                                
                                    Salim Roukos, 
                                
                                    Todd J. Ward
                                
                                et al.
                            
                            (2001). 
                            BLEU. 
                            
                            , 311-311.
                            https://doi.org/10.3115/1073083.1073135

Identifiers

DOI: 10.3115/1073083.1073135