Abstract
ROUGE stands for Recall-Oriented Understudy for Gisting Evaluation. It includes measures to automatically determine the quality of a summary by comparing it to other (ideal) summaries created by humans. The measures count the number of overlapping units such as n-gram, word sequences, and word pairs between the computer-generated summary to be evaluated and the ideal summaries created by humans. This paper introduces four different ROUGE measures: ROUGE-N, ROUGE-L, ROUGE-W, and ROUGE-S included in the ROUGE summarization evaluation package and their evaluations. Three of them have been used in the Document Understanding Conference (DUC) 2004, a large-scale summarization evaluation sponsored by NIST. 1
Keywords
ROUGEAutomatic summarizationNISTComputer scienceNatural language processingIdeal (ethics)Word (group theory)Artificial intelligenceInformation retrievalMathematics
Related Publications
Looking for a Few Good Metrics: ROUGE and its Evaluation
ROUGE stands for Recall-Oriented Understudy for Gisting Evaluation. It includes measures to automatically determine the quality of a summary by comparing it to other (ideal) sum...
Automatic evaluation of summaries using N-gram co-occurrence statistics
Following the recent adoption by the machine translation community of automatic evaluation using the BLEU/NIST scoring process, we conduct an in-depth study of a similar idea fo...
Evaluation of Text Summarization in a Cross-lingual Information Retrieval Framework
We report on research in multi-document summarization and on evaluation of summarization in the framework of cross-lingual information retrieval. This work was carried out durin...
A Scalable Hierarchical Distributed Language Model
Neural probabilistic language models (NPLMs) have been shown to be competitive with and occasionally superior to the widely-used n-gram language models. The main drawback of NPL...
A Deep Reinforced Model for Abstractive Summarization
Attentional, RNN-based encoder-decoder models for abstractive summarization have achieved good performance on short input and output sequences. For longer documents and summarie...
Publication Info
- Year
- 2004
- Type
- article
- Pages
- 74-81
- Citations
- 8287
- Access
- Closed
External Links
Citation Metrics
8287
OpenAlex
Cite This
Chin-Yew Lin
(2004).
ROUGE: A Package for Automatic Evaluation of Summaries.
, 74-81.