A

Chen Lin; Timothy A. Miller; Dmitriy Dligach; Steven Bethard; Guergana Savova

doi:10.18653/v1/w19-1908

Abstract

Classic methods for clinical temporal relation extraction focus on relational candidates within a sentence. On the other hand, break-through Bidirectional Encoder Representations from Transformers (BERT) are trained on large quantities of arbitrary spans of contiguous text instead of sentences. In this study, we aim to build a sentence-agnostic framework for the task of CONTAINS temporal relation extraction. We establish a new state-of-the-art result for the task, 0.684F for in-domain (0.055-point improvement) and 0.565F for cross-domain (0.018-point improvement), by fine-tuning BERT and pre-training domain-specific BERT models on sentence-agnostic temporal relation instances with WordPiece-compatible encodings, and augmenting the labeled data with automatically generated “silver” instances.

Keywords

Computer scienceSentenceFocus (optics)Artificial intelligenceEncoderTransformerRelationship extractionTask (project management)Natural language processingRelation (database)Domain (mathematical analysis)Point (geometry)Information extractionData miningMathematics

Related Publications

BioBERT: a pre-trained biomedical language representation model for biomedical text mining

Jinhyuk Lee , Wonjin Yoon , Sungdong Kim +4 more

Abstract Motivation Biomedical text mining is becoming increasingly important as the number of biomedical documents rapidly grows. With the progress in natural language processi...

2019 Bioinformatics 6148 citations

Publicly Available Clinical

Emily Alsentzer , John R. Murphy , William Boag +4 more

Contextual word embedding models such as ELMo and BERT have dramatically improved performance for many natural language processing (NLP) tasks in recent months. However, these m...

2019 Proceedings of the 2nd Clinical Natur... 1422 citations

Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing

裕二池谷 , Robert Tinn , Hao Cheng +6 more

Pretraining large neural language models, such as BERT, has led to impressive gains on many natural language processing (NLP) tasks. However, most pretraining efforts focus on g...

2021 ACM Transactions on Computing for Hea... 1737 citations

Evaluating the Effectiveness of Large Language Models in Representing Textual Descriptions of Geometry and Spatial Relations (Short Paper)

T. B. Brown , Benjamin F. Mann , Nick Ryder +28 more

This research focuses on assessing the ability of large language models (LLMs) in representing geometries and their spatial relations. We utilize LLMs including GPT-2 and BERT t...

2023 Leibniz-Zentrum für Informatik (Schlo... 14006 citations

TinyBERT: Distilling BERT for Natural Language Understanding

Xiaoqi Jiao , Yichun Yin , Lifeng Shang +5 more

Language model pre-training, such as BERT, has significantly improved the performances of many natural language processing tasks. However, pre-trained language models are usuall...

2020 Findings of the Association for Compu... 1491 citations

Publication Info

Year: 2019
Type: article
Citations: 77
Access: Closed

External Links

View on DOI.org

Social Impact

Altmetric

PlumX Metrics

Social media, news, blog, policy document mentions

Citation Metrics

OpenAlex

Cite This

APA Style

                            
                                    Chen Lin, 
                                
                                    Timothy A. Miller, 
                                
                                    Dmitriy Dligach
                                
                                et al.
                            
                            (2019). 
                            A. 
                            
                            .
                            https://doi.org/10.18653/v1/w19-1908

Identifiers

DOI: 10.18653/v1/w19-1908