Evaluating the Effectiveness of Large Language Models in Representing Textual Descriptions of Geometry and Spatial Relations (Short Paper)

Abstract

This research focuses on assessing the ability of large language models (LLMs) in representing geometries and their spatial relations. We utilize LLMs including GPT-2 and BERT to encode the well-known text (WKT) format of geometries and then feed their embeddings into classifiers and regressors to evaluate the effectiveness of the LLMs-generated embeddings for geometric attributes. The experiments demonstrate that while the LLMs-generated embeddings can preserve geometry types and capture some spatial relations (up to 73% accuracy), challenges remain in estimating numeric values and retrieving spatially related objects. This research highlights the need for improvement in terms of capturing the nuances and complexities of the underlying geospatial data and integrating domain knowledge to support various GeoAI applications using foundation models.

Keywords

Computer scienceTask (project management)Language modelNatural language processingSentenceArtificial intelligenceWord (group theory)Simple (philosophy)Linguistics

Related Publications

BioBERT: a pre-trained biomedical language representation model for biomedical text mining

Jinhyuk Lee , Wonjin Yoon , Sungdong Kim +4 more

Abstract Motivation Biomedical text mining is becoming increasingly important as the number of biomedical documents rapidly grows. With the progress in natural language processi...

2019 Bioinformatics 6148 citations

ALBERT: A Lite BERT for Self-supervised Learning of Language\n Representations

Zhenzhong Lan , Mingda Chen , Sebastian Goodman +3 more

Increasing model size when pretraining natural language representations often\nresults in improved performance on downstream tasks. However, at some point\nfurther model increas...

2019 arXiv (Cornell University) 4051 citations

Exploring the Limits of Transfer Learning with a Unified Text-to-Text\n Transformer

Colin Raffel , Noam Shazeer , Adam Roberts +6 more

Transfer learning, where a model is first pre-trained on a data-rich task\nbefore being fine-tuned on a downstream task, has emerged as a powerful\ntechnique in natural language...

2019 arXiv (Cornell University) 8299 citations

Finding Structure in Time

Jeffrey L. Elman

Time underlies many interesting human behaviors. Thus, the question of how to represent time in connectionist models is very important. One approach is to represent time implici...

1990 Cognitive Science 10427 citations

The Capacity of Linear Channels with Additive Gaussian Noise

R. K. Mueller , G.J. Foschini

The standard method of computing the mutual information between two stochastic processes with finite energy replaces the processes with their Fourier coefficients. This procedur...

1970 Bell System Technical Journal 22 citations

Publication Info

Year: 2023
Type: preprint
Citations: 14006
Access: Closed

External Links

View on DOI.org

Social Impact

Altmetric

Evaluating the Effectiveness of Large Language Models in Representing Textual Descriptions of Geometry and Spatial Relations (Short Paper)

PlumX Metrics

Social media, news, blog, policy document mentions

Citation Metrics

14006

OpenAlex

Cite This

APA Style

                            
                                    T. B. Brown, 
                                
                                    Benjamin F. Mann, 
                                
                                    Nick Ryder
                                
                                et al.
                            
                            (2023). 
                            Evaluating the Effectiveness of Large Language Models in Representing Textual Descriptions of Geometry and Spatial Relations (Short Paper). 
                            Leibniz-Zentrum für Informatik (Schloss Dagstuhl)
                            
                            .
                            https://doi.org/10.4230/lipics.giscience.2023.43

Identifiers

DOI: 10.4230/lipics.giscience.2023.43

Data Quality

Data completeness: 77%