Applying a standardised benchmarking framework to evaluate AI methods for precipitation downscaling over Australia

Abstract

Abstract Downscaling techniques are essential for refining coarse-resolution climate projections to scales relevant for local and regional impact assessments, with Artificial Intelligence (AI) emerging as a promising approach for this task. However, a standardised benchmarking framework for evaluating these AI-based downscaling methods has been lacking. This study presents the first evaluation of AI-based downscaling methods using established performance expectations within a standardised benchmarking framework. Three Machine Learning (ML) models, including a Generative Diffusion Model, a Vision Transformer, and a Recurrent Neural Network, are assessed against observational data and compared with 24 simulations by Regional Climate Models (RCMs). The evaluation employs minimum standard metrics focused on four fundamental rainfall characteristics across Australia: total precipitation, spatial distribution, seasonal cycle, and temporal trends. Results show that all three ML models and ten RCMs meet the minimum performance benchmarks, with rankings varying depending on the rainfall characteristic and region assessed. ML models demonstrate comparable performance to RCMs while offering substantial computational advantages. This highlights the potential of ML models to supplement traditional downscaled simulations, thereby enhancing climate projection ensembles and improving uncertainty quantification. Such an approach aligns with recommendations advocating for diverse modelling methodologies in national assessments. By addressing a critical gap through a standardised evaluation framework, this work provides a comprehensive benchmark dataset comprising ML and RCM outputs for Australian precipitation, facilitating the evaluation of emerging AI downscaling approaches and contributing to the standardisation of regional climate modelling practices.

Affiliated Institutions

• Australian National University AU
• The University of Melbourne AU
• Bureau of Meteorology AU
• National Computational Infrastructure AU
• UNSW Sydney AU
• Monash University AU
• TU Wien AT

Related Publications