Climate Model Evaluation

Model Evaluation Overview

Under Construction!

The Climate Data Guide project is seeking support to develop a Climate Model Evaluation Guide. We will promote a systematic approach to model evaluation, utilizing state-of-the-art data sets and statistical practices. Constraining numerical models with observations can lead to significant model performance improvements, though the pathway from data-model comparisons to enhanced model skill is often bumpy and rocky; it is not currently traveled in a systematic and fully transparent way across the diverse climate modeling community. We will build a crossroads that facilitates the interaction of experts in climate modeling, observations, and statistics. This new community-focused resource will offer:

1. Access to many additional observational data sets used to evaluate Earth system models, especially satellite data sets. The major strengths and limitations of these data will be discussed by experts in the field.

2. Discussion of both traditional and leading-edge approaches to model evaluation that are scientifically sound and commensurate with the big data era.

3. Concrete examples of the use of key observational data sets to benchmark Earth system models.

4. Quantitative performance metrics for Earth system models.

5. An enhanced web interface that offers new search and discussion tools and integrates with the web services of major data centers, including real-time updates of metadata.

Common Issues in Model Evaluation

In addition to providing guidance and commentary on observational datasets, the Climate Data Guide will include approaches and strategies for model evaluation from experts in the field. This will include statistical methodologies for comparing the model and observations with respect to time-averages, variability, extremes, and trends in various climate parameters. For example, how many years are needed to assess whether the model climate and the observed climate are the same within a given error tolerance? This question may be addressed using standard statistical approaches such as those presented in Wehner (2000: A method to aid in the determination of the sampling size of AGCM ensemble simulations. Clim. Dyn., 16, 321-331) and Taschetto and England (2008: Estimating ensemble size requirements of AGCM simulations. Meteorol. Atmos. Phys., 100, 23-36) and our aim will be to provide additional guidance in the application of these approaches.

In addition to addressing statistical issues, the Climate Data Guide will provide insights and commentary on other issues that arise frequently in the evaluation of models, both generally and targeted towards coupled model archives including the CMIP3 and imminent CMIP5 archives. For example, this section will address issues such as how to best evaluate various model configurations (e.g. AMIP, slab-ocean, fully-coupled) and how to account for the inevitable differences in imposed forcing that exist between models and nature. In addition, issues inherent to model evaluation, such as those that arise in comparing simulated clouds, which can have extent but no water path and therefore are radiatively inactive, with observed clouds, which often are subject to finite detection thresholds and spatial extents that depend uniquely on the observing platform. Similarly, this site will include discussion of the structural contrasts that exist between observations and models, such as for example in the computation of cloud radiative forcing ‒ where models simply remove clouds from the radiative calculation whereas observations rely on opportunities to observe clear-sky conditions and are therefore biased towards certain meteorological regimes. These and similar issues are an inevitable consideration in model evaluation and thus their recognition and discussion will be a valuable component of this site.

Current overview pages in this section may be accessed from the list below:

Cite this page

National Center for Atmospheric Research Staff (Eds). Last modified 20 Apr 2022. "The Climate Data Guide: Model Evaluation Overview." Retrieved from

Acknowledgement of any material taken from this page is appreciated. On behalf of experts who have contributed data, advice, and/or figures, please cite their work as well.