Expert User Guidance

The following was contributed by John Fasullo, NCAR, NOV, 2011:

#The use of 'n-th generation reanalysis' to classify different reanalyses [which you will see on the pages linked to in the table below] is based on the incremental advancement of reanalysis techniques. While uniform distinctions between groups of reanalyses are somewhat artificial, the introduction of major advancements in the reanalysis methodologies over time and the opportunity to learn from earlier reanalysis efforts are clear. In this context, our description of reanalysis efforts as 1st-generation (such as the initial reanalysis efforts of NCEP-NCAR and NCEP-DOE) and 2nd-generation (ERA40,JRA25) correspond, in part, to their evolving abilities to assimilate the full suite of available satellite radiances. For example, the NCEP-NCAR and NCEP-DOE reanalyses do not assimilate satellite water vapor channels over ocean while follow-on efforts do. The 3rd-generation efforts, such as ERAI, MERRA, CFSR, used more sophisticted data assimilation approaches (e.g. analysis increments and 4D-Var) and models and, again, addressed issues discovered in the 2nd-generation reanalysis efforts. Nonetheless, the artificial nature of these distinctions is evident in CFSR, which in many ways can be viewed as a 1st generation effort as it is the first reanalysis to include an interactive ocean in its reanalysis forecasts. Other reanalyses, such as the 20th Century reanalysis, further defy this simple classification as in attempting to extend its record back in time to the late 19th century, it intentionally excludes much of the modern observational record.##

The following was contributed by Dennis Shea, NCAR, NOV, 2011:

#Generally, users are advised to use third generation reanalysis products (ERA-Interim, MERRA, CFSR). To illustrate "use with appropriate care", the included figure of evaporation minus precipitation (Trenberth et al., 2011 J Clim v24 p4907), shows agreement over much of the globe. However, over Africa the derived estimates are of different sign!

Intercomparisons between similar generation reanalysis products may yield estimates of the uncertainty of some variables and derived quantities. Additionally, reanalyses can serve as focal point for model intercomparisons.##

The following was contributed by Dick Dee, ECMWF, March, 2012 (full contribution on the ERA-Interim page):

# Reanalysis data sets in general

Key strengths:

• The data are multivariate, spatially and temporally complete, and gridded
• The data combine information from many sources (observations and models)
• The data set is physically and dynamically coherent, according to the models used

Key weaknesses:

• Changes in the observing system can cause changes in mean errors
• Mixing observations with models tends to violate conservation properties
• Uncertainties in the reanalysis data are difficult to understand and quantify 

...

Reanalysis data are often used to represent the "true state of the atmosphere according to observations." In actual fact, reanalysis combines inaccurate and incomplete observations with imperfect models, using methods and procedures that are technically and scientifically complex. Limitations and caveats of reanalysis data mainly result from:

• Lack of observations. The atmosphere is not now, nor ever has been, fully observed.
• Errors in the observations, and lack of information about those errors.
• Shortcomings in the assimilating model, and lack of information about model errors.
• Shortcomings in data assimilation methodology.
• Technical errors and mistakes.
• Computational limitations (e.g. limitations in spatial and temporal resolution)

Several of these items have to do with a lack of information. They represent fundamental limitations that are not restricted to reanalysis but play a role in any observational data set. (Note: replacing a skillfull forecast model by straightforward spatial interpolation does not solve anything - it is tantamount to removing, not adding, information).

To assess uncertainties in specific variables produced by reanalysis requires answering the following questions:

• How strongly is the variable constrained by observations? Is it directly or indirectly observed?
• What is the spatial and temporal distribution of the observations? How does this change in time?
• How accurately can the model represent the variable? Does the model have skill in extrapolating and/or predicting it?

Users interested in the quality of low-frequency variability and/or trend estimates need to consider these aspects throughout the time period in question. Temporal variation in the observational constraint can produce artificial shifts in the reanalysis time series, especially if the assimilating model has systematic errors. See Section 8 in Dee and Uppala (2008) for a stratospheric example of this problem.

Given the continuous changes in the observing system, and the fact that all models have some systematic errors, users should be cautious when using reanalysis data for climate studies. It is necessary (but not always possible) to verify trend estimates by comparing with independent data sets, e.g. as in Simmons et al (2010).

Most users do not have access to the information needed to answer the difficult questions listed above. On the other hand, producers of reanalysis data do not have the resources (nor the application-specific knowledge) to answer them either. The challenge is to provide better tools and information to support users in making their own uncertainty assessments. In particular, it should be made much easier for a user to get detailed information about the observations used in reanalysis, including the quality assessment and bias adjustments produced by the reanalysis process itself.##