Analysis Tools and Methods

Overview: Climate Data Processing

The following is by Dennis Shea (NCAR):

The first rule of data processing is to look at your data!

What data format is being used? What variables are available and what are the units? What type of grid: rectilinear, curvlinear or unstructured? What is the orientation of the grid: north-to-south or south-to-north? What longitude order? What temporal and vertical coordinate(s) are used? Are there missing values present? (If so, how are they identified?) Does the file contents adhere to any convention? Do not make the asssumption that the data are without problems.  Unfortunately, processing climate data can be rather intimidating due the large sizes of the data sets. Some climate model components (eg, ocean) can produce individual monthly mean files that exceed 17GB while some observationally based reanalysis datasets exceed several tera bytes . Since climate, by definition, is the statistics of one or more variables over a period of time, it often means that there are many files to be processed. Further, some climate projection scenarios may span 50-100 years into the future and there may be multiple model runs (ensembles) to assess model sensitivity. Assorted data formats and a variety of processing tools may be used by different groups to archive and process the data.

Experts contributing to this page:

Cite this page

Shea, Dennis & National Center for Atmospheric Research Staff (Eds). Last modified 28 Aug 2013. "The Climate Data Guide: Overview: Climate Data Processing." Retrieved from https://climatedataguide.ucar.edu/climate-data-tools-and-analysis/overview-climate-data-processing.

Acknowledgement of any material taken from this page is appreciated. On behalf of experts who have contributed data, advice, and/or figures, please cite their work as well.