ICOADS Surface Marine Weather Observations

#General Description

The International Comprehensive Atmosphere-Ocean Data Set (ICOADS) is the most extensive and widely used digital collection of quality-controlled surface weather observations available for the world oceans for studies of marine climate and its variability. The data set begins in 1800, and currently extends through 2007 with updates every several years. The ICOADS includes monthly values each year of sea surface temperature (SST), air temperature, wind, cloudiness, barometric pressure, and humidity, as well as derived variables such as turbulent heat and momentum pseudo-fluxes ("pseudo" because they neglect transfer coefficients). The number of observations and the standard deviation of the individual observations that make up each monthly value are also archived for each variable. This important data set forms the basis for our empirical knowledge of surface marine climate and its variability during the past two centuries.

The majority of the observations come from ships-of-opportunity, supplemented in recent years by research vessels, moored environmental buoys, drifting buoys, and near-surface measurements from hydrographic profiles. The data are binned into 2 x 2 latitude-longitude boxes (1x1 summaries are also available since 1960). Each variable is subjected to quality-control procedures to remove outliers and duplicates. The screened values are not corrected for changes in instrumentation, observing practice, ship type, etc; missing grid boxes are not filled in; and no "analysis" of the data is performed (e.g., no spatial or temporal smoothing or interpolation).

Due to the uneven distribution of commercial shipping routes and changes in those routes over time, data coverage is poor in certain regions and periods (see "Coverage Maps" for data distribution by variable by decade). Broadly speaking, the North Atlantic, western South Atlantic and northern Indian Oceans contain the highest density of observations, with reasonable coverage back to about 1870. Data coverage is limited in the North Pacific before about 1946 and in the Tropics before about 1960; the Southern Oceans remain poorly sampled throughout the record.

The lack of spatial and temporal smoothing in the ICOADS archive, along with large uncertainties in individual monthly mean values due to inadequate sampling, makes it difficult to produce comprehensible maps of a particular climatic variable for a specific month and year without additional processing of the data. Aggregating the data over many months and/or years, and use of judicious smoothing and/or interpolation in space, can dramatically enhance the large-scale coherency of anomaly patterns by reducing noise associated with random errors and sampling fluctuations (recall that the standard error of the mean decreases by the square-root of the number of observations: see Trenberth et al., 1992 for further discussion). Smoothing/interpolation procedures may be as simple as running means in the zonal and meridional directions, followed by linear interpolation across a specified number of missing grid boxes (c.f. Deser and Wallace, 1990; Mitchell and Wallace ?? for examples in the Tropical Pacific). More sophisticated procedures such as statistical optimization methods may also be employed. For example, Kaplan (1997, 1999) uses Empirical Orthogonal Function (EOF) analysis as a basis for interpolating across missing grid boxes and smoothing in space and time for SST and SLP (similar procedures are employed for SST in Rayner et al., 1996).

The quality-controlled surface marine climate observations that constitute the ICOADS archive also serve as input to more sophisticated "data sets" which employ statistical and empirical procedures directed at improving data homogeneity and signal-to-noise ratios; for example, filling in missing grid boxes, correcting for changes in instrumentation and observing practice, and smoothing in space and time. These "value-added" data sets are tremendously useful for certain applications (for example, as contourable fields or as model boundary conditions), but it should be remembered that observational analyses of climatic variations are limited ultimately by the quality, quantity and distribution of the original measurements (c.f., Hurrell and Trenberth, 1999). As such, the limited processing of the data contained in the ICOADS archive remains a virtue and serves as "ground truth" for our empirical knowledge of surface marine climate and its variability since the mid 19th century.

Strategies for making optimal use of ICOADS data

Physical consistency among variables

Because ICOADS contains many climatic variables which are measured independently but are physically related, evaluating the data for physical consistency provides a powerful tool for assessing the reliability of climatic signals. This approach has been adopted in many studies for wind and temperature: for example, SLP gradients may be compared with winds following a geostrophic or frictional momemtum balance, and SSTs may be compared against marine air temperatures due to the close coupling between these quantities via the turbulent flux of sensible heat. Relevant studies on wind and SLP comparisons include Ramage (1987), Wright (1988), Harrison (1989), Cardone et al., 1990, Deser and Blackmon (1993) and Ward and Hoskins (1996). Some of these studies report spurious trends in certain regions and time periods: for example, an upward trend in wind speed associated with the change from visual estimation of sea state to measurements by anemometers. Useful studies on SST and air temperature relationships include those of Wright (1986), Trenberth et al. (1992), Kent et al. (1993) and Folland and Parker (1995). These studies document an upward trend in SST associated with the change from bucket to engine-intake temperatures over much of the world oceans. Data sets which attempt to correct for these spurious trends include Bottomley et al. (1990) and da Silva et al. (1994). The former study also advocates the use of nighttime marine air temperatures due to the possible contamination of daytime air temperatures by solar heating of thermistor shields.

Removal of the mean annual cycle before averaging in space and time

If one is interested in climatic signals on time scales longer than the annual cycle, it is best to work with monthly anomalies rather than monthly totals, where monthly anomalies are defined as departures from the long-term mean annual cycle: e.g., the difference between a value for a given month and a particular year from the climatological value for that month based upon a sufficiently long period of record. If monthly totals are needed, the climatological annual cycle may then be added back in to the monthly anomalies. Before defining monthly anomalies, it may be desirable to form smoothed climatological background fields to further reduce noise levels. The reason that it is preferable to work with monthly anomalies rather than monthly totals in a data set such as ICOADS is that missing data may result in severe aliasing of variability associated with the annual cycle into variability on longer time scales (e.g., interannual and beyond). For example, time averages (seasonal, annual, etc.) of monthly totals in which different months are missing in different years will result in spurious signals associated with inadequate sampling of the annual cycle rather than real climatic variability. Similarly, inadequate sampling of spatial gradients due to missing grid boxes (with different boxes missing in different months/years) will alias signals associated with long-term mean spatial gradients into spurious temporal variability. The study by Bottomley et al. (1990) provides a nice illustration of the practical application of some of these procedures to SST data.

Additional comments by variable

 

SST

Trenberth et al. (1992) provide a comprehensive discussion of the sources of errors in estimating monthly mean SSTs from ship data, and provide a detailed analysis of noise levels by month and region. Comparison is also made between ICOADS SSTs and air temperatures, and between SSTs from ICOADS and Bottomley et al. (1990). Much of the discussion of errors and noise levels may be carried over to other variables such as SLP or wind. As far as we know, there are no analogous, comprehensive studies of noise levels in other ICOADS fields, although this is desirable.  

(Clara Deser NCAR, May, 2001) [minor changes to the comments were made to reflect Release 2.5]##