HadIOD: Met Office-Hadley Centre Integrated Ocean Database
HadIOD (Met Office Hadley Centre Integrated Ocean Database) is a global database of in situ ocean temperature and salinity observations, spanning 1850-present on a daily timestep. It is somewhat unique among temperature and salinity datasets in that it combines observations from both surface and sub-surface observing platforms. As such, it is well suited to activities aiming to assimilate climate-quality observations, such as ocean reanalyses. Moreover, the observations are supplemented by metadata that include quality flags, bias corrections, and estimates of measurement uncertainty. HadIOD is updated regularly. However, HadIOD does not fully probe structural error, so users may wish to incorporate additional datasets into their analyses (one option is CORA (Coriolis Ocean database for Reanalysis)). HadIOD is not an interpolated product, and the density of observations varies considerably in space and time.
Key Strengths
Brings together temperature and salinity observations from the surface and subsurface oceans into a single product
Long record (1850 to present) with regular updates
Provides observation bias corrections, quantified uncertainties and quality flags
Key Limitations
Data coverage can be sparse; in general, the oceans are better sampled in space (horizontally) and depth as time progresses
Less metadata provided than for some observation collections (by design), observation specialists may find other datasets more suitable for observation study
Structural uncertainties (the fundamental choices made during dataset construction) are not fully explored; users should consider using other datasets as well.
When publishing work using these data, please quote the product version number, the date the data were downloaded, and cite the folling paper:
Atkinson, C. P., N. A. Rayner, J. J. Kennedy and S. A. Good, 2014. An Integrated Database of Ocean Temperature and Salinity Observations, Journal of Geophysical Research: Oceans, 119, 7139-7163, doi:10.1002/2014JC010053
intended for non commercial use
Expert Developer Guidance
The following was contributed by Chris Atkinson, UK Met Office, September 2020:
HadIOD (Met Office Hadley Centre Integrated Ocean Database) is a database of global historical in situ ocean temperature and salinity observations. It brings together observations made by surface-only observing platforms (like ships and buoys) and observations from sub-surface ocean profiling platforms (like profiling floats and bathythermographs) and supplements these with metadata including quality flags, bias corrections and estimates of measurement uncertainty. ‘Integrated’ refers to the merging of data from sources concerned with different components of the observing system (e.g. the surface or subsurface ocean) into a single data store, and the inclusion of appropriate metadata (such as bias corrections and duplicate flags) to enable greater internal consistency.
Data coverage is global from 1850-present. Monthly updates are made on a best endeavours basis with approximately a few months delay. Data are extracted from the database into CF-compliant NetCDF format for release. The data and a detailed user guide can be obtained from https://www.metoffice.gov.uk/hadobs/hadiod/. Releases are versioned and the current version is HadIOD.1.2.0.0. Published information (describing HadIOD.1.0.0.0) can be found in Atkinson et al. [2014].
The product is targeted at activities wanting to ‘pick up and use’ (ungridded) climate-quality ocean observations e.g. ocean reanalyses. The focus is on providing the essential information needed for these activities in a convenient format. It is important that users of ocean data correct for known biases (which can introduce artificial variability in the data) and explore the sensitivity of their applications to observational uncertainties. By combining this information in a single data set, we allow users to more easily utilize current understanding of observation error in their applications. Less focus is given to providing the comprehensive observation metadata found in some observation collections (e.g. data provenance) and as such the product may be of lesser interest to observation specialists. Note that further metadata and IDs linking the observations to their source data collections are available in our own internal database but not currently in the files we release, users can contact us if these are of interest.
Observation errors are modelled as the true value plus a bias associated with the type of platform/instrument, a bias specific to an individual platform/instrument and a random measurement error. Where possible bias corrections are provided to reduce these biases. Uncertainty estimates are provided for the residual biases in the corrections and for the random error. The extent to which this information can be provided for each observation varies by observation type dependent on current knowledge. With applications such as data assimilation in mind, we try to provide at a minimum some estimate of uncertainty and an overall quality flag for every observation in HadIOD. Platform/instrument type bias corrections are currently provided for temperature observations made by ships and XBT/MBTs. The uncertainty in these bias corrections can have complex structures in space and time. To facilitate exploring this uncertainty, multiple sets of bias corrections are provided that can be used interchangeably.
Although HadIOD estimates observation errors, users will need to consider whether other sources of error are present in their application. For example, sampling errors may arise if an observation does not exactly represent the quantity of interest (e.g. if a point observation is used to represent an area average). Furthermore, although HadIOD provides estimates of observational uncertainty, these do not capture the structural uncertainty arising from the fundamental choices made in the dataset construction process (e.g. observation error model used, bias correction schemes used, handling of missing metadata, quality control applied, data collections included). Consequently, the uncertainty in any analysis based on HadIOD may be underestimated and non-climatic features may remain in the data. Users should consider also using alternative datasets in their analysis to gain a broader understanding of the uncertainties, for example a user might consider blending HadIOD surface data with a different source of profile data or an alternative source of historical SST bias corrections. Further exploring structural uncertainty is something we may consider for future versions of HadIOD.
It is emphasised that HadIOD brings together observations and metadata from a variety of sources and as such relies on the work of many other people. The sources of observations in the HadIOD.1.2.0.0 public release are: ICOADS.2.5.1 [Woodruff et al., 2011], EN4 [Good et al., 2013] and CMEMS [http://marine.copernicus.eu/] drifting buoy data. EN4 in turn draws on sources including the World Ocean Database [Boyer et al., 2013; http://www.nodc.noaa.gov/OC5/WOD13/], Global Temperature and Salinity Profile Project [http://www.nodc.noaa.gov/GTSPP/] and the Argo project [http://www.argo.net].
Several alternative observation datasets to HadIOD are available, including those used by HadIOD. The objectives and features of each dataset vary. HadIOD is intended to complement existing data collections and its key strengths and limitations are given below. A possible alternative to the HadIOD dataset is CORA [http://www.coriolis.eu.org/Data-Products/Products/CORA] which also provides surface and subsurface temperature and salinity observations in a single dataset and is aimed at similar users.
Key Strengths:
- Blending of temperature and salinity observations from the surface and subsurface ocean in a single product.
- Long record (1850 to present) with regular updates.
- Observation bias corrections, uncertainty estimates and quality flags.
- Data release targeted at users wanting to just ‘pick up and use’ climate-quality ocean observations appropriately.
- Detailed user guide (including guidance on working with bias corrections and measurement uncertainties).
Key Limitations:
- Data coverage can be sparse; in general, the oceans are better sampled in space (horizontally) and depth as time progresses (summary plots are available in the user guide).
- Improved or new versions of some underpinning data/metadata sources are now available (e.g. ICOADS 3.0, latest ship/XBT/MBT bias corrections). Surface salinity content could be improved by adding other sources. These will be included in future releases.
- Less metadata provided than for some observation collections (by design), observation specialists may find other datasets more suitable for observation study (further metadata and IDs linking the observations to their source collections are available in our database but not in the files we currently release, users can contact us if this is of interest).
- Structural uncertainties (the fundamental choices made during dataset construction) are not fully explored, users should consider using other datasets as well.
Key Figures
Fig 1: Maps showing maximum observed depths (meters) of temperature observations in HadIOD for April 1905, 1935, 1965, 1985, 1995 and 2010. Only observations passing quality control and duplicate checks are shown. For ocean profile observations (bold, coloured crosses), maximum observed depth is the depth of the deepest measurement in a profile. For surface-only observations (fine, dark-red crosses), maximum observed depth is simply the (near-surface) depth of the measurement. These plots are from Atkinson et al. [2014] for HadIOD.1.0.0.0, but HadIOD.1.2.0.0 will be similar; the intention is to broadly illustrate changing observation coverage over time, see Atkinson et al. [2014] for further description. (contributed by C. Atkinson)
Fig 2: Maps showing maximum observed depths (meters) of salinity observations in HadIOD for April 1935, 1965, 1985, 1995, 2005 and 2010. Only observations that pass quality control checks are shown. Coloured crosses correspond to the depth of the deepest measurement in a profile. At present no salinity observations are provided by surface-only platforms in HadIOD.1.2.0.0. These plots are from Atkinson et al. [2014] for HadIOD.1.0.0.0, but HadIOD.1.2.0.0 will be similar; the intention is to broadly illustrate changing observation coverage over time, see Atkinson et al. [2014] for further description. (contributed by C. Atkinson)
Fig 3: Monthly numbers of (top) temperature and (bottom) salinity observations in HadIOD.1.2.0.0 files (right column shows January 2000 onwards only) passing quality control and duplicate checks. Observations are shown grouped into categories based on the ICOADS [Woodruff et al., 2011] platform ID scheme. “Ship” includes data from ocean station vessels; “Ocean Station Data” (OSD) includes low vertical resolution data from both bottle and Conductivity Temperature Depth (CTD) profiles; MBT=Mechanical BathyThermograph data; XBT=eXpendable BathyThermograph data; “Profiling Float” is chiefly data from the Argo programme; “CTD” contains high vertical resolution profile data from CTDs and expendable CTDs; “Other” comprises various platform types contributing a smaller number of observations e.g. U.S. Coastal-Marine Automated Network, ocean drilling rigs/platforms, undulating oceanographic recorders and gliders; “Missing” are observations for which platform type metadata is absent. (contributed by C. Atkinson)
Fig 4: Monthly numbers of SST observations from ships in HadIOD.1.2.0.0 for different measurement methods following metadata recovery. Buckets are shown in shaded dark grey, engine room intakes in shaded light grey, and unknowns in white (note measurements were all from buckets prior to 1900), based on Figure 1 of Kennedy et al. [2011]. (contributed by C. Atkinson)
Fig 5: Monthly globally averaged SST platform type bias in ship observations in HadIOD.1.2.0.0 (°C) (in the data files we provide bias corrections that remove these biases). 100 realisations of HadSST.3.1.1.0 bucket (light blue) and engine room intake (light red) biases are shown, with the ‘mean ensemble members’ used for allocating biases in the HadIOD.1.2.0.0 ‘main’ files overlaid (dark blue and dark red lines). Platform type biases are allocated to individual ship observations based on their inferred measurement method (Figure 4). Monthly averages of the ship platform type biases (black) and ship platform type bias uncertainties (grey, denoting +/- 2 std.dev.) in the HadIOD.1.2.0.0 main files are shown. The 100 realisations are available in supplementary files and can be used to explore uncertainty in the ship platform type bias, noting that the ‘mean ensemble member’ used in the main files is simply semi-central to the ensemble and is not a best estimate. (contributed by C. Atkinson)
Other Information
surface based data including from ships & buoys; sub-surface data including from floats, bathythermographs and CTDs