Global near-surface humidity data: HadISDH

The following was submitted by Kate Willett, December, 2024:

HadISDH uses in situ weather station observations from NOAA’s ISD over land (Smith et al., 2011) and ship observations from ICOADS over ocean (Freeman et al. 2017). The product is available as land only (HadISDH.land, ~2m), marine only (HadISDH.marine, ~10m) and a blended product (HadISDH.blend, ~2m over land and ~10m over ocean). These each have been quality controlled at the hourly level (as part of the HadISD processing, Dunn et al., 2016) to remove random errors. The T and Td observations are then converted to all of the other humidity variables. Ship observations are height adjusted to ~10m and also adjusted to account for moist biases in poorly ventilated instruments. Monthly means are calculated and homogenised over land to remove systematic biases caused by things such as station moves and instrument changes. The monthly means and anomalies are then averaged over each 5°x5° gridbox. 

What are the intended research applications of this data set?
HadISDH is intended for monitoring large scale trends and variability in the mean climate state of near-surface humidity. It has been processed to minimise any biases that could introduce spurious long-term trends. Although national and regional differences exist in terms of instrument type and observing practices, the observations have been treated consistently across the globe to minimise regional bias. Given the spatial and temporal gaps in the observations and gridded product we strongly recommend fitting trends to anomalies, not the absolute values. In addition, analyses should incorporate the provided uncertainties which include homogeneity adjustment uncertainty (land obs) and bias adjustment uncertainty (ocean obs).

How is uncertainty characterized in these data?
HadISDH contains 2 sigma (~95% confidence interval) uncertainty estimates for every gridbox month. The uncertainty assessment includes the station/ship level uncertainty (measurement, inhomogeneity/bias adjustment and climatology), as well as the sampling uncertainty arising from estimating a 5°x5° grid box from point observations. Regional average time series are also provided which include an additional coverage uncertainty term to account for missing 5°x5° grid boxes.

What are some comparable data sets, if any?
There are currently no other in situ only near-surface humidity products. Comparisons with ERA5 show strong similarity over land, although ERA5 shows stronger drying/weaker moistening, with notable divergence from 2019 onwards that is not common to other reanalyses. Over oceans, ERA5 specific humidity looks reasonably similar, but with weaker trends than HadISDH. There is large uncertainty in ocean relative humidity in both ERA5 and HadISDH. Overall, both products show decreasing ocean RH but with quite different temporal variability. The decrease along with the decreasing and now very limited ship observation coverage is an issue for HadISDH in estimating these trends. Note that ERA5 does not assimilate the temperature and humidity from ships but is influenced by changing satellite data streams over time.

Known Issues and Cautions:
There are a few things to be aware of when using HadISDH. Over land, there is an increase in station density until the 1990s and then a slight reduction from 2015. To minimise the effects of a changing station network, stations are only included if they have at least 15 years of data with at least one in each decade of the 1991-2020 climatology period. There are then further, stricter tests on station completeness to ensure included stations provide reliable long-term records. Over the ocean, the change in ship observation numbers is a very large concern, decreasing considerably from the 1990s to 2015. Current coverage is now mostly limited to the northern mid-latitudes.

Observing system instrumental changes have occurred over land and ocean. The change from manual wet-bulb thermometers to automated electronic sensors is now almost world-wide, with changes often being made nationally over a short time period. This makes it difficult for the automated homogenisation algorithm to detect them. Over China, this change occurred in the early 2000s and was found to cause a spurious drying in humidity records (Li et al., 2020, Freychet et al., 2020). It is likely that HadISDH hasn’t fully removed this drying inhomogeneity.

Over oceans, a major issue is the pre-1982 moist bias which is most apparent in the relative humidity, and possibly linked to changes in reporting of whole numbers to decimal precision. The limited spatial coverage makes the ocean humidity record more vulnerable to observing system changes than the land data. Its advisable only to fit trends from 1982 onwards. Other major changes are the increase in ship heights over time, which has been adjusted for in HadISDH, although incomplete metadata means that adjustments are approximations in some cases. The ship structure can cause a solar heating bias which can affect the relative humidity. This has not been adjusted for in HadISDH. Comparisons between daytime and nighttime trends show little difference but users should be aware of it. A new paper (Willett and Blannin, in prep) explores these marine humidity data issues in more detail.