Reference
Ho, C.K., Hawkins, E., Shaffrey, L., Brocker, J., Hermanson, L., Murphy, J.M., Smith, D.M. and Eade, R. 2013. Examining reliability of seasonal to decadal sea surface temperature forecasts: The role of ensemble dispersion. Geophysical Research Letters 40: 5770-5775.
Background
The eight researchers write that "since skillful decadal climate forecasts could bring benefits to climate change adaptation planning, there has been significant development of such predictions in recent years, using global climate models (GCMs) initialized with atmospheric and oceanic observations." However, they say that "previous assessments of the quality of forecasts from ensemble decadal prediction systems have almost always focused on the accuracy of ensemble mean forecasts," noting that "a useful ensemble prediction system should also give reliable forecasts, which means that the forecast probabilities match the observed relative frequencies."
What was done
With these thoughts in mind, Ho et al. evaluated "the dispersion characteristics, a necessary condition for ensemble reliability, of SST [sea surface temperature] forecasts from the UK Met Office Decadal Prediction System (DePreSys)" by examining "how the dispersion characteristics vary spatially with forecast lead time from seasonal to decadal timescales." And in doing so, they made three important discoveries.
What was learned
(1) "Dispersion characteristics of decadal prediction ensembles for SSTs vary considerably, both spatially and with forecast lead time." (2) "For lead times of less than two years, the initialized ensembles tend to be under-dispersed and give over-confident and, hence, unreliable forecasts, especially in the tropics, consistent with many previous studies on this timescale." (3) "For longer lead times, up to 9 years, the ensembles become over-dispersed in most regions and thus give under-confident and also unreliable forecasts."
What it means
The team of UK scientists says their findings "highlight the need to carefully evaluate simulated variability in seasonal and decadal prediction systems." And it might also be good if the creators of the models they studied would actually do something about the inadequacies that Ho et al. discovered in them.