Background
"In the past," in the words of the author, "climate model verification primarily focused on the representation of climatological means." But "on the other hand," as he continues, "a good representation of second-order moments (i.e., variability) on different time scales (e.g., daily, month-to-month or interannual, etc.) is crucial and probably provides an even better test as to whether [real-world] physical processes are well represented in the models."

What was done
Working with 20th-century climate model runs prepared within the context of the IPCC AR4 assessment (now called the CMIP3 data set), Scherrer compared model simulations of the interannual variability (IAV) of 2-m-height air temperature (T), sea level pressure (SLP) and precipitation (P) over the 20th-century with observational and reanalysis data sets for the same time period using standard deviation-based variability indices.

What was learned
The Swiss scientist describes a number of problems he encountered with the CMIP3 models in his examination of them. With respect to SLP, the situation was pretty good, as he writes that "only minor IAV problems are found." With respect to temperature, however, he writes that differences between observations and models are, in general, "larger than those for SLP." And for precipitation he says that "IAV is 'all over the place' and no clear relations with T and SLP IAV problems can be established."

Concentrating thereafter mostly on temperature, Scherrer notes that "a few models represent T IAV much worse than others and create spurious relations of IAV representation and the climate change signal." Among the "better" IAV models, he also finds that "the 'good' IAV models in the tropics are in general not also the 'good' IAV models in the extra-tropics," and that "the 'good' IAV models over the sea are in general not the 'good' IAV models over land," while noting that "similar results are found for the relation between T IAV representation and the amplitude of projected changes in temperature."

What it means
"In general," as Scherrer writes in the final paragraph of his paper, "it is concluded that, aggregated over very large regions, hardly any robust relations exist between the models' ability to correctly represent IAV and the projected temperature change." And he says that these results represent "a plea to remove the 'obviously wrong' models (e.g., like those that have sea ice extending to below 50°N in the Atlantic and DJF temperature biases of ~40°C in Iceland, cf. Raisanen, 2007) before doing climate analyses." Even then, however, the CMIP3 models would likely still have problems with their temperature projections, and they would probably continue to produce precipitation projections that would be "all over the place."

Reference
Raisanen, J. 2007. How reliable are climate models? Tellus 59A: 2-29.