January 9, 2008
Forecast Verification for Climate Science, Part 3
By popular demand, here is a graph showing the two main analyses of global temperatures from satellite, from RSS and UAH, as well as the two main analyses of global temperatures from the surface record, UKMET and NASA, plotted with the temperature predictions reported in IPCC AR4, as described in Part 1 of this series.

Some things to note:
1) I have not graphed observational uncertainties, but I’d guess that they are about +/-0.05 (and someone please correct me if this is wildly off), and their inclusion would not alter the discussion here.
2) A feast for cherrypickers. One can arrive at whatever conclusion one wants with respect to the IPCC predictions. Want the temperature record to be consistent with IPCC? OK, then you like NASA. How about inconsistent? Well, then you are a fan of RSS. On the fence? Well, UAH and UKMET serve that purpose pretty well.
3) Something fishy is going on. The IPCC and CCSP recently argued that the surface and satellite records are reconciled. This might be the case from the standpoint of long-term liner trends. But the data here suggest that there is some work left to do. The UAH and NASA curves are remarkably consistent. But RSS dramatically contradicts both. UKMET shows 2007 as the coolest year since 2001, whereas NASA has 2007 as the second warmest. In particular estimates for 2007 seem to diverge in unique ways. It’d be nice to see the scientific community explain all of this.
4) All show continued warming since 2000!
5) From the standpoint of forecast verification, which is where all of this began, the climate community really needs to construct a verification dataset for global temperature and other variables that will be (a) the focus of predictions, and (b) the ground truth against which those predictions will be verified.
Absent an ability to rigorously evaluate forecasts, in the presence of multiple valid approaches to observational data we run the risk of engaging in all sorts of cognitive traps — such as availability bias and confirmation bias. So here is a plea to the climate community: when you say that you are predicting something like global temperature or sea ice extent or hurricanes — tell us is specific detail what those variables are, who is measuring them, and where to look in the future to verify the predictions. If weather forecasters, stock brokers, and gamblers can do it, then you can too.