“A colossal mistake” is how Jerry Mahlman describes in an article in the Chronicle of Higher Education the IPCC’s decision to feature the so-called “hockey stick” in its Summary for Policy Makers.

I am somewhat surprised that discussion of the hockey stick continues to be about the he said-he said conflcit between the camps in Real Climate and Climate Audit, as described by the Chronicle. As much fun as the personalities and politics are, at some point it is probably worth discussing the broader significance of the hockey stick debate for how we think about scientific assessments and their contributions to the needs of decision makers.

Along these lines, here are some questions worth asking about the hockey stick experience that are not receiving enough attention:

1. The Chronicle article notes that the author(s) of the hockey stick article were responsible for its inclusion in the IPCC report. What are the issues associated with having people (not just limited to the HS, of course) involved with assessing their own work? This would never fly for, say, journal peer review or the drug approval process. Other experience suggests that science can be misrepresented when people review their own work in assesments for policy makers. What are the alternatives? What are the general lessons for the emapnelment of science assessment bodies?

2. The Chronicle article noted of the IPCC’s presentation of the HS, “caveats faded from view when leaders of the IPCC boiled down the 994-page scientific report into a 20-page synopsis.” Representing complex science in a sound bite necessarily requires simplification, and arguably in this case, and many others, over-simplification of the science. Such over-simplifcations are amplified by the media and used in politically convenient ways by policy advocates on all sides of an issue (as described by the Chronicle, e.g., in the use of the HS by the U.S. National Asssessment and the WSJ). Does it make sense to “boil down” science in a manner that inevitably leads to a mischaracterization of that science? What are the alternatives? This isn’t the only example where communication has suffered in the IPCC due to oversimplification of the relevant science.

In my view, both questions raised above might be addressed, at least in part, if the IPCC (or any assessment) were to ask and aswer “So what for action?” of science findings when bringing them to policy makers. The inclusion of any information in a “summary for policy makers” or a “policy relevant” document is based on an assumption that such information is in fact relevant to those policy makers. Scientists should be explcit about why, exactly, they are including some information and not others and what the criteria of relevance actually is. In the case of the IPCC, criteria of relevance are out of sight, and as far as I know completely arbitrary.