January 27, 2006
Hypotheses about IPCC and Peer Review
The IPCC is the 800 pound gorilla in the climate debate. It has been the locus of legitimate and credible climate science (salience is another matter, but I digress). It is increasingly coming under criticism in a number of dimensions for some very good reasons. In this post I’d like to suggest a few hypotheses about how the IPCC has indirectly contributed to the politicization of climate science in ways we’ve not discussed here. These are for discussion, and I’d welcome evidence for/against and other sorts of examples.
Laundering Grey Literature
The IPCC has a requirement that its assessments be based on peer reviewed literature. It has not always held itself to this standard, particularly in its Working Groups II and III. I have noticed recently a number of peer-reviewed papers that reference so-called “grey literature” (e.g., agency, company, NGO reports) which hasn’t itself been peer reviewed. Then the peer-reviewed study that cites the grey literature is subsequently cited in another publication to refer to the information in the original non-peer reviewed source. This is a way to give the veneer of peer review to a non-peer-reviewed study. Here is an example of this dynamic.
Fun with Deadlines
The IPCC sets a deadline for when papers must be accepted in order to be considered in a particular assessment report. This guarantees that the assessment won’t have to be continually updated, but it also means that the assessment is automatically out-of-date in some case where new findings have been released. Because editors and journals have considerable discretion in when they publish what papers, the IPCC’s deadline can set the stage for some mischief in the publication process as papers with a particular slant are published before the deadline and other published after. I don’t have any data on this, but it’d be interesting to compare the time-to-publication of key papers cited in IPCC reports with a journal’s standard practices. This issue came to mind as I read this comment from RealClimate,
There are several more papers “in the mill” which we are not at liberty to discuss right now [Ed.- Embargoed, see below], which insure that the weight of peer-reviewed studies available for consideration in the next IPCC report will point towards a strengthening, not a weakening, of the IPCC ‘01 conclusions regarding the anomalous nature of recent hemispheric and global warmth in a long-term context.
Maybe it is just inartful language, but claims to “insure” previously found results do not make me comfortable about the agendas of climate scientists.
Embargoes as Silencers
This one is not about the IPCC, but Science and Nature. I was recently at a science talk at NCAR and a number of leading scientists refused to discuss their work because it would potentially be under “embargo” with Science or Nature, if accepted. My understanding is that embargoes refer to releasing papers accepted for publication to the media in advance of the artificial deadlines set by Science and Nature to generate news-worthiness. They do not apply to scientists talking among themselves in scientific settings. So when scientists use potential embargoes as a way to silence discussion and debate on their work, it reduces the internal vetting of scientific ideas and makes the leading journals the only place where debate can occur. Since Science and Nature are highly selective is what they allow as far as intellectual exchanges following up papers they publish, the entire process of scientific debate and learning is arguably slowed down. Meantime, this allows findings supporting one view or another to gain much greater standing in political debate than they might otherwise have.
Comments? Other observations?