Earlier this year while I was involved in preparing our contribution to an exchange with colleagues for the Bulletin of the American Meteorological Society on hurricanes, a particular sentence in the response (PDF) to our paper piqued my interest. The sentence read in full:

As summarized in the Working Group II assessment of McCarthy et al. (2001), model projections of the mean annual number of people who would be flooded by coastal storm surges increase several-fold (75–200 million, depending on the adaptive response) for midrange scenarios of a 0.4-m sea level rise by the 2080s relative to scenarios with no sea level rise.

The sentence caught my attention because it was stunningly ambiguous and unclear. What could it mean? So I set out to learn more about it. What I found and my experiences trying to publish what I have found provides some insights into the increasingly curious world of climate science.


The sentence is actually a direct quote from the IPCC Working Group II Third Assessment Report’s Summary for Policymakers, p. 13 (here in PDF). As such it has been widely quoted in the years since the Third Assessment. But a review of how it has been interpreted shows considerable confusion about what it actually means. For example:

A peer-reviewed article in the British Medical Journal came to one conclusion:

“The number of people at risk from flooding by coastal storm surges is projected to increase from the current 75 million to 200 million in a scenario of mid-range climate changes, in which a rise in the sea level of 40 cm is envisaged by the 2080s.”

A report by an oil industry group came to a second interpretation (PDF):

“Even a somewhat conservative scenario of a 40-cm [15.8-in.] sea-level rise by the 2080s would add 75 to 200 million people to the number currently at risk of being flooded by coastal storm surges.”

The Red Cross offers still another interpretation,

“The average annual number of people whose houses are flooded by storm surges along coastlines is expected to increase from a few million each year to between 75 and 200 million by the year 2080, estimates the IPCC.”

Such differing perspectives are chracteristic of this particular bit of the IPCC consensus. No one, it seems, knows what this sentence means. Now this might not be surprising; after all, the IPCC is written by committee and it would not be surprising to see a confusing sentence come out of such a process. However, this is where it gets considerably more interesting. It appears that the IPCC did not explain what the sentence means, and going back to the original scientific source for the statement does not allow one to arrive at a clear interpretation of what it means.

Given that the IPCC is frequently touted as the most highly peer-reviewed document in climate science I found this quite interesting, and somewhat troubling. So I wrote up what I had learned, only to find after months of trying that publishing such a story can be quite challenging.

But before discussing my attempts to publish, let me describe the source of the sentence and the challenges in interpreting what, exactly, it means.

The IPCC WGII SPM points to Section 4.5 of the full assessment report as the source for the sentence. So going to that section, one finds absolutely nothing related to the sentence. However, thanks to Google and the online availability of the IPCC the trail does not end there. Searching on various combinations of “sea level rise” and “storm surge” leads to Chapter 7 of the IPCC which has this passage:

Worldwide, depending on the degree of adaptive response, the number of people at risk from annual flooding as a result of a 40-cm sea-level rise and population increase in the coastal zone is expected to increase from today’s level of 10 million to 22-29 million by the 2020s, 50-80 million by 2050s, and 88-241 million by the 2080s (Nicholls et al., 1999). Without sea-level rise, the numbers were projected at 22-23 million in the 2020s, 2732 million in the 2050s, and 13-36 million in the 2080s. The 40 cm sea-level rise is consistent with the middle of the range currently being projected for 2100 by Working Group I. In 2050, more than 70% (90% by the 2080s) of people in settlements that potentially would be flooded by sea-level rise are likely to be located in a few regions: west Africa, east Africa, the southern Mediterranean, south Asia, and southeast Asia. In terms of relative increase, however, some of the biggest impacts are in the small island states (Nicholls et al., 1999).

Nowhere in this passage is it clear however where the SPM got the 75-200 million number, or what it might mean.

Nicholls et al. 1999 is this paper:

Nicholls J.N., F.M.J. Hoozemans and M. Marchand, 1999. Increasing flood risk and wetland losses due to global sea-level rise: regional and global analyses, Global Environmental Change, 9:S69-S87. (PDF)

And after several careful readings it turns out that all of the questions lead to Table 7 in that paper which I reproduce below.

table7.jpg

My first reaction was that the IPCC confused the cumulative people affected by a 1 in 1000 year storm surge event (People to Respond or PTR in the Table 7) with the “mean annual number of people who would be flooded by coastal storm” (AAPF in the Table 7) because these numbers — 70-205 million — look pretty close to those in the SPM (75-200 million) and it would perhaps be easy to confuse the last digits. If so then this would be a huge error, conflating a cumulative number with a cumulative number.

So I emailed Robert Nicholls who graciously responded suggesting that there was another possible interpretation that got one reasonably close to the IPCC numbers. That would be taking the Average Annual People flooded under “Evolving Protection” for the 2080s – 88 million – and subtracting the 13 million for the 2080s under evolving protection with no sea level rise, leaving one with 75 million. Very promising.

However, there is no similar calculation that leads to 200 million. The analogous calculation under the “Constant Protection” column is 228-36 = 192 million. One could confuse the different models to arrive at 237(+4)-36 = 205, but this would be an obvious methodological error (comparing across different models) compounded with a typo. And if not a typo why alter the value from 205 to 200? I can’t figure it out.

So where exactly the numbers came from remains a mystery. No one I’ve asked knows the answer. It makes no sense to me that the numbers would be generated through some procedure outside the peer-reviewed literature, as this is outside the scope of the IPCC’s mandate. And there appears to be no logical or intuitive combination of values that result in the values that appear in the SPM from Nicholls et al. Of course, the IPCC is a sprawling document, and the derivation of these numbers may yet be hidden somewhere inside, but I haven’t found them. Maybe there is an obvious or straightforward explanation, but I have not heard it.

OK, so what? The IPCC has a sentence that is confusing at best, and more likely is just nonsense. Nonsense happens, right?

Well, no. When I first searched for information about it last February, I found that the sentence in question has been used in official documents related to climate policy by the Japanese (PDF), Canadian, and British governments. From a follow up search that I just conducted it seems that the UK citation has been superceded, but I am sure that it remains in cyberspace somewhere.

The IPCC SPM is used to justify policy actions on climate change, and thus it would seem pretty important that its information be accurate and understandable. In this case, it appears that neither criterion was met.

So I thought – naively — that this was a pretty interesting story, particularly given that the IPCC is presently in the midst of preparing its fourth assessment, and wrote a short essay describing what I had found – and more importantly had not found — about the SPM sentence and the main report, when compared to the original literature. I have tried to publish at two outlets that one would think might have an interest in the accuracy of IPCC assessments, only to be turned down, quickly in one case because it didn’t fit a category, and after an extremely long delay in the second case with the chief editor explaining to me, essentially, that the emperor’s clothes are really quite beautiful. The case of storm surge and sea level rise in the IPCC WGII SPM seems to be so obviously an open and shut case – the facts are readily available for anyone to examine for themselves – that I admit some considerable surprise at the difficultly in getting it published. But I suppose that is what blogs are for.

Here is how I opened my essay that I wrote last February:

In the children’s game of “telephone” a group of children sit in a circle and someone starts the game by whispering a phrase to the person seated neat to them and so on all around the circle. After enough transmissions a phrase that begins as “five stories” might come out the end as “jive turkey” to everyone’s delight. The game of telephone provides a cautionary tale for producers and users of scientific assessments. The case of mistaken and misinterpreted information about storm surge impacts illustrates the Intergovernmental Panel on Climate Change (IPCC) suggests that the IPCC may have its own “jive turkey” problem minus the delight.

I justified why this issue is important as follows:

The IPCC has great potential to inform policy makers, however its credibility rest on being accurate and faithful to the literature. Errors will inevitably occur, but in this case an error in the IPCC’s most important summaries has been used uncritically in policy documents and academic studies, apparently not having been noticed until now. This problem may be more systemic, e.g., I have elsewhere written about the IPCC’s erroneous treatment of non-peer reviewed studies of the attribution of climate disasters to greenhouse gases (e.g., Pielke, 2005). However broad the problem is, as the IPCC gears up for its fourth assessment report it seems critical to carefully evaluate its procedures for accuracy, and for users of the IPCC to understand the strengths and limits of assessments. There is a more fundamental problem and that is the distilling of complex, nuanced research into one-sentence sound bites that perhaps inevitably cannot accurately capture what is to be found in a lengthy scientific article.

I sought to provide a more accurate summary of Nicholls et al. 1999 than one can find in the IPCC TAR, which in fact leads to conclusions at odds with that distilled by those who have misinterpreted the IPCC sentence. In fact, Nicholls et al. proivide considerable support for the argument tha increasing coasal habitation is a far greater issue than sea level rise when it comes to storm surge (surprise, surprise):

A more accurate summary of Nicholls et al. (1999) would start by observing that the paper used a scenario that had that number of people subjected to 1 in 1000 or greater risk of coastal storm surges increasing from 197 million in 1990 to 575 million in the 2080s without any rise in sea level, due only to population growth and demographic changes. Under this scenarios, with sea level rise (and no societal reaction) the 2085 population-at-risk would be 630 to 640 million. With no sea level rise Nicholls et al. (1999) calculate an increase in the average annual number of people flooded rising from 10 million (1990) to 36 million (2085) with 1990 levels of protection. With protection that evolves as a function of GDP (but not otherwise changed due to sea level rise) the number of people flooded increases only by 3 million, from 10 million (1990) to 13 million (2085), even though population-at-risk increased by 378 million people. This suggests that only 0.8% of the additional people who inhabit the coastal zone by 2085 with a 1 in 1000 or greater risk would on average experience an annual flood. This compares with 5.1% of inhabitants who experience a flood in 1990 based on the assumptions of Nicholls et al.

Putting this together, according to Nicholls et al. (1999) population-at-risk increases by 438 million by 2085 while the average annual number of people flooded increases by 78 million people, from 10 million in 1990 to 88 million in 2085 (and by 3 million, from 10 million to 13 million under the no sea level rise scenario). So the IPCC should have reported that:

Nicholls et al. estimate that of the 438 million additional people expected to inhabit the coastal zone by 2085 with a 1 in 1000 or greater rise of storm surge, about 0.7% – 18% of them, or on average between 3 million and 78 million additional people annually would experience the effects of coast flooding under the assumptions of the sensitivity analysis.

The difference between assuming that only 0.8% of the additional people who inhabit the coastal zone at risk to flooding under a scenario of no sea level rise versus 18% under a scenario of sea level rise results from the fact that the methods assumes an appropriate adaptive response in the first case but not the latter. For instance, to reach the 18% portion requires than many people would move to locations far riskier than they choose to inhabit today. Using the same ratio of people who experience floods to those at risk (i.e., assuming that people adapt where they live according to the contemporaneous risk) as Nicholls et al. use for 1990 (i.e., 5.1%) would suggest an additional 22.3 million people would be exposed to floods with a population increase of 438 million. Using the ratio suggested for 2085 under no sea level rise results in an additional 3.5 million people who experience floods. Nicholls et al. provide no reason to expect that society will be significantly more risk averse in 2085 than today, nor why the presence of sea level rise would change the acceptable risk from a scenario of no sea level rise.

I concluded with some words of caution:

These details for just this one study of storm surge are very complicated and to understand them requires a careful examination of the primary literature. Clearly, what appears in the 2001 IPCC bears only a distant relation to what the original study actually said. For this reason, scientific assessments cannot replace the primary literature, and some thought should be given by scholars to how best to deal with knowledge that is highly simplified through assessment and then recirculated into academic inquiry. Like the children’s game of telephone, this is a recipe for miscommunication, mischaracterization of scientific research, and a foundation of knowledge that rests a few feet above the ground. A more appropriate role for assessments would be to focus more explicitly on the information needs of policy makers and focus attention on a wide range policy options and their possible consequences. Efforts to summarize complex science may not ultimately prove useful to policymakers if they result in oversimplifications and mischaracterizations.

Climate science is a curious business, that’s for sure.