Disaster losses have increased dramatically in recent decades. Yet as discussed here frequently there is no scientific evidence showing that any part of this increase can be attributed to changes in climate, whether anthropogenic in origin or not. This is a long post on this subject. It contains a lot of gory detail on what I consider to be a major misuse of science in the climate debate, viewed through the lens of a recent paper in Science. I focus on this issue mainly because this is an area where I have considerable expertise, and in this context my work is often mis-cited or ignored. This misuse of science is pretty much overlooked by scientists (here is one exception) advocates on either side of the debate, and the media (here is one exception). A number of colleagues and I have a letter on this subject coming out in the November Bulletin of the American Meteorological Society (I’ll post a pre-publication version of this soon). Also, in partnership with Munich Re we are organizing a major workshop on attribution of causes underlying the observed trend of ever-escalating disaster damages. Munich Re seems very supportive of rigorous science on this topic. So clearly, I intend to pursue this subject.

Some important things to say before proceeding — As I have written often on these pages, I accept the IPCC WGI consensus position on climate change and I am a strong advocate for policy action on climate change. I am also quite concerned by the role of science and scientists in the highly politicized context of climate.

I have titled this post “The Other Hockey Stick” drawing on some comments made by Hans von Storch in a talk at NCAR last month. The “other hockey stick” refers to the graph used by the IPCC based on Munich re data to show increasing disaster costs and has been widely used to argue for evidence of a climate change signal in disasters. Such claims are made by prominent scientists (such as Rajendra Pachauri and John Houghton) and can be found frequently in the scientific literature. The motivation for the present discussion is a paper in the 12 August 2005 issue of Science. Evan Mills, a scientist at Lawrence Livermore National Laboratory. Wrote in the essay,


“According to the latest Intergovernmental Panel on Climate Change (IPCC) assessment, climate change has played a role in the rising costs of natural disasters.” And on the “relative weights of anthropogenic climate change and increased exposure” in the loss trend Mills concludes “quantification is premature.” Mills uncritically accepts the IPCC statement, which as I show below is based on a pretty weak source and he is simply wrong on the latter point. Mills either ignores or is unaware of a robust literature on this subject (see here and here and here). Mills’ analysis rests on a very thin basis of support. For reasons discussed below, it is amazing to me that Mills’ paper survived peer review at Science. It should not have. Whether Science has a quality control problem or an inability to question analyses that may be politically inconvenient, the publication of Mills’ paper sure does raise some questions.

Mills has a section of his paper focused on attribution of causes explaining recent trends in disaster losses. Let’s take a close look at ten of the sources he cites in that section to support claims of a climate change signal in the damage record:

Kunkel et al. 1999. Mills cites this paper, (on which I am a co-author) to support this claim, “Socioeconomic and demographic trends clearly play important-and likely dominant-roles in the observed upward loss trends.” Here is what Kunkel et al. concluded, “the results of the review strongly suggest that the increasing financial losses from weather extremes are primarily due to a variety of societal changes.” Perhaps a slightly different characterization than the paper suggests, but lets move on.

Changnon and Demissie, 1996. This paper says nothing about trends in flood damage. What it does say is that in a comparison of urbanizing and rural river basins, there were large societal influences on streamflow in the urbanizing basins. This paper provides no basis for asserting anything related to flood damages.

Zhang et al. 2005. This paper is about the relationship of coastal erosion and sea level rise. It says nothing about trends in disaster losses.

Easterling et al. 2000. This paper states, “Most of the increase has been due to societal shifts and not to major increases in weather extremes. The growth of population, demographic shifts to more storm-prone locations, and the growth of wealth have collectively made the nation more vulnerable to climate extremes.” This paper originated in a workshop held in Aspen that I participated in and I was originally a co-author on early drafts of this paper. I dropped off because I thought that the paper’s conclusions were not supported by the evidence. In this case, there is an important difference in the cited sentence between the words “Most” and “All”. “Most” happens to be grammatically correct, but in this case is synonymous with “All”. My concern was that the paper would be mis-cited to assert an attribution when none was found. Here is a good example.

Karl and Trenberth 2003. This paper calls for the development of a global observing system. It says nothing about trends in disasters.

Next the paper asserts that “Global weather-related losses in recent years have been trending upward much faster than population, inflation, or insurance penetration, and faster than non-weather-related events (Fig. 2D). By some estimates, losses have increased by a factor of 2, after accounting for these factors plus increased density of insured values.”

It cites 2 references to make this claim. The first reference is to a talk by Howard Kunreuther. I have known Howard for a while and respect him and his work a great deal. I emailed him to ask his source for this claim and interestingly he referred me to the second source cited by Mills. So Mills is citing the same source twice, using two apparently different sources. Not good.

The second source is a 2000 report by Munich Re on catastrophes. The relevant sections of the report can be found at pp. 79-81. Here Munich Re accurately cites my work to correctly argue for the normalization of historical loss data to account for societal changes. Then Munich Re provides some summary data following a black box calculation of changes in disaster damages after normalizing for societal changes. Munich Re finds that global disasters cost an adjusted $636 billion in the 1990s compared with $315 billion in the 1970s, and concludes, ” Mills “factor of 2″ comes from this calculation (i.e., 636/315).

Methodologically the calculation is suspect for a number of reasons. First, Munich Re provides neither their methods nor data. Second, Munich Re admits that data on changes in wealth are not available around the world and changes in GDP are not always a good proxy for data on wealth. Third, Munich Re’s data apparently includes weather and non-weather events (e.g., see figure “d” on page 81, which refers to earthquake damages).

But let’s assume that all of these issues raised above can be overcome and in the end there remains a 2 to 1 ratio. The fact is that the large decadal variability in normalized losses makes it quite dodgy to assert a trend between two different ten-year periods over a period of 30 years. Let me illustrate this with an example from our database of normalized hurricane losses. If we adjust the hurricane loss data to 2004 values and then compare decades we see some interesting things. First the ratio of the 1990s:1970s is quite similar to the Munich Re analysis, 2.1 ($91B/$43B). But if we look at other decadal comparisons, the picture looks quite different, 1990s:1940s = 1.0 ($91B/$90B) and 1990s:1920s = 0.6 ($91B/$154B). Bottom line: The Munich Re analysis tells us nothing about attribution.

The Munich Re analysis may prove correct in the end from the standpoint of disasters in the 1970s compared to the 1990s. But all that it would allows us to say is that the 1990s had more costly disasters than the 1970s, and provides absolutely no basis for attribution of the causes of the differences. At a minimum analyses such as Minuch Re’s should be submitted for peer review in the scientific literature to allow for an open discussion of data and methods.

Back to the papers cited by Mills:

Association of British Insurers, 2004. Mills cites this report as follows, “The Association of British Insurers states that changes in weather could already be driving UK property losses up 2 to 4% per year (7) owing to increasing extreme weather events.”

The executive summary of the AIb report does claim, “Weather risks are already increasing by 2 – 4 % per year on the household and property accounts due to changing weather.” But if you read down just a bit further (on p. 8) the executive summary says something a little different “On reasonable projections of extreme events, the pure risk rate for weather catastrophes is already rising at an unseen rate of 2 – 4 % per year.” It has now raised the issue of projections. And if you take a look at the Technical Annex to the report, you find something different still, “Thus on the basis of the Foresight Programme view of future flood risk, realistically the risk of flood damage is projected to increase by between 2.1% and 3.9% per annum, or a range of two to four percent per year.”

There are two important points here. First, the 2%-4% per year increase in damages is a projection made out over the next 80 years. Second, we discussed the Foresight project quite favorable here last year. The Foresight project was notable because it considered both climate and societal factors in its projections. The 2%-4% number is not based on climate factors alone. Mills’ statement is thus incorrect in two ways – the increase in damage is projected, not observed, and it is the consequence of societal and climate factors, not an observed increase in extreme events.

Mills next cites the IPCC WGII, Chapter 8 to justify the claim “According to the latest Intergovernmental Panel on Climate Change (IPCC) assessment, climate change has played a role in the rising costs of natural disasters.” If you go to the IPCC you see that the single basis for this claim is the Munich Re 2000 report discussed above. This is the third different reference to the same analysis. At best this is sloppy citing. At worst it appears as if there is an attenmpt to portray a broader intellectual base of support for these claims than there actually is.

Lastly in this section, Mills et al. 2002 discuss the relationship of lightning claims and temperature, which makes sense as lightning tends to be associated with thunderstorms and thunderstorms occur in summer not winter. As they state “An additional issue is that peak lightning periods occur in summer, when electricity reliability problems are likely to cause other business interruption losses, as suggested by the illustration.” There is no data here relevant to understanding historical trends in disasters (or insurance claims).

So here is my tally:

3 sources are each traced back to a single non-peer reviewed source, Munich Re 2000, that raises some serious questions of methods and interpretation.

4 papers are cited but are not at all relevant to the issue of disaster losses or attribution.

1 paper (AIB) is mis-cited, which is easy to see if you actually look beyond the first page of its executive summary. (Ironically, the Foresight report which forms the basis for the AIB claims actually makes a good case for the overwhelming dominance of societal factors in future flood losses in the UK.)

1 paper is cited accurately, Easterling et al., but in my opinion this paper plays fast and lose with language to allow a mis-interpretation of its results.

1 paper Kunkel et al. is cited accurately, though I might take issue with the spin, it is probably within the bounds of appropriateness.

Of 10 citations, 9 are highly questionable. And this is Science magazine.

The bottom line is that the issue of attribution of trends in disaster losses rests on the thin reed of a single citation in the IPCC WG2 Chapter 8 to a 2000 Munich Re report that seems to be cited over and over again. Through citing this report several times via different secondary sources, the citing of multiple irrelevant sources and the careful parsing of two papers, Mills comes to the conclusion that climate change is responsible for some part of the observed trend in losses. There is a much, much larger literature on this subject that Mills does not cite.

These are not characteristics that one expects to see in a paper in Science, arguably one of the two most influential publications on science in the world. I have submitted a brief comment in the form of a letter to Science on this paper referencing some of this broader literature. Let’s see what happens.

In the end, scientific research may yet prove that anthropogenic climate change plays a observable role in disaster losses. But today, August 22, 2005, shoddy science, bad peer review and a failure of the science community to demand high standards is not the best recipe for helping science to contribute effectively to policy.