Greg Holland and Peter Webster have a new paper accepted on the statistics of Atlantic hurricanes. While there are many interesting questions that might be raised about the data and statistics in the paper, here I comment on the paper’s treatment of the existing literature, some of which involves work I have contributed to. In this instance I find their characterization of the literature to grossly misrepresent what the existing research actually says. I have shared my comments with Drs. Holland and Webster, to which I received the following reaction from Greg Holland: “We shall not be modifying the paper as a result of your comments.”

Below I present their original text and my comments. We think that readers can judge for themselves whether a mischaracterization of the literature has occurred. I promised Peter Webster that I wouldn’t speculate on their motivations, and so I’ll stick to the facts in what I present below. I do know that when scientists misrepresent each others work, it is likely to stymie the advancement of knowledge in the community, and thus should be of general concern. When such misrepresentations are missed in the peer review process this also should raise some concerns. In this case I find the misrepresentations obvious to see and egregious, occurring in just about every sentence in the relevant paragraph.

Do note that the comments below do not get into their statistical analysis, which is worth considering separately on its own merits, but which goes beyond the focus of this post. Both Drs. Holland and Webster are widely published and respected scientists with admirable track records. They are welcome to respond here if they’d like. And I do note that different people can interpret the literature in different ways, so the below is my reading only.


Holland and Webster’s new paper can be found here in PDF and the text I have excerpted below in bold comes from their pp. 5-6. My comments are interlaid within their text.

Questions have been raised over the quality of the NATL data even for such a broad brush accounting. For example, a recent study by Landsea et al (2006) claimed that long-term trends in tropical cyclone numbers and characteristics cannot be determined because of the poor quality of the data base in the NATL even after the incorporation of satellite data into the data base. Landsea et al. also state unequivocally that there is no trend in any tropical storm characteristics (frequency or intensity) after 1960, despite this being established in earlier papers by Emanuel (2005) and Webster et al. (2005), and more recently by Hoyos et al. (2006).

Here is what I read in Landsea et al. (2006) (PDF): “There may indeed be real trends in tropical cyclone intensity . . .” Holland and Webster report the opposite of what Landsea et al. (2006) actually says. Landsea et al. (2006) state that they do not believe that the data record is of sufficient quality to definitively detect trends. They do not say that there are no trends. Holland and Webster ascribe a claim to Landsea et al. that they do not make.

Figure 1 shows a strong statistically significant trend since the 1970s similar to that found by Hoyos et al. (2006) and Curry et al. (2006). The overall Landsea et al. analysis is curious and is based on the premise that the data must be wrong because the models suggest a much smaller change in hurricane characteristics relative to the observed SST warming (e.g., Henderson-Sellers et al 1998).

Here is what Landsea at al. (2006) actually say: “Theoretical considerations based on sea surface temperature increases suggest an increase of ~4% in maximum sustained surface wind per degree Celsius (4, 5). But such trends are very likely to be much smaller (or even negligible) than those found in the recent studies (1-3).” Landsea et al. (2006) are reporting a finding accepted in the community. Indeed, the recent WMO statement (written and signed by Greg Holland) states, “The more relevant question is how large a change: a relatively small one several decades into the future or large changes occurring today? Currently published theory and numerical modeling results suggest the former, which is inconsistent with the observational studies of Emanuel (2005) and Webster et al. (2005) by a factor of 5 to 8 (for the Emanuel study).” Holland and Webster do not cite the WMO statement.

In contrast, Michaels, Knappenberger and Landsea (2005) argue the opposite, that the models must be wrong because they do not agree with the data. We shall show later that there are factors not included in the models that may explain some of the differences between model and observed trends.

Michaels et al. (2005) do not say that “the models must be wrong because they do not agree with the data.” They say that if you run the models with different inputs you get different results. They write (PDF), “when [Knutson and Tuleya’s model is] driven by real-world observations rather than unrealistically parameterized and constrained model conditions, the prospects for a detectable increase in hurricane strength in coming decades are reduced to the noise level of the data.” Michaels et al. are not comparing data with models, but looking at modeled output using different inputs.

Further, noticeable by omission is that Holland and Webster ignore relevant work that discusses the relationship of models, theory, and observations that includes Landsea as an author (which seems to be the focus of this paragraph). In particular the following paper discusses this subject explicitly:

Pielke, Jr., R. A., C.W. Landsea, M. Mayfield, J. Laver, R. Pasch, 2006. Reply to Hurricanes and Global Warming Potential Linkages and Consequences, Bulletin of the American Meteorological Society, Vol. 87, pp. 628-631. (PDF)

It particular Pielke et al. (2006) responds to a statement in a related paper (Anthes et al., Greg Holland included in the et al., PDF), that says that there is “broad consistency between observations, models, and theory.” This statement is contradicted by Pielke et al. (2006) and WMO (2006), the latter is actually signed by Holland.

Of greater concern is that the conclusions in the Landsea et al. paper are at odds with several previous publications that include the same authors (e.g. Owens and Landsea 2003, Landsea et al. 1999), without introducing any additional evidence. These papers state clearly that the author’s considered that the period of reliable and accurate NATL records commenced in 1944 with the implementation of aircraft reconnaissance.

I coauthored Landsea et al. (1999) (PDF) and in that paper there are indeed statements on concerns about post-1944 hurricane data (e.g., at p. 94). Further, Landsea et al. (2006) cite a range of post-1999 studies acknowledging new uncertainties in data and methodologies, (e.g., C. Velden et al., Bull. Am. Meteorol. Soc., in press.; J. A. Knaff, R. M. Zehr, Weather Forecast., in press). To say that there has been no additional evidence cited by Landsea et al. (2006) (when Holland and Webster’s work is key to that new evidence) is simply misleading and wrong.

The bottom line here is that while this is just one paragraph in one paper, there is perhaps reason to be concerned about the fidelity of the literature, whatever the underlying causes may be. We have documented other shortfalls in the literature on several occasions on this site. To the extent that these data points are representative of broader problems in the climate literature, scientists should redouble their efforts to exert high standards of quality control. For if I can spot these misrepresentations in the literature, then others will as well.