Catastrophe (cat) models are computer models of expected losses that allow the insurance and reinsurance industries to have a quantitative basis for calculating the risks that they face and hence set prices in a manner that is actuarially sound (for some background see this post). At least, that is how it is supposed to work in theory. (Warning: This is a long and detailed post.)

In practice things are far more complex, not least because cat models are “black boxes” developed outside of the public view, which makes it impossible to evaluate them independently. Cat models, and their opaqueness, are the focus of an emerging debate between consumer groups and cat model companies. It is not an exaggeration to suggest that in this debate are some indications that hurricane insurance may be changing dramatically. Insurance Journal reported the following last week (Note: Risk Management Solutions (RMS) is a leading provider of “catastrophe models.”)

Risk Management Solutions has defended its hurricane risk models in the face of consumer groups’ criticisms that the models are more political than they are scientific and that they are designed to justify insurance premium increases.

RMS declined to address the specific allegations made by the Consumer Federation of America and the Center for Economic Justice but issued a statement claiming that the groups’ viewpoint “is a misrepresentation” of its role in the insurance industry.

On March 27, CFA and CEJ wrote to the National Association of Insurance Commissioners raising questions about recent upgrades in the RMS wind models that the groups maintain would lead to “unjustified increases in homeowners and other property casualty insurance rates.”

The letter, signed by CFA’s J. Robert Hunter and CEJ’s Bernie Birnbaum, called for state regulators to increase regulation of RMS and other third-party organizations, including credit-scoring firms, whose work impacts insurance rates and availability.

The groups also blasted state regulators for failing to closely monitor the activities of RMS and other third-party rating organizations.

The consumer watchdogs referred to a recent announcement buy RMS that it is changing its hurricane models. RMS said that “increases to hurricane landfall frequencies in the company’s U.S. hurricane model will increase modeled annualized insurance losses by 40 percent on average across the Gulf Coast, Florida and the Southeast, and by 25-30 percent in the Mid-Atlantic and Northeast coastal regions relative to those derived using long-term 1900-2005 historical average hurricane frequencies.”

The groups, claiming this would mean overall double-digit rate increases from Maine to Texas, contend that while RMS says that this increase is necessary for scientific reasons, “the evidence indicates that the primary reason for the change appears to be not science at all, but politics.”

The letter from the Consumer Federation of America (CFA) and the Center for Economic Justice (CEJ) can be found here in PDF. The letter states:

Consumers were told that, after the big price increases in the wake of Hurricane Andrew, they would see price stability. This was because the projections were not based on short-term weather history, as they had been in the past, but on very long-term data from 10,000 to 100,000 years of projected experience. The rate requests at the time were based upon the average of these long-range projections. Decades with no hurricane activity were assessed in the projections as were decades of severe hurricane activity, as most weather experts agree we are experiencing now. Small storms predominated, but there were projections of huge, category 5 hurricanes hitting Miami or New York as well, causing hundreds of billions of dollars in damage. Consumers were assured that, although hurricane activity was cyclical, they would not see significant price decreases during periods of little or no hurricane activity, nor price increases during periods of frequent activity. That promise has now been broken.

The CEA and CEJ are expressing frustration that the science of hurricanes has evolved since the models were first developed. For its part, RMS goes some distance towards concluding that hurricane history is now irrelevant. In a white paper (PDF) used to justify its new approach to risk, RMS writes:

Given a constant climatological state (or if annual variations from that state are short lived and unpredictable) the activity rate in a catastrophe model can best be represented as the average of long-term history. In this situation there is no need to characterize the period over which the activity is considered to apply because, with current knowledge, it is expected that rate will continue indefinitely. The assumption that activity remains consistent breaks down, however, where there are either multi-year fluctuations in activity or persistent trends. It then becomes necessary to characterize the time period over which the activity in the Cat model is intended to apply.

It does not seem to me that RMS recognizes how profoundly revolutionary this perspective is, or its potential consequences for their own business. What they are say is that the historical climatology of hurricane activity is no longer a valid basis for estimating future risks. This means that the catastrophe models that they provide are untethered from experience. Imagine if you are playing a game of poker, and the dealer tells you that the composition of the deck has been completely changed – now you don’t know whether there are 4 aces in the deck or 20. It would make gambling based on probabilities a pretty dodgy exercise. If RMS is correct, then it has planted the seed that has potential to completely transform its business and the modern insurance and reinsurance industries.

What happens if history is no longer a guide to the future? One answer is that you set your expectations about the future based on factors other than experience. One such approach is to ask the relevant experts what they expect. This is what RMS did last fall, convening Kerry Emanuel, Tom Knutson, Jim Elsner, and Mark Saunders in order to conduct an “expert elicitation”. Here is how RMS described their process and its results:

The experts were asked to address and resolve the following questions:

• What is the expected basin activity of category 1-5 and category 3-5 hurricanes in the Atlantic basin over the next five years?

• What is the expected activity for category 1-5 and category 3-5 hurricanes at U.S. landfall over the next five years?

• How much longer can we expect the recent period of high Atlantic hurricane activity to persist?

• What is the expected activity of category 1-5 and category 3-5 hurricanes in the Caribbean over the next five years?

The experts discussed each question for one hour and a consensus opinion was then established for each question. The main conclusions reached by the experts were:

1. Activity in the Atlantic basin for the next five years is expected to be close to the average of the past 11 years. The probability for the activity to return to levels corresponding to the long term baseline is small over the next five years.

2. The experts each provided perspectives on the probability of the five-year U.S. landfalling rates being above or below certain thresholds, with the probability estimates provided by each expert considered interdependent (under a Poisson assumption). Relative to the historic 1900-2005 baseline, the increases in landfalling activity rates averaged across the group convert into about a 20% increase in the rate of category 1-2 storms and a more than 30% increase for category 3-5 storms.

3. The high levels of activity observed over the last 11 years are expected to last for at least another 10-15 years.

4. The medium-term activity in the Caribbean region is expected to be consistent with the perspective of activity developed for the full basin.

RMS then used the information provided by the panel of experts to implement the five-year view of activity rates in both the U.S. and Caribbean Hurricane models.

In general, expert elicitation is a very useful method for aggregating the views of a community of specialists on focused questions. In this instance, RMS has left itself wide open to some valid criticism. For instance, although each scientist included in its elicitation is well-respected in the field, the four experts represent a small subset of the relevant and available expertise on the questions that were asked. Including more of the community in an expert elicitation would add to the legitimacy of the results, even if the conclusions themselves don’t change. Because the elicitation resulted in a forecast of persistence, I’d guess that including more experts probably wouldn’t much change the results, although it might increase the uncertainty of the conclusions.

Also, RMS conducted its elicitation October, 2005 with the intent that it will shape its risk estimates for the next 5 years. This is wholly unrealistic in such a fast moving area of science. It is unlikely that the perspectives elicited from these 4 scientists will characterize the views of the relevant community (or even their own views!) over the next five years as further research is published and hurricane seasons unfold. Because RMS has changed from a historical approach to defining risk, which changes very, very slowly, if at all over time, to an expert-focused approach, it should fully expect to see very large changes in expert views as science evolves. This is a recipe for price instability, exactly the opposite from what the consumer groups, and insurance commissioners, want.

From the perspective of the basic functioning of the insurance and reinsurance industries, the change in approach by RMS is an admission that the future is far more uncertain than has been the norm for this community. Such uncertainty may call into question the very basis of hurricane insurance and reinsurance which lies in an ability to quantify and anticipate risks. If the industry can’t anticipate risks, or simply come to a consensus on how to calculate risks (even if inaccurate), then this removes one of the key characteristics of successful insurance. Debate on this issue has only just begun.