A higher standard for climate risk modeling?

Lately it seems there’s nowhere to hide from troublesome weather. The summer of 2023 was the hottest on record by a wide margin^[1], and many local records have been smashed, including in China a scorching new highest-ever reported temperature of 52.2 °C (126 °F)^[2]. Despite Canada’s deserved reputation as a cold climate country, for much of the summer many of its cities have been enveloped by thick smoke from an unprecedented wildfire season farther north^[3]. In the wet tropics of central Panama, the usual summer rains never arrived, pushing water levels in the Canal to an all-time low and shipping rates to record highs^[4]. And in Minneapolis where I live, the Twin Cities Marathon - scheduled for October 1 - was cancelled with only two hours’ notice due to dangerous heat^[5].

In the aggregate, events like these carry an outrageously high price tag. Worldwide, the economic cost of weather and climate disasters was nearly $1.5 trillion over the decade 2010-2019^[6]. So far in 2023, the United States has experienced 23 separate billion-dollar disasters, the highest number ever (and the year is not yet over)^[7]. This year’s summer heatwaves may have caused China’s gross domestic product to decline by more than a full percentage point, with Spain and Greece probably also having felt similar economic harm^[8].

As a result, it’s no surprise the demand for climate analytics is booming. This past year, climate risk management attracted $343 million in new investment, with increasing contributions from later stage capital^[9]. By 2027 the global market for climate risk solutions is projected to grow to more than $4 billion^[10]. The sector’s human capital is also expanding, as more and more researchers move from academia to industry so their expertise can be applied more directly to the climate crisis^[11]. I’m part of that trend, having left a tenured faculty position at the University of Minnesota to join WTW last year. It’s exciting to step outside the ivory tower and help colleagues and clients make smarter decisions about weather- and climate-related risks. By guiding investment capital, affecting housing and insurance prices, and setting priorities for adaptation projects, our industry has a significant impact on the economy^[12].

But because of that influence, regulators and policymakers are now paying closer attention to the business of climate risk. In the United States, this past April the President’s Council of Advisors on Science and Technology (PCAST) released a report to help Americans understand and prepare for the risks of extreme weather^[13]. Then in September, the White House Council of Economic Advisors hosted a roundtable discussion on climate modeling for risk management applications^[14]. I had the privilege to represent WTW at this meeting and want to share a few ideas discussed in that forum about the future of climate risk modeling.

Hidden reefs and shoals

In their April 2023 report, PCAST made three major recommendations to reduce or avoid harm from extreme weather under a changing climate. First, US federal agencies should work together to produce reliable estimates of weather risks (including extreme temperatures, intense rainfall, and high winds) for all parts of the country and every year between now and 2050. Next, the US ought to develop a national adaptation plan so communities are better prepared for extreme weather, disaster relief is distributed fairly, and the benefits and costs of adaptation options are better understood. And finally (and most relevant to the climate risk industry), the federal government should challenge the private sector to improve its tools for predicting the severity and frequency of weather hazards and the human and economic losses they will cause.

Why do we need better tools to assess weather risks? At the White House, several speakers emphasized that, under a changing climate, the recent past is not a reliable guide to the future. In many fields, the conventional approach to risk assessment is based on a retrospective survey of historic hazards and impacts. But when perils evolve — becoming more severe, more frequent, or more correlated — that same backwards-looking perspective will cause the true costs of climate-related damages to be underestimated. Of course, that gap provides the raison d’être for the burgeoning climate risk industry, which aims to predict how a host of weather-related perils will behave on a warmer world. Participants in the roundtable acknowledged the central contribution made by climate analytics to the (re)insurance industry^[15] and climate-related financial disclosures^[16]. But the same group also expressed strong concerns about the state of climate and catastrophe analytics, arguing these tools have critical shortcomings that lurk just below the surface.

• For most end users, it is not possible to examine or understand the inner workings of climate risk and catastrophe models. Climate risk tools combine information about weather, topography, and surface conditions to predict weather-related hazards and other, more sophisticated models (catastrophe models) also project financial and human losses due to those hazards. Both types of models are usually proprietary and closed source, and the detailed structure, inputs, or assumptions of these models is not visible to outsiders. Panelists also highlighted the fact that some tools have not passed the peer review of public science and so have not been subject to independent assessment by domain experts. As a consequence, some end users and consumer representatives are hesitant to rely upon the results of either type of model and, in certain jurisdictions their use can be at odds with transparency laws.
• Almost everyone needs information about weather and climate risk, but not everyone can get it. The first catastrophe models were built to help insurance and reinsurance companies estimate their potential financial losses due to extreme weather events^[17]. The remit of such services has grown to include losses due to non-weather events such as terrorism and cyber attacks, but are still largely restricted to those perils that matter most to (re)insurers. Several speakers said the same high-quality risk information is also needed for hazards that do not trigger insurance claims but still do significant social or economic harm, such as drought or extreme heat. But while the PCAST report encouraged the climate risk industry to expand its customer base to serve “every public and private entity that faces climate-related risk” (which is to say, all public and private entities), the roundtable also conceded that such specialized services can be expensive and are beyond the reach of many businesses or organizations.
• It’s hard to judge the reliability of weather and climate risk estimates. As the saying goes, it’s tough to make predictions, especially about the future^[18]. It is particularly tricky to forecast the future behavior of the most extreme weather events within a new and unfamiliar global climate. In the research community, standard practice is to test new weather or climate models by challenging them to predict known historical events. But after its survey of industry leaders and academic experts, PCAST concluded that commercial weather-hazard modeling does not have a similarly rigorous tradition of skill scoring. One member of the roundtable went so far to say that, in their view, certain members of the climate risk industry prefer to avoid tough conversations about modeling error and uncertainty. In the absence of industry-standard metrics to test model performance, PCAST complained that, right now, it is not easy to get a simple, clear answer to the question: “How well does your model work?”

Bolstering the climate risk ecosystem

Inside the White House, participants enjoyed a frank and wide-ranging discussion of the strengths and limitations of current climate risk offerings. Some industry representatives expressed concern about the necessary tension between end users’ desire for openness and the need for vendors to safeguard their own intellectual property. But across the room, there was unanimous agreement that any person or organization facing a climate risk deserves to know about it. In the ideal, everyone should have access to risk information tailored to their specific needs so they can identify places at high risk and make smart decisions about risk mitigation. To meet that goal, PCAST has advised the US federal government to take three specific actions in order to promote the development of a “stronger academic and private ecosystem of climate risk assessment”.

• The US government should inventory and release data that could be used to develop and test weather-hazard models. In order to understand how extreme weather leads to financial and non-financial losses, it’s essential to have reliable data on past events and their real-world impacts. Federal agencies hold a wealth of economic loss information from weather hazards, including data on financial damages at the property level. PCAST wants the government to make these data more easily accessible to the research, non-profit, and private-sector modeling communities at the census block or census tract level. It also recommends that qualified academic researchers be granted access to property-level loss data so they can explore how financial damages are influenced by specific mitigation actions.
• Federal agencies should develop guidelines to gauge the accuracy of climate risk and catastrophe models. Weather forecasting has a long tradition of skill scoring, where model predictions are compared against real historic events. And in select cases, academic research has shown the same kind of test can be applied to weather-hazard or hazard-loss models. PCAST has charged the National Oceanic and Atmospheric Administration and Federal Emergency Management Agency to produce guidelines so the performance of climate risk and catastrophe models can be evaluated by skill scores. Going further, the report also recommends the federal government should prioritize or even require the use of skillful models in all applications for funding to mitigate weather hazards. In their view, skill-scoring would allow end users to “comparison shop” for the best tool, encourage a faster pace for model development, and ultimately help the nation to develop a healthier climate risk assessment industry.
• Federal agencies need to fund research on risk-assessment modeling, train the skilled people required by industry, and ultimately, create an open-source catastrophe model for weather and climate hazards. As set out by PCAST, the climate risk sector is now confronted by several interlocking challenges. The industry needs to track an evolving set of risks arising from climate change, manage a highly competitive landscape, and diversify its customer base well beyond its traditional clientele of insurers and reinsurers. To overcome these difficulties, PCAST recommends the federal government invest in expanded academic research and training in weather hazard and hazard-loss modeling. In the medium to long term, this support would produce new insights into the likelihood and economic costs of extreme weather events, spur improvements in risk tools, and provide the skilled labor needed by the climate analytics industry. They also want federal agencies to fund the cooperative development of an open-source climate and weather risk assessment system (a multi-peril catastrophe model). Although PCAST admits such a product would doubtlessly be seen as competing with existing industry offerings, they argue a free, government-built model would serve as a useful baseline to compare against tailored commercial solutions aimed at paying clients.

A smarter way to manage climate risk

When you work as a climate scientist, you live under an essential contradiction: we do our best to make reliable predictions about future climate change, but really, we would rather those predictions not come to pass. That point was raised as part of my discussion on reproducibility in climate science with NASA’s Dr. Gavin Schmidt at a public event organized by the Minnesota Center for the Philosophy of Science earlier this year. Personally, I’d feel enormous relief if global temperatures would stop their march upwards, western, and boreal wildfires would become less widespread, and the southwestern United States would receive more water rather than less. But facts are that Earth’s climate has changed and unless we make rapid progress to reduce emissions, we must expect climate-dependent perils to respond in kind. For the climate risk industry, our core challenge is to uptake new scientific advances, produce accurate and individualized estimates of current and future risks, and clearly communicate those data and insights to our clients.

My colleagues and I in the WTW Research Network serve as the junction between our firm and its extensive and well-established network of research partners, all of whom are top experts in climate and hazard science. Whether we are harnessing the power of satellites to take the measure of destructive hailstorms, demystifying the impact of climate litigation on the insurance market, or considering the geopolitical implications of climate change, our research network has exactly the expertise required to translate the latest science on weather catastrophes into smarter risk management. And through our model evaluation team, we can provide deep insights into the performance of catastrophe and climate risk models and support our clients to take their greatest advantage from each tool.

Precisely because there’s nowhere to hide from weather and climate risks, our sector should expect the demand for its services — and the attention paid to its findings — to continue to grow.

Footnotes

1. Copernicus Climate Change Service (2023), Summer 2023: The hottest on record, September 6, 2023. Return to article
2. Reuters (2023), China logs 52.2 Celsius as extreme weather rewrites records, July 17, 2023. Return to article
3. Fox Weather (2023), Canada's supercharged wildfire forecast could mean bad air quality in US through fall. September 11, 2023. Return to article
4. U.S. Energy Information Administration (2023), Shipping rates reach record highs as historic drought at the Panama Canal causes delays, September 27, 2023. September 11, 2023. Return to article
5. Star-Tribune (2023), Record heat that canceled Twin Cities marathon spiked by climate change, some say. October 2, 2023. Return to article
6. World Economic Forum (2023), The economic costs of extreme weather are soaring, but number of deaths is falling fast. Here’s why. June 2, 2023. Return to article
7. National Oceanic and Atmospheric Administration (2023), U.S. saw its 9th-warmest August on record, September 11, 2023. Return to article
8. Allianz Research (2023), Global boiling: Heatwave may have cost 0.6pp of GDP. August 4, 2023. Return to article
9. CommerzVentures (2023), Climate FinTech: FinTechs driving net-zero continue to attract record VC funding, February 2023. Return to article
10. Verdantix (2022), Market Size And Forecast: Climate Risk Digital Solutions 2021-2027 (Global). Return to article
11. Ma (2022), Scientists are leaving the ivory tower for climate tech startups. Here’s why. Protocol, September 27, 2022. Return to article
12. Condon (2023), Climate services: The business of physical risk. Arizona State Law Journal 147. Return to article
13. President’s Council of Advisors on Science and Technology (2023), Report To The President
    Extreme weather risk in a changing climate: Enhancing prediction and protecting communities. April 2023. Return to article
14. The White House (2023), Readout from climate risk modeling roundtable. September 20, 2023. Return to article
15. Hersher (2023), Insurance firms need more climate change information. Scientists say they can help. National Public Radio, May 23, 2023. Return to article
16. Hamm (2023), Climate reporting: Comparing disclosure regimes with TCFD. WTW, June 23, 2023. Return to article
17. National Association of Insurance Commissioners (2023), Catastrophe models (property). April 3, 2023. Return to article
18. The fact that this quote is usually attributed to either (a) a Nobel prize-winning quantum physicist from Copenhagen or (b) a Hall of Fame American baseball player from St. Louis also reminds us how hard it is to keep an accurate record of the recent past! Return to article