The accuracy of risk models is assessed in new JLT Re report

Risk models are good at estimating the losses arising from ‘conventional hurricane events’ but their performance decreases when it comes to more severe or unusual events, according to a study by JLT Re, which analysed the precision of modelled market loss estimates for significant hurricanes since 2004.

The white paper, called Viewpoint Report – Catastrophe models: In the eye of the storm, examines the precision of modelled market loss estimates for significant hurricanes since 2004 and provides a unique perspective in assessing the modelling companies’ real-time loss estimation process.

Its three main conclusions were that: stronger model performance is observed when losses are anticipated and contained; accuracy suffers when events bring unforeseen (and often un-modelled) consequences; and accuracy has improved over time but modelling firms can assist the market further with greater transparency around underlying assumptions.

David Flandro, global head of analytics, JLT Re, said: “This snapshot goes some way to explaining why the industry loss estimates provided by catastrophe modelling firms have led to general scepticism within the re/insurance market over the last 15 years or so. At first glance, there is an overriding trend towards significant loss underestimation (the 2017 hurricanes apart), and it is not even immediately apparent that the range of loss estimates narrows during the lifespan of storms, or that they always become more accurate.”

However, breaking down industry loss estimates into groups of storms with similar characteristics reveals some interesting insights about model performance. JLT Re’s study shows that vendor models have historically performed relatively well for wind events that incurred moderate losses, regardless of landfall location. Or, in other words, conventional hurricane events that do not assume super-cat characteristics are typically captured adequately by vendor catastrophe models. The models, however, have not performed as well for hurricane events where losses extend beyond wind into areas that are not modelled or well understood.

Josh Darr, lead meteorologist, JLT Re, said: “This highlights the inherent difficulties modelling companies face in predicting losses for complex hurricane events that strike highly populated urban areas. These types of events often bring unforeseen consequences that cause losses to spiral. Results for hurricanes Katrina, Ike and Sandy show that catastrophe models have struggled to generate accurate loss ranges in such circumstances.

“In each case, un-modelled loss components accounted for a significant proportion (if not the majority) of the total cost. Vendor firms have in recent years drawn on important lessons learned during these events to recalibrate their models and incorporate a whole host of previously un-modelled perils.”

The report notes that hurricanes Harvey, Irma and Maria (HIM) were an important test for the latest generation of commercial hurricane models. Whilst the accuracy of the modelled losses released for HIM in 2017 was mixed, certain results taken in isolation revealed some encouraging signs given the levels of complexities involved. The wealth of data collected during HIM, along with technological advancements, are likely to result in improved accuracy going forward.

Keith Leung, head of Catastrophe Modelling – International, JLT Re, added: “2017 reinforced the importance of understanding differences between estimates. Working with a range of modelled, post-event views can be valuable as long as the important drivers are clearly highlighted, particularly in situations where significant divergences occur.

“Catastrophe modelling firms can assist the market further here by better communicating the key drivers of loss within each estimate and providing more transparency around the various assumptions being applied. It is likewise incumbent on market participants to review rigorously, or even challenge, some of the more extreme loss estimates released by modelling firms. We hope our report informs the debate and JLT Re looks forward to helping clients understand the important areas of uncertainty associated with future modelled industry loss estimates.”

Make sure you are GDPR compliant and  confirm your email address to keep getting our daily emails

More of today's news

Fairfax mulls Advent-Brit synergies at Lloyd’s as it faces 'considerable challenges'

Markel overhauls leadership; hires ex-Chubb COO to accelerate growth

Market M&A drives MGA growth as entrepreneurs form start-ups

Everest promotes from within for chief claims officer

RSA appoints new head of Dubai operations

LIIBA forms new delegated authority group focused on LM TOM

Stonybrook appoints former AIG as COO, general counsel

Don't miss our insurtech email newsletter - sign up today