The COVID-19 Models That Rule Our Lives Got It Wrong

The government-ordered shutdown of much of Michigan’s economic and social activity in March saved tens of thousands of lives, Gov. Gretchen Whitmer and her top aides claimed last week. In doing so, they cited a recent analysis from a COVID-19 project at Imperial College London. But the models developed by that group have been wildly off in their predictions and have faced severe criticism from other experts.

The report cited by the governor suggested that Michigan was one of the leading U.S. states in reducing citizen mobility, and thus transmission of a disease which, unabated, could have killed 74,000 state residents, according to The Detroit News. As of today, fewer than 6,000 have died in Michigan.

“The vast majority of Michiganders have done the right thing by staying safer at home . . . help(ing) us flatten the curve and save lives,” the governor said, according to the Detroit News.

Joneigh Khaldun, the state’s chief medical officer, said in a state press release, “The data shows that our aggressive actions against the virus are working, and that implementing them has saved lives.”

This narrative was repeated in media reports in Michigan and elsewhere.

It is hard to find any example of a media report questioning the underlying assumptions and data used to develop the estimates, despite ongoing concerns about the accuracy of models produced by Imperial College and its lead epidemiologist, Neil Ferguson.

COVID-19 models published by Imperial College have influenced elected officials in the U.K. and the U.S.

When Whitmer announced her stay-at-home order and a ban on activity deemed nonessential March 23, she cited a projection derived in large part from Ferguson’s Imperial College team. It said then that 460,000 Michiganders could die from the coronavirus, absent drastic intervention, according to Michigan Advance.

Michigan reported nearly 6,000 confirmed COVID-19 fatalities (and over 67,000 infections) as of June 22, but the daily toll has been in steady decline since mid-April.

The accuracy of models forecasting the spread and lethality of the disease has come under intense scrutiny, in part because the model estimates themselves, and their incorporation into authoritative government policies and orders, have been widely divergent and sometimes incomprehensible.

Imperial College, in particular, has been criticized for the opacity of its calculations, refusing to release the data underlying its initial COVID-19 projections for weeks. Eventually, it issued a revised version that some experts said was a “buggy mess” and unreproducible.

David Richards, co-founder of the British data technology company WANdisco, was not impressed by the Imperial College data he reviewed. He said in a press release, “In our commercial reality, we would fire anyone for developing code like this and any business that relied on it to produce software for sale would likely go bust.”

A veteran software engineer and former Google employee, writing anonymously on the website lockdownskeptics.org said the model’s software was useless. The engineer wrote, “Due to bugs, the (Imperial) code can produce very different results given identical inputs. This problem makes the code unusable for scientific purposes.”

The Imperial model also assumes a dramatically higher rate of fatality from COVID-19 than that incorporated into other forecasts.

“We assume around a 1% infection mortality rate,” Seth Flaxman, a mathematician at Imperial College told The Detroit News. In that interview, he explained how the updated Imperial forecast showed a potential of 74,000 deaths in Michigan, absent mitigation.

By contrast, Stanford epidemiologist John Ioannidis and the Centers for Disease Control and Prevention have estimated the COVID-19 infection fatality rate at less than half that, under 0.4%. Ioannidis has been a critic of the Imperial model, saying its assumptions could lead to substantially inflated death estimates.

The COVID models are not the first Imperial College analyses to be questioned. In 2002, Ferguson said that mad cow disease could kill up to 150,000 by 2080, a forecast that resulted in the mass slaughter of healthy livestock in the U.K. To date, a total of 177 people have died as a result of mad cow disease.

In 2005, an Imperial College model forecast up to 150 million deaths worldwide from the avian flu known as H5N1. Between 2003-09, the disease killed 282, according to the World Health Organization.

A spokeswoman from the Michigan Department of Health and Human Services said the shutdown and phased reopening ordered by the governor was based on a variety of academic assessments, “us(ing) actual on-the-ground data.”

“We have used multiple models for predicting, including models from the University of Michigan Public Health and Medicine, COVID Act Now and COVID19.healthdata.org,” said Lynn Sutfin. “Every model has limitations, which is why we use multiple models when we make decisions.”