Many countries have used mathematical models to make predictions on COVID-19 in order to be able to intervene effectively, but modelling has its limitations and putting too much emphasis on it may have misguided policy-makers, according to two reports.
“The COVID-19 pandemic, by virtue of its global scope and scale, has starkly revealed the limits of a time-tested medical epidemiologic model. It is too narrowly focused, contributing to missed opportunities at every step of the way,” says engineer and physician Dr. Harvey Schipper in a report for the Macdonald-Laurier Institute.
“The global response has been framed from the perspective of a single disease, to the exclusion of all other medical causes of morbidity and mortality. Beyond that, the model does not take into account the broader consequences of the new disease and our response to it: economic, social and political.”
Much of the initial response to COVID-19, which in many countries included strict lockdowns, social distancing, and a shut-down of the economy, was based on a model resulting from a study done at Imperial College in Britain by epidemiologist Neil Ferguson.
In a paper produced in March, Ferguson made grim predictions for various countries including Canada, warning that tens of millions would die worldwide if these sorts of radical actions weren’t taken.
Titled “The Flawed COVID-19 Model That Locked Down Canada,” the critique notes that the computer code that generated Ferguson’s doomsday predictions was neither public nor peer reviewed, and that evidence has emerged that the professor has a long history of over-predicting deaths by a wide margin.
“Two months after Professor Ferguson’s publication, experts raised serious problems with his model. A software engineer from Google found inexplicable errors in the model, and it contains unverifiable information,” says St. Onge in a press release.
The critique says Ferguson’s epidemiological model “took the world by storm,” but it’s clear now that “Canada and other countries may have vastly overreacted to a single bad projection.”
For Canada, Ferguson predicted that around 326,000 people would die in 2020, but if “interpersonal contact rates” were reduced by 75 percent, deaths would fall to 46,000. As of July 13, however, there were 108,000 confirmed cases of COVID-19 in Canada and around 8,787 deaths. His model predicted 1.4 million deaths in Japan, whereas the actual number as of July 14 was 980, and Japan never implemented a shutdown.
St. Onge and Campan note that epidemiological models have since been revised dramatically downward from Ferguson’s extreme projections, and restrictions continue to be gradually eased in Canada, “but as there is talk of a second wave, it is more important than ever to question the case for lockdown.”
Schipper says models are useful because they are “mental constructs that help us understand things. They are the necessary oversimplification of reality that enables us to act.”
He notes, however, that they are not “ultimate truth” and when it comes to pandemics specifically, models are “relatively narrow” and based on a certain set of questions regarding the “vulnerable host” and the “causal agent.” They intend to identify who is most at risk, what happens after infection, what happens when someone is exposed, how infection spreads, and what the infection needs in order to survive.
When determining how to respond to COVID-19, much of this information wasn’t available, which means “we are forced to assume the worst, from an infectious disease perspective,” he says.
“That leads directly to an inescapable and unachievable truism. If it were possible to isolate every person on earth from one another for around two months COVID-19 would be gone. But, of course, we can’t.”
Going forward, Schipper says models for pandemics should address shortcomings by being mindful of different factors for measurement including economic impact, the geography/culture of a place, a lexicon based on common language, and all causes of morbidity and mortality.
“If we are to establish a paradigm with global application it will have to be based on a common understanding of key measurable elements, such as diagnostic criteria, incidence, case and mortality attribution from the classical medical perspective, and similar consistent measures of other attributes such as social density, economic conditions, even cultural attributes,” he writes.
“Epidemics and pandemics are nature’s way of exposing the gaps in our societal structure. There will be more, and they could be more lethal. What COVID-19 is telling us is that our excellent infectious disease medical model only takes us so far.”
St. Onge and Campan say there needs to be proper scrutiny and oversight of the scientific models policy-makers rely on.
“Anonymized data and computer code should be open to public scrutiny as the minimum requirement for any study that is used to justify public policy, from lockdowns to carbon taxes,” they write.
“As for the present crisis, any continuing or future lockdowns and restrictions must be based on transparent and verifiable science. We emphatically have a right to expect that any policy that threatens millions of Canadians’ livelihoods and civil liberties is based on the very highest level of scientific rigour.”
