Another downward revision in key coronavirus model raises more red flags on reliability

Get the latest BPR news delivered free to your inbox daily. SIGN UP HERE.

The key coronavirus model being used to guide policy decisions across the country has been revised downward again, leading to more concerns about its poor reliability, with one university researcher declaring outright that it’s not “well-suited” for the current job.

In an update published Friday, the University of Washington’s Institute for Health Metrics and Evaluation (IHME) revised its expected Aug. 4th coronavirus death toll down by a whopping 8,533 deaths, thus reducing the final toll from 68,841 deaths to 60,308.

The previous death toll of 68,841 was itself a revised toll — one significantly lower than the IHME’s original 240,000 death estimate.

IHME’s hospitalization models have been just as incorrect:

These constant downward revisions have not earned it much fanfare.

It’s not a model that most of us in the infectious disease epidemiology field think is well-suited [to projecting coronavirus deaths and hospitalizations],” epidemiologist Marc Lipsitch of the Harvard T.H. Chan School of Public Health reportedly said to reporters earlier this week.

While that’s one way of putting it, epidemiologist Ruth Etzioni of the Fred Hutchinson Cancer Center had a far harsher analysis.

That the IHME model keeps changing is evidence of its lack of reliability as a predictive tool,” she reportedly said. “That it is being used for policy decisions and its results interpreted wrongly is a travesty unfolding before our eyes.”

According to Fox News, proponents of the IHME model have defended it by describing it as an ever-changing “snapshot” of the current situation.

“Those supportive of such modeling say that it is not a crystal ball, but a snapshot of a situation based on the data and facts available at the time,” Fox reported Saturday.

“As those facts change, so do the models. Since those 84,000 estimates, more states have implemented sweeping social distancing and lockdown measures in an effort to slow the spread of infections. Revising models show that those strategies are working, they say.”

But speaking with “America’s Newsroom” anchor Ed Henry on Friday, one of the University of Washington faculty members responsible for helping produce the model, Dr. Christopher Murray, admitted that the model is about as reliable as “forecasting the weather.”

“We try to take into account all the data that is coming in from all the states and so all of our numbers are going to be revised as we see progress faster or slower,” he said.


(Source: Fox News)

The problem with the IHME model is that it’s based on an unproven strategy.

“There are two tried-and-true ways to model an epidemic,” StatNews notes. “The most established, dating back a century, calculates how many people are susceptible to a virus (in the case of the new coronavirus, everyone), how many become exposed, how many of those become infected, and how many recover and therefore have immunity (at least for a while).”

“Such ‘SEIR’ models then use what researchers know about a virus’s behavior, such as how easily it spreads and how long it takes for symptoms of infection to appear, to calculate how long it takes for people to move from susceptible to infected to recovered (or dead).”

The second “tried-and-true” way, the “agent-based” model, involves using computing power to “simulate the interactions of millions of individuals as they work, play, travel, and otherwise go about their lives. Both of these approaches have often nailed projections of, for instance, U.S. cases of seasonal flu.”

The IHME model relies on neither strategy, according to StatNews: “It doesn’t even try to model the transmission of disease, or the incubation period, or other features of Covid-19, as SEIR and agent-based models at Imperial College London and others do.”

“It doesn’t try to account for how many infected people interact with how many others, how many additional cases each earlier case causes, or other facts of disease transmission that have been the foundation of epidemiology models for decades.”

Instead, it forms its projections by starting “with data from cities where Covid-19 struck before it hit the U.S.,” never mind any fundamental differences pertaining to demographics, healthcare systems, etc.

“It then produces a graph showing the number of deaths rising and falling as the epidemic exploded and then dissipated in those cities, resulting in a bell curve. Then (to oversimplify somewhat) it finds where U.S. data fits on that curve,” StatNews notes.

“The death curves in cities outside the U.S. are assumed to describe the U.S., too, with no attempt to judge whether countermeasures —lockdowns and other social-distancing strategies — in the U.S. are and will be as effective as elsewhere, especially Wuhan.”

Despite the flaws in the IHME model, it was used to justify the nation’s economically devastating lockdowns and is now being used to justify extending them past May 1 — and this despite competing models painting a far rosier picture:

Powered by Topple

Vivek Saxena


Latest Articles