Forecasting for COVID-19 has failed

Status
Not open for further replies.
We need to get over the idea that an organization (WHO, CDC, NIAID etc) or individual (Dr Fauci) is/are deceptive/evil/harmful because models they were using turned out to be inaccurate.
Welcome to the world of science & medicine; things are not clearcut and people do their best with the information they have available.
 
In the context of the covid epidemic, his name should always be prefaced with 'the discredited'.
 
First, what was the purpose of the various forecasts?

To accurately model the anticipated progression of the disease?

Or to spur action in a public that would be apathetic to anything but a "doomsday scenario"?

I find it hard to believe that lessons learned early in just about any technical field(a deep understanding of what data means(and doesn't mean) is critical, changing more than one variable at time greatly increases the difficulty of seeing a "cause and effect" relationship, selection bias is rampant, etc.) were lost on those "at the top of their field".
 
These sorts of models are fantastically difficult to do, since the data can change markedly by the time it gets keyed into the computer. And day to day conditions can change do to changes in people's behavior should they see models. COVID19 would be stopped dead in its tracks if EVERYBODY stayed home for two weeks.
 
steingar said:
These sorts of models are fantastically difficult to do, since the data can change markedly by the time it gets keyed into the computer. And day to day conditions can change do to changes in people's behavior should they see models. COVID19 would be stopped dead in its tracks if EVERYBODY stayed home for two weeks.
Click to expand...
COVID19 would be stopped dead in it's tracks in 10 minutes if EVERYBODY committed suicide.
 
steingar said:
These sorts of models are fantastically difficult to do, since the data can change markedly by the time it gets keyed into the computer. And day to day conditions can change do to changes in people's behavior should they see models. COVID19 would be stopped dead in its tracks if EVERYBODY stayed home for two weeks.
Click to expand...

Agreed.

It's incredibly hard to get people to wear masks, and yet if everyone wore a mask transmission rates would go way down, and we could fully re-open.

However, people being what they are, with some of them seeing wearing of a mask as some sort of major affront to their liberty, independence, cool factor, or whatever - rather than as a socially responsible thing to do that benefits others as well as ourselves, you now have a confounding variable in the model that either changes the rates in unpredictable ways and/or substantially reduces the effectiveness of the model.

But then we all blame the scientists rather than blaming people who won't cooperate in implementing the protective measure.

No one can make you wear a mask. No one can make a pilot do a thorough preflight, or take 5 minutes to at least look at an area brief at 1800wxbrief.com on their smart phone before they blast off. You'd think they were all common sense things to do for most people, but you'd be wrong.

The problem with all three is that failing to do them puts others at risk and wearing a mask is the most insidious as it's the one where people are least likely to see their role in infecting someone else.
 
This article seems to be a pedantic way of stating two fundamental truths that every engineering student learns as an undergrad: (1) all models are wrong and (2) some models are useful.

For a model to be useful, the user must understand the ways in which it is wrong, and thus informed use the model appropriately, within its limitations.

We have had a situation wherein the users were laymen without good understanding of the errors inherent in the models, and it was incumbent on the model creators to communicate the shortcomings in a fashion understandable by the users and help constrain actions taken as a result. In this, the professionals seem to have failed. The models have been used to make dire predictions which were largely unconstrained by the modelers, and the users were in a situation where they had little choice but to assume the models were correct and take drastic action.

We can rant and rave about the politicians, but it seems to me the real failure lies with the professional community that was informing them.
 
steingar said:
COVID19 would be stopped dead in its tracks if EVERYBODY stayed home for two weeks.
Click to expand...

Yep, we should have cut off all access to food, and medical services. We should have also shut down the power grid, shut down the water and sewage, shut down the trash pickup. Yep, that would have fixed everything. o_O :rolleyes:
 
I'm thinking back about state of weather forecasting models years and year ago...
 
Larry Vrooman said:
However, people being what they are, with some of them seeing wearing of a mask as some sort of major affront to their liberty, independence, cool factor, or whatever - rather than as a socially responsible thing to do that benefits others as well as ourselves, you now have a confounding variable in the model that either changes the rates in unpredictable ways and/or substantially reduces the effectiveness of the model.
Click to expand...
It would be a whole lot more palatable to me to wear a mask if I knew there was an end game to this. As it is, apparently this is a forever deal. THAT I do not accept.

As for me,I would rather catch the thing and get it over with and be done with it.
 
Greg Bockelman said:
It would be a whole lot more palatable to me to wear a mask if I knew there was an end game to this. As it is, apparently this is a forever deal. THAT I do not accept.

As for me,I would rather catch the thing and get it over with and be done with it.
Click to expand...
I wish there was a "double like" button.
 
Capt. Geoffrey Thorpe said:
"All models are wrong, some are useful" George E.P. Box
Click to expand...


Exactly. He further wrote, "So since all models are wrong, it is very important to know what to worry about; or, to put it in another way, what models are likely to produce procedures that work in practice (where exact assumptions are never true)."

I think that's the part that was largely missed. Those who decided what to put into practice were not those who understood the models and their limitations.
 
I'm surprised that people need to be told that the models would not be perfect/exact and that some may be wrong. People really thought we had 100% understanding of a virus that the world had never seen within months?

On second thought, why am i surprised? There are people that think the world is flat.
 
PeterNSteinmetz said:
Commentary by Ioannidis at Stanford on the lack of accuracy in Covid-19 models.

Since we have a lot of other Covid-19 threads, I would suggest this one be used for discussions of the models and the accuracy/inaccuracy.

https://forecasters.org/blog/2020/06/14/forecasting-for-covid-19-has-failed/
Click to expand...
Most models seem to fail; I don't see my 0.5 meter rise in sea level, at a place I've visited for nearly half a century, either.
But in the case of COVID-19, there were as many models as you could want out there. I suppose the one that is closest "wins"?
 
NoBShere said:
I'm surprised that people need to be told that the models would not be perfect/exact and that some may be wrong. People really thought we had 100% understanding of a virus that the world had never seen within months?
Click to expand...

Sometimes of course the model outputs were implicitly embedded in other things people were told.

For example, most of the claims which politicians have made that their interventions reduced Covid-19 cases or deaths were based on comparing a model output of higher cases and deaths than observed and a model prediction of higher cases or deaths if an intervention term was dropped. That is different than a statistical comparison of two observed cases, one with and one without the intervention. But most people likely did not understand that the claimed advantage was just a model output and not a statistical comparison.
 
PeterNSteinmetz said:
Sometimes of course the model outputs were implicitly embedded in other things people were told.

For example, most of the claims which politicians have made that their interventions reduced Covid-19 cases or deaths were based on comparing a model output of higher cases and deaths than observed and a model prediction of higher cases or deaths if an intervention term was dropped. That is different than a statistical comparison of two observed cases, one with and one without the intervention. But most people likely did not understand that the claimed advantage was just a model output and not a statistical comparison.
Click to expand...
In many situations, there are not two outcomes to be empirically compared. But most people are simply stupid, and those that aren't tend toward ignorance (pointing out that for many, "opening up" meant "safe to resume normal operations". We're already seeing the hotspots from that attitude.)
 
It’s hard to even determine whether the models have failed because we know that there is no good data on cases and deaths. Florida fired the person compiling their data, we know that nursing homes were hiding deaths, the number of deaths over normal in states like New York indicate that COVID-related deaths are underestimated by 30-50%.

The models also presumed that we’d be testing by now, which we aren’t. So it’s hard to judge any models if you don’t have good data.
 
NoBShere said:
On second thought, why am i surprised? There are people that think the world is flat.
Click to expand...

Simple way to disprove those people. If the world was flat cats would have knocked everything off the edge by now.

Another problem with the models is the people who push them as the basis of their pronouncements have changed the target over time. A couple months ago the goal of all the social distancing, etc. was to "flatten the curve" so the ICUs in the hospitals wouldn't be overwhelmed. Done, with the side effect that hospitals have been put in financial difficulty due to all the other things that aren't happening in the meantime. Now the same people (politicians, typically) are saying that we need to do the same things to stop the spread until we have vaccines, etc. Once we have those, what will be their excuse to continue being dictators? I'm sure they'll find something. Just watch King Jay in Washington.

The point is that the models (which have vastly overstated the problem in the past) are not explained, even to those of us who might understand the weaknesses contained in them, but are instead put forward as being the accurate representations of the future. Most of us know better, but the people pushing their actions based on them (again, mostly politicians) just say "trust the science" and keep on acting like dictators. And many of us are getting tired of that act. Yes, COVID-19 is serious, but how serious? I can't tell from all the nonsense that I hear from politicians and the media.
 
Models must account for human behavior, and that is extremely difficult to quantify as peoples' behavior is influenced by constantly changing pandemic dynamics (infection rates, hospitalizations, ICU utilizations, and fatalities), changing understanding of the SARS-CoV-2 virus, and varying responses to the pandemic in different geographical regions as related to use of masks, social distancing, frequent handwashing, keeping hands away from face, etc.

There are just way too many variables to have accurate predictive models. Several models are predicting 200,000 deaths by September 1st. Let's see how accurate that forecast is.
 
Larry Vrooman said:
It's incredibly hard to get people to wear masks, and yet if everyone wore a mask transmission rates would go way down, . . . .
Click to expand...
There's no reason to believe this is true. Until mid-May, the official word was that masks did nothing for the general public. Then one day, the president and Fauci said, "If it makes you feel better to wear a mask, go ahead." Not 48 hours later, the usefulness of masks was something "everyone knows."

Stan Cooper said:
Several models are predicting 200,000 deaths by September 1st. Let's see how accurate that forecast is.
Click to expand...
Presumably, it'll be more accurate than the estimated 2.2 million from February that justified everything since then.
 
OneCharlieTango said:
Presumably, it'll be more accurate than the estimated 2.2 million from February that justified everything since then.
Click to expand...
That estimate was a very early "worst case" scenario by the British Ferguson/Imperial College report which assumed that 81% of the population in Britain and the U.S. would be infected over the course of the epidemic if no attempts to mitigate the spread of the infection were taken.

How One Mode Simulated 2.2 Million U.S. Deaths from COVID-19
 
Stan Cooper said:
...Several models are predicting 200,000 deaths by September 1st. Let's see how accurate that forecast is.
Click to expand...
It will be as accurate as how the deaths are counted, which can be 50 ways from Sunday.

Without being facetious, a certain individual that has recently died by the hands of another is reported to have had the virus in the autopsy report.

That death may get counted as a COVID death without regard to the manner of death.
 
Last edited:
TCABM said:
It will be as accurate as how the deaths are counted, which can be 50 ways from Sunday.

Without being facetious, a certain individual that has recently died by the hands of another is reported to have had the virus in the autopsy report.

That death may get counted as a COVID death without regard to the manner of death.
Click to expand...
That's quite a leap; maybe somebody can dig up a copy of the death certificate. There are some pretty outrageous claims about how Covid-19 deaths are counted for political purposes, but @steingar is right in that it cuts both ways. Most of the evidence of miscounts is anecdotal.

EDIT - The autopsy report case title is "CARDIOPULMONARY ARREST COMPLICATING LAW ENFORCEMENT SUBDUAL, RESTRAINT, AND NECK COMPRESSION." The SARS-CoV-2 virus infection is mentioned in passing with a notation that the deceased was probably asymptomatic. The complete autopsy report is 20 pages long.

https://www.scribd.com/document/464472105/Autopsy-2020-3700-Floyd#download
 
Last edited:
Everyone I know has gone into full blown who cares it’s survival of the fittest. Americans have insanely short attention spans that make goldfish look good plus a culture of promoting individual freedom. Aka short of Ebola level death people won’t listen or care.

The models are only as accurate as the data it’s fed. For us to have a model that was even one sigma in accuracy we’d basically need mandatory testing of EVERYONE regularly. That will never happen. So our data is limited to people who chose to get tested. Which means that group is only a percentage of the whole infected group. Due to this our data is horribly skewed. So the best hope is to fit this data to other virus models and throw out anything with a deviation of more then 3. Which is what we are doing. It’s barely better then throwing a dart.
 
steingar said:
These sorts of models are fantastically difficult to do, since the data can change markedly by the time it gets keyed into the computer. And day to day conditions can change do to changes in people's behavior should they see models. COVID19 would be stopped dead in its tracks if EVERYBODY stayed home for two weeks.
Click to expand...

No it wouldn’t. Takes longer than 2 weeks.

We know this because we had a ship (USS KIDD) that was over 30 days since it’s last port visit when it developed the first symptoms that then ran throughout the crew.

The idea that we could solve the problem by having everyone just stay home for 2 weeks and then resume life is a fallacy.
 
Greg Bockelman said:
It would be a whole lot more palatable to me to wear a mask if I knew there was an end game to this. As it is, apparently this is a forever deal. THAT I do not accept.

As for me,I would rather catch the thing and get it over with and be done with it.
Click to expand...

Here is a bowl full of M&Ms. One of them could kill your or at least make your life a living hell for over a month.

Do you eat the M&Ms?
 
Fearless Tower said:
No it wouldn’t. Takes longer than 2 weeks.

We know this because we had a ship (USS KIDD) that was over 30 days since it’s last port visit when it developed the first symptoms that then ran throughout the crew.

The idea that we could solve the problem by having everyone just stay home for 2 weeks and then resume life is a fallacy.
Click to expand...
But they weren't isolating from each other, were they? Some could have been asymptomatic. I'm not sure how long it would take, and be impractical/impossible logistically anyway, but it seems that people would need to be alone or in very small groups, not onboard a ship where people are freely associating.
 
Fearless Tower said:
Here is a bowl full of M&Ms. One of them could kill your or at least make your life a living hell for over a month.

Do you eat the M&Ms?
Click to expand...
I would rather take my chances with the M&Ms than to live my life in fear of them.

I am reasonably sure that Covid isn’t going to kill me if I catch it.
 
Everskyward said:
But they weren't isolating from each other, were they? Some could have been asymptomatic. I'm not sure how long it would take, and be impractical/impossible logistically anyway, but it seems that people would need to be alone or in very small groups, not onboard a ship where people are freely associating.
Click to expand...

No, because no one knew anything was wrong until symptoms showed up.

And that’s the problem with this 2 week isolation idea. If a family of 4 or 5 gets unknowingly exposed and then goes into quarantine, due to the possibility of asymptomatic cases and the timeline from exposure to positive case, you would have people emerging from isolation two weeks later thinking everything is fine and back to normal and resume the spread.
 
Status
Not open for further replies.
Back
Top