Ferguson said 'my bad' when revising the model down 2200% 2 months ago. Too bad we're all being held at gun point to 'follow the science' from then and not the science from today.
Can anybody tell me now, the difference between the white robed scientists that pull "scientific conclusions" out of thin air to move masses into compliance, and the black robed priests who used "gods words" out of thin translations to move the masses into compliance?
Dont they both just play the same game?
Control source of info and control translation of sources info.....
You can use non-deterministic simulations to make scientifically assisted predictions and the modelling would be valid at least in principle. Doing things right doesn't guarantee a correct result however.
In this case though they are describing dramatic differences in outcomes which is problematic.
It's not that it's technically non-deterministic but that it is greatly so.
My guess is they will say he ran it a thousand times and took the average from the results. Sometimes that's valid but more often people just do it with the assumption it fixes the problem. A rookie mistake. They assume consensus will find the right answer as is by magic.
These programs don't even replicate results when run from the same starting point.
This is not because of incompetence; it is by design. In fact, if they did not explicitly introduce chaos (in the form of one or more randomly-generated variables), they would produce the same result from the same seed each time, and that would defeat the purpose.
Yup, my experience with Monte Carlo simulations is in finance, specifically the Trinity Study. As I understand it, with these kinds of simulations, they say that xx% of the time it will yield a certain threshold result. For example, in the Trinity Study, it's safe to withdraw 4% of one's portfolio every year and there is a 95% chance the portfolio will survive one's entire retirement.
In the case of the IMHE model, it was framed as 100% chance and without any thresholds. That doesn't sound right to me.
As someone who gets paid to write code, I was disturbed at the lack of polish such a codebase used for driving public policy had. Makes you wonder how well constructed climate change models are...
Granted, it was only used for research purposes, but that's an easy excuse anyone can keep making right until the day you can't.
It's worse than that. One input variable into this garbage code is R0. Imagine being able to tell the model exactly how infectious you want the disease to be in the model. BOOM! 2.2 MILLION USA DEATHS!
To think that supposedly smart people bough into this BS is the most infuriating part. But of course, unelected government 'fauxicials' can't ever be wrong.
I read the synopsis and looked over a few thousand lines of the cleaned up code.
The analysis includes digging through revision history which is useful for understanding what the developers were trying to do versus what they did do. There is good insight in looking at it that way.
The code I read, on its own, is a horrific mess. Not unusual, mind you, but still a mess.
Well-written code is structured to reflect what it is doing. Well-written code is compartmentalized, separated into many, many small pieces that are easy to read and easy to understand. This has many advantages, one of which is to allow new programmers to add or change functionality quickly and without misunderstanding side effects they don’t intend.
In just a simple perusal of the code, I noticed an if statement that checks for airports. If there are none, then it sets the number of hotels to zero. Imagine being somewhere else in the code and trying to add some simulation that relies on the number of hotels and your code has no effect even though you keep changing the input of the number of hotels. If you didn’t change the code that reads in the number of airports, then you are going to spin your wheels and make mistakes. If you do find the airport code, then you are going to try and change the airport code for your needs while trying not to break the reason why that was put there which is not what you are working on and don’t know how to test. The result after you touch it will be a bit worse for the next programmer that comes along. The way this code reads, this has happened to it a lot of times.
Suffice it to say, this code is not production quality, let alone valid for making policy decisions on.
Agreed, and also looked at some of the code. It isn't even alpha quality. It's hard to imagine that's the improved, refactored version.
They closed the github issues thread where people were asking for the original code (the one file version) a week ago when they realized people weren't really buying their excuses for not making it available.
In just a simple perusal of the code, I noticed an if statement that checks for airports. If there are none, then it sets the number of hotels to zero.
With as many smart people as we have in the world, why would we ever rely on just one model?? Even college football has hundreds if not thousands of prediction models.
Looking at a single file just enrages me as a programmer. 300 line for loop with no dependency injection is complete heresy. The person in charge of that abomination should be beaten with every book that mentions design practices.
So essentially the results of the modeling program used to whip the world into a panic were multiplied by an extremely random seed that varies based on cpu tick rate and was run until they got an extremely large and unrealistic number? Color me shocked.
TFW you'd get more accurate predictions playing Plague Inc.
I've already heard the argument "That's not the same code. It was changed by Microsoft!" And not in a "they fixed some of the many bugs and cleaned up the code" way, but in a "that's why this release doesn't work right" kinda way.
I'm not defending this shit-tastic code or its equally shit-tastic fake prediction. But you should know there's a thing called a Monte Carlo simulation. The simple description of it is to set up a simulation with (sensibly) randomized inputs and then to run the simulation a million times, and once you've got the results, look to see if they converge on an answer or have an otherwise identifiable pattern.
Indeed, that's one reason this code is shit -- the way they re-order one of their internal lists during execution is non-deterministic (ie probably skull fucked by the os thread scheduling). Nevertheless, assuming an acceptably-random value is generated whenever your rand() function is called, it's worth pointing out that the whole point of a Monte Carlo simulation is to show that a model converges on an answer or a pattern of answers, regardless of what random values are used from pass to pass. In this case, the failure to maintain determinism is more of a bug than it is an inherent design flaw, and ultimately has no bearing on the accuracy of the answer.
If the code is the same and the inputs are the same including the random seed you should get the exact same outputs.
In this code review it notes that if you run it on a single CPU it behaves correctly, if you enable multi-threading then it's non-deterministic which indicates a pretty serious error.
It's settled science, you bigot!
No, just kidding... I am furious at these people. There needs to be consequences for actions like this.
Ferguson said 'my bad' when revising the model down 2200% 2 months ago. Too bad we're all being held at gun point to 'follow the science' from then and not the science from today.
I'm starting to think there is some kind of agenda.
Can anybody tell me now, the difference between the white robed scientists that pull "scientific conclusions" out of thin air to move masses into compliance, and the black robed priests who used "gods words" out of thin translations to move the masses into compliance?
Dont they both just play the same game?
Control source of info and control translation of sources info.....
Be careful with that edge. Whew lad.
You can use non-deterministic simulations to make scientifically assisted predictions and the modelling would be valid at least in principle. Doing things right doesn't guarantee a correct result however.
In this case though they are describing dramatic differences in outcomes which is problematic.
It's not that it's technically non-deterministic but that it is greatly so.
My guess is they will say he ran it a thousand times and took the average from the results. Sometimes that's valid but more often people just do it with the assumption it fixes the problem. A rookie mistake. They assume consensus will find the right answer as is by magic.
Correct.
Non-determinism in algorithms is okay; machine learning and Monte Carlo simulations do it all the time.
Forming conclusions and acting upon unstable (and buggy) non-determinism is not okay, and that's what happened here.
This is not because of incompetence; it is by design. In fact, if they did not explicitly introduce chaos (in the form of one or more randomly-generated variables), they would produce the same result from the same seed each time, and that would defeat the purpose.
Yup, my experience with Monte Carlo simulations is in finance, specifically the Trinity Study. As I understand it, with these kinds of simulations, they say that xx% of the time it will yield a certain threshold result. For example, in the Trinity Study, it's safe to withdraw 4% of one's portfolio every year and there is a 95% chance the portfolio will survive one's entire retirement.
In the case of the IMHE model, it was framed as 100% chance and without any thresholds. That doesn't sound right to me.
You are assuming altruistic motivations.
Do you think this code would have been any better if Ferguson weren't thinking with his dick while writing it? I have my doubts.
As someone who gets paid to write code, I was disturbed at the lack of polish such a codebase used for driving public policy had. Makes you wonder how well constructed climate change models are...
Granted, it was only used for research purposes, but that's an easy excuse anyone can keep making right until the day you can't.
Someone posted awhile ago about a model they built for the CDC for predicting disease spread.
They didn’t want it.
It's worse than that. One input variable into this garbage code is R0. Imagine being able to tell the model exactly how infectious you want the disease to be in the model. BOOM! 2.2 MILLION USA DEATHS!
To think that supposedly smart people bough into this BS is the most infuriating part. But of course, unelected government 'fauxicials' can't ever be wrong.
on top of that, according to the article, Microsoft tried to cover up the "bugs" in the program before releasing it. More Gates fuckery. I am furious.
In Wuhan in the first few weeks, when saying anything was wrong at all was a Racist Conspiracy Theory™, the calculated R was above 8.
In South Korea, we were looking at an R of about 6 in Daegu, and something below 4 in the rest of the country.
In the US, the R has been declining. Six weeks ago, the median was 1.1, today it's 0.9.
Any virus-modeling software that did not offer the option to input different values for R0 would have been worthless.
Theres a great write up on the code here
https://lockdownsceptics.org/code-review-of-fergusons-model/
The program was literally using the R0 as an input AND and output in the model! Insane!
I read the synopsis and looked over a few thousand lines of the cleaned up code.
The analysis includes digging through revision history which is useful for understanding what the developers were trying to do versus what they did do. There is good insight in looking at it that way.
The code I read, on its own, is a horrific mess. Not unusual, mind you, but still a mess.
Well-written code is structured to reflect what it is doing. Well-written code is compartmentalized, separated into many, many small pieces that are easy to read and easy to understand. This has many advantages, one of which is to allow new programmers to add or change functionality quickly and without misunderstanding side effects they don’t intend.
In just a simple perusal of the code, I noticed an if statement that checks for airports. If there are none, then it sets the number of hotels to zero. Imagine being somewhere else in the code and trying to add some simulation that relies on the number of hotels and your code has no effect even though you keep changing the input of the number of hotels. If you didn’t change the code that reads in the number of airports, then you are going to spin your wheels and make mistakes. If you do find the airport code, then you are going to try and change the airport code for your needs while trying not to break the reason why that was put there which is not what you are working on and don’t know how to test. The result after you touch it will be a bit worse for the next programmer that comes along. The way this code reads, this has happened to it a lot of times.
Suffice it to say, this code is not production quality, let alone valid for making policy decisions on.
Agreed, and also looked at some of the code. It isn't even alpha quality. It's hard to imagine that's the improved, refactored version.
They closed the github issues thread where people were asking for the original code (the one file version) a week ago when they realized people weren't really buying their excuses for not making it available.
Lol
that was such an amazing breakdown, TY!
Oh, you mean like a certain "Hockey Stick" graph?
GIGO, it's as accurate as the climate models.
sauce https://en.wikipedia.org/wiki/Garbage_in,_garbage_out
With as many smart people as we have in the world, why would we ever rely on just one model?? Even college football has hundreds if not thousands of prediction models.
Fauci might as well be relying on a "Magic 8-ball."
"Unclear try again"
We need to shelter in place for 6 more weeks.
KEK
He couldn't read the code. His nose is on the way.
Mind=Blown
Oopsie!
"Did I do that?" In Urkel voice
Still not the actual source code. Actual is likely even [far] worse.
Looking at a single file just enrages me as a programmer. 300 line for loop with no dependency injection is complete heresy. The person in charge of that abomination should be beaten with every book that mentions design practices.
So essentially the results of the modeling program used to whip the world into a panic were multiplied by an extremely random seed that varies based on cpu tick rate and was run until they got an extremely large and unrealistic number? Color me shocked.
TFW you'd get more accurate predictions playing Plague Inc.
My H1B can do better than this... my fucking H1B can do better than this... for fucks sake.
Its kiddie fucking money laundering treasonous SWAMP science!
I've already heard the argument "That's not the same code. It was changed by Microsoft!" And not in a "they fixed some of the many bugs and cleaned up the code" way, but in a "that's why this release doesn't work right" kinda way.
Where was it said that this model was used to lock down the country? I missed it.
Wow. Just... Wow
I'm not defending this shit-tastic code or its equally shit-tastic fake prediction. But you should know there's a thing called a Monte Carlo simulation. The simple description of it is to set up a simulation with (sensibly) randomized inputs and then to run the simulation a million times, and once you've got the results, look to see if they converge on an answer or have an otherwise identifiable pattern.
Indeed, that's one reason this code is shit -- the way they re-order one of their internal lists during execution is non-deterministic (ie probably skull fucked by the os thread scheduling). Nevertheless, assuming an acceptably-random value is generated whenever your rand() function is called, it's worth pointing out that the whole point of a Monte Carlo simulation is to show that a model converges on an answer or a pattern of answers, regardless of what random values are used from pass to pass. In this case, the failure to maintain determinism is more of a bug than it is an inherent design flaw, and ultimately has no bearing on the accuracy of the answer.
If the code is the same and the inputs are the same including the random seed you should get the exact same outputs.
In this code review it notes that if you run it on a single CPU it behaves correctly, if you enable multi-threading then it's non-deterministic which indicates a pretty serious error.