The Survivorship Bias Is What Convinces People That Psychics Are Real
The Science
How often does it happen in science? Without a doubt some people pull (consciously or not) the same trick. Drug companies, when they submit drugs for testing, have to register all tests of a drug. Some companies got in the habit of sponsoring multiple tests, and only showing the results of those that were up to snuff. Not that Big Pharma is the only industry that pulls this.
There are plenty of people who believe that natural remedies work. They have the studies to prove it, they say. And they’re right. It’s just that they’re either ignoring, or have never seen, the many other studies that prove their natural remedy has no effect. If you do a hundred studies on whether wheat grass juice stops migraines, one of them might have striking results. That doesn’t actually make wheat grass juice effective.
Which is why a paper, Why Most Published Research Findings are False, made waves. There are a lot of ways to unintentially apply survivorship bias. Widen the criteria for certain effects, leave them open to interpretation, use too small a selection pool, or too wide a selection pool with too many criteria. Eventually, one test is going to show something that is a coincidence and not reliable results.
Even publishing itself can skew results. Few people are going to feel the need to go public with a title like, “Tried it. Didn’t Work.” The only way for that title to get attention is if it followed a paper called, “Tried it. Did Work.” And just like that, you get multiple results, that people can interpret in multiple ways.
More: The Survivorship Bias Is What Convinces People That Psychics Are Real