Texas Sharpshooter Fallacy
The Psychology Behind It
This fallacy is a combination of the clustering illusion and confirmation bias. It happens when we define the rules of success after we see the data.
In science, you are supposed to state your hypothesis ("I think X causes Y") and then collect data to test it. In the Texas Sharpshooter fallacy, you collect the data first, look for any correlation (e.g., "Look, people who eat jellybeans have less acne!"), and then pretend that was your hypothesis all along. By drawing the target after the shots are fired, you guarantee a bullseye, but the bullseye is meaningless.
Real-World Examples
Epidemiology
A classic example involves power lines and cancer. Researchers looked at hundreds of ailments and found that leukemia was slightly higher near power lines. They published this "finding." However, because they checked so many ailments, one was bound to be high by chance. Subsequent studies failed to replicate the link.
Psychics and Prophets
Nostradamus's quatrains are vague. After a major event (like 9/11), people scour his writings to find a verse that vaguely fits and claim he predicted it. They are painting the target around the event after it happened.
Customer Success Stories
A company claims their training program creates millionaires. They point to 5 students who became rich. They ignore the 5,000 students who took the same course and went broke. They drew the target around the winners.
Consequences
The Texas Sharpshooter fallacy can lead to:
- False Medical Cures: Believing a quack remedy works because of a few anecdotal success stories.
- Bad Investments: Buying a fund because it had a "hot streak," not realizing the manager was just one of thousands of monkeys flipping coins.
- Policy Errors: Creating laws based on statistical anomalies rather than robust trends.
How to Mitigate It
We must demand prediction, not post-diction.
- Pre-Registration: In science, researchers now register their hypothesis before collecting data to prevent this fallacy.
- Look at the Denominator: When someone shows you the "hits" (the cluster), ask to see the "misses" (the rest of the barn wall). How many shots were fired in total?
- Replication: If a pattern is real, it should appear in a new dataset. If you can't replicate it, it was likely just a Texas Sharpshooter target.
Conclusion
Randomness looks like order if you are allowed to draw the boundaries wherever you want. The Texas Sharpshooter fallacy teaches us that finding a target is easy; hitting a pre-defined target is hard.