Ben Goldacre of Bad Science wrote an excellent piece yesterday about anecdotal evidence. He wrote about the media coverage of the British National Health Service’s (NHS) decision to not cover a very expensive stomach cancer drug. The media all told the story of one woman who lived FOUR YEARS longer than expected, most likely due to taking this drug. Unfortunately, the large randomized trial (1401 patients) of the drug’s efficacy showed that the average patient on the drug only lived an average 6 months longer than those who received the placebo. The drug costs £21,000 (approx. $32,592 USD), and the patient is likely to only live an average of six months longer. The one woman in the story is an anomaly, but it makes good press.
I love the saying:
The plural of anecdote is anecdotes, not evidence.
(which may have come from Frank Kotsonis’ “The plural of anecdote is not data”)
Wikipedia has a nice couple of descriptions of anecdotal evidence:
- Evidence in the form of an anecdote or hearsay is called anecdotal if there is doubt about its veracity; the evidence itself is considered untrustworthy.
- Evidence, which may itself be true and verifiable, used to deduce a conclusion which does not follow from it, usually by generalizing from an insufficient amount of evidence.
We use anecdotal evidence in the form of personal experience all the time. We bend rules when they don’t make sense to us or when we think they shouldn’t apply, we gamble in spite of the odds, we do dangerous things like ride motorcycles, go sky-diving and do back flips off of diving boards. Some of us take herbal supplements because we swear we feel better or get over colds faster than if we don’t, even though all the evidence points to the contrary.
But if we want to make informed, evidence-based decisions, we have to stick to the…umm…evidence. We have to ignore our gut feelings because homo sapiens is really bad at making good decisions based on instinct and emotion.
Brian Dunning recently did a Skeptoid episode about these fallacies, as well as a few others. The link will take you to the podcast’s written transcript, but you can also download this episode on iTunes (episode #217).