5X5 Episode 112

This episode needs:  proof-reading,  categories,  categorized redirect.

How to Contribute

5X5 Episode 112
Anecdotal Evidence
2 May 2012
5X5 111	5X5 113
Skeptical Rogues
S: Steven Novella
R: Rebecca Watson
B: Bob Novella
J: Jay Novella
E: Evan Bernstein
Links
Download Podcast
Show Notes
Forum Topic

Anecdotal Evidence

Voice-over: You're listening to the Skeptics' Guide 5x5, five minutes with five skeptics, with Steve, Jay, Rebecca, Bob and Evan.

S: This is the SGU 5x5 and tonight we're talking about anecdotal evidence. Anecdotal evidence is a causal observation, one that is not done under any strict scientific protocol. There are extreme limitations to using anecdotal evidence as evidence, and it does not qualify as scientific evidence for these reasons. For example, anecdotes can not be statistically analyzed, because they are not gathered in a systematic or a thorough way. Therefore they're subject to so-called cherry picking. People may remember or just point to those anecdotes that tend to support something that they wish to believe in. This leads to what we call confirmation bias, selecting only confirming evidence, and dismissing or forgetting disconfirming evidence. There are lots of logical fallacies inherent in anecdotal evidence as well, such as the hasty generalization. Because anecdotes are not controlled or systematic, there is no way to know if anecdotal experiences are typical. Therefore, they, trying to generalize from an anecdote is a form of hasty generalization. So, anecdotal experience is not the same as scientific evidence, because a scientific study endeavours to capture all of the information on a relevant question systematically, so that it is possible to do statistical analysis, outcomes are randomized and efforts are made to make sure that there is a representative sample, so that the outcome can be generalized to whatever the population in question is. In short; scientific studies allow for the control of variables, controlling for possible confounding factors, whereas anecdotal experience does not. It's quirky, it's biased, and it is not therefore a source of reliable evidence.

R: That said, there are instances in which a particular type of anecdotal evidence can be useful to science. For instance what we call case studies. Case studies are in-depth analyses of specific events, taking into account the context in which those events happen. Now, this is not in any way to be considered, for instance, a scientific study on its own. However can be useful in for instance falsifying certain hypotheses. Karl Popper, a famous philosopher of science, came up with a famous example, in which he said "all swans are white" and then proposed that one single observation of a black swan would be enough to falsify that proposition, and that observation of the black swan would be the case study. A case study is very good for finding those black swans and detailing them, and that type of case study can be very helpful in forming a hypothesis, from which you can then go and do more scientific research using more evidence that you can gather.

S: Yeah, I think a good way to summarize the role of anecdotes in science is that they are useful for generating hypotheses, they are just not useful for testing hypotheses, for that you need scientific data.

Unknown: Steve, another problem with anecdotal evidence has to do with the fallibility of human memory. People are innately poor historians, our memories have limitations and over time our memories tend to decay, or warp, or merge with other memories. In the case for example of a person suffering from some sort of medical illness, anecdotes can become contaminated with false memories and exaggerations due to the sensitive and deeply personal nature of the experience. There's a tendency for details to evolve over time and make a story sound more clean and profound. So in a patients own memory, they might exaggerate certain things, such as the severity of the symptoms prior to the treatment or exaggerate the response to the actual treatment.

S: SGU 5x5 is a companion podcast to the Skeptics' Guide to the Universe, a weekly science podcast brought to you by the New England Skeptical Society in association with skepchick.org. For more information on this and other episodes, visit our website at www.theskepticsguide.org. Music is provided by Jake Wilson.