5X5 Episode 64

5X5 Episode 64
The Clever Hans Effect
2nd April 2009 Transcript Verified
5X5 63	5X5 65
Skeptical Rogues
S: Steven Novella
R: Rebecca Watson
B: Bob Novella
J: Jay Novella
E: Evan Bernstein
Links
Download Podcast
Show Notes
Forum Topic

The Clever Hans Effect

Voice-over: You're listening to the Skeptics' Guide 5x5, five minutes with five skeptics, with Steve, Jay, Rebecca, Bob and Evan.

S: This is the SGU 5x5 and we are talking about the—

J: (exaggerated accent) Clever Hans!

S: —the Clever Hans effect.

This term refers originally to a trained horse named Kluge Hans referred to in the literature as "Clever Hans" who was taught by his master, William von Osten, to do math and to report on this math. So, William would ask the horse—ask Hans—to add 3 plus 2 and Hans would clop his hoof five times—

E: Ooh.

S: For example. This made it seem as if the horse understood not only the spoken word but understood math—was able to do mathematical calculations on his own—unprecedented in the animal kingdom. Several scientists reviewed this demonstration and declared it genuine. However, it turns out that when the proper controls were put into place, namely that the people who knew the correct answer to whatever the question Hans was being told were not within sight of Hans, Hans could not come up with the correct answer. His performance was later to be found to be due to the Ideomotor effect or to subtle cueing. What William von Osten was doing was, after he asked the question of Hans, he would stand a certain way, tilt his head a certain way. Hans recognised that as his cue to begin clopping when he got to the right answer, then William would make another gesture; he would stand up or nod his head; he would give some indication that Hans had gotten the answer correct, so the horse would stop clopping.

B: All involuntary, by the way.

S: Yes, it's very important to notice that it was all completely involuntary.

E: Right.

J: Meaning that his trainer really wasn't trying to signal the horse to tell him to stop doing it.

R: His trainer genuinely thought that he had an amazing math-doing horse.

E: He did not realise that he was giving off these subtle signals.

J: So, you know, it actually... to me it's still pretty cool that the horse was able to pick up on those subtle movements that other people couldn't right away and eventually, of course, they did figure it out, but—

R: I think that's why they still call him Clever Hans, 'cause he really was clever.

S: As opposed to "Mathematician Hans"!

E: That's right. And there are other examples of this same phenomenon that have occurred in... when scientists have been studying animals and their behaviours, such as the Nim Chimpsky project. So Nim was a chimpanzee and this project was studied in the 1970s.

R: Nim Chimpsky!

B: Which is, of course, a play on words with Noam Chomsky, which is a famous linguist.

E: Thank you, Bob.

R: Yeah, you know what, Bob? It's not as funny when you explain it!

E: So there was scientific interest in primate sign language training and they thought that this chimpanzee named Nim had abilities of signing beyond other chimpanzees to the point of almost a language that was being developed. But, when they studied the research, what they realised what was happening is that the monkey was simply imitating, more often than not, the hand gestures and arm gestures of the trainer. And, in order for him to obtain some reward, 'cause he knew he was going to get a piece of food or something to that effect. And this is, again, another Clever Hans effect.

S: That's right. In fact, this kind of—the Clever Hans effect has plagued animal communication research throughout its history. It's very difficult to completely control for any subconscious or subtle cueing. There is also a lot of forced fitting going on with animal communication research. For example, in another effect in addition to just imitating what the human trainer was doing, it's also the chimpanzees, or in some cases, gorillas would do almost random signs until they got the result they were wanting. And the researchers who were interpreting the signs would use a lot of the same kind of confirmation basis techniques that even ESP researchers or others have fallen prey to, like ignoring the first couple of signs until they got ones that made some kind of sense. Or saying that sometimes the chimpanzee has a sense of humour because he will say things that make no sense. When he makes sense, he is communicating; when he doesn't make sense, he is making a joke. And if they have to ignore a couple of signs here or there in order for it to make sense that's OK too. So there is a lot of ways in which it can appear that an animal is communicating when, in fact, they may not actually understand syntax, understand language, understand the words but they are doing a learned behaviour in order to get a result from their trainers.

J: And one more thing, Steve: even when a researcher is consciously aware that they don't want to give a Clever Hans effect by unconsciously cueing the animal, a lot of times they can't even help themselves. They'll do it without even knowing it.

S: Yeah, absolutely. That's the key here, is that it's all unintended and subconscious. This comes into play in situations with people as well. I think there is a lot of overlap between the Clever Hans effect and what we were seeing with the facilitated communication phenomenon where facilitators were, through the Ideomotor effect, unconsciously controlling the hands of children who were non-vocal because of development delay or mental retardation. And they were convinced that this child was able to communicate with this method and they were completely unaware of the fact that they were doing all the communicating themselves. It's essentially the same kind of phenomenon.

R: Also, just today there was a news story—we are recording this on April 1st—and a couple of people sent me this and asked if it's an April Fool's joke. But there's an article on the BBC's website and the headline is, "Baby chicks do basic arithmetic", and basically researchers hid balls behind little screens and the chicks would always go to the screen with more balls hidden behind it when they saw them placed back there. And it's kinda of compelling, but at the same time it's pretty light on the details, so you really have to wonder whether or not everything has been controlled for because it could be yet another Clever Hans but in adorable chick form.

S: Right, essentially whenever you see any study of animal behaviour like that, the first thing you think to yourself: are they controlling for Clever-Hans-type effects. If not, in my opinion, the research is almost worthless.

S: SGU 5x5 is a companion podcast to the Skeptics' Guide to the Universe, a weekly science podcast brought to you by the New England Skeptical Society in association with skepchick.org. For more information on this and other episodes, visit our website at www.theskepticsguide.org. Music is provided by Jake Wilson.