Confirmation Bias

All of us who read & write primarily in the non-scientific sub-spheres of the blogosphere are likely frequent victims of confirmation bias*… and when the subjects are veering more toward politics, the effect is likely enhanced.

This almost went in with the eye boogers – where, if you’re reading, you may have noticed that there isn’t much air left in the “climategate” balloon. But, since I have really, honestly, hope to die, not had time to write anything here lately, I thought I’d take a minute to actually add a few words to go along with this link to miller’s explanation that confirmation bias is the root of the “controversy”.

So, let me add my caveats… miller simplifies the meat of the controversy way too much for you to get an idea of what it’s all about, and in doing so maybe glosses over some actual guilt on the part of some of the folks that swapped e-mails at Anglia. So, if you still haven’t figured out what went on, and how much real controversy exists, then read around (and don’t skip the eye boogers). That’s number one. Number two is, don’t be foolish enough to judge the state of climate science based on the amount of controversy at CRU. In fact if these folks had been caught killing kittens and manipulating data, there would still be a universe of data out there on which to measure the state of climate science. I guess that confirmation bias might ask a skeptic to make more of the kitten-killing and data-manipulation than is merited based on its overall impact on the state of climate science, but that’s what this post is about.

Ok… so about miller’s post. He sums it up just fine – confirmation bias is operative. My only elaboration on that is to remind people reading here who may have heard about the effect in a psychology class 20 years ago about it and some related terms. Confirmation bias is a type of selective attention. The mind can only focus on a certain number of things at once, and it has mechanisms in place for choosing what’s “important” to process in front of other, hopefully less useful information. That’s selective attention. It’s why a healthy person will notice a lion charging toward them, and completely miss the fact that one of the clouds near the horizon… just above the lion, in fact… resembles a Peanuts character. The mechanisms for selective attention apparently rely in part on what we have experienced in the past – and what has given us a payoff.

So, if we’ve been well-served to notice whether that stove eye is glowing red more than whether the wall-paper is blue… then next time you’re in the kitchen, you may be more inclined to notice the stove.

Now, I am no more an expert on neurology than miller is on climate change, but my understanding (which I encourage you to take with a grain of salt) is that the rewards we get for noticing the stove eye is chemically mediated. So, when we notice the stove eye in time to keep from getting burned, we get a little shot of dopamine (or something) that makes us feel good and cements the importance of the thing we’ve noticed in our mind.

Now this is the part that I’m really sketchy on… but it also feels good to notice something that works with what we already think or believe. That makes sense… if we have been living successfully in the real world for a while, we think or believe a lot of things that have been helpful to us in managing that. When we learn something new that confirms those beliefs & thoughts, chances are it is useful.. and we don’t have the displeasure of having to unlearn something we thought we knew. So, if we’ve come to the conclusion that Whiskey Egg-Nog is nutritious for some reason… we may be less apt to pay attention to the labels on the ingredients and more attention to nice articles like this. When we ignore the well documented and publicized health risks associated with whiskey… and egg-nog… in favor of the reports that drinking (in moderation) might help the heart – it’s because we have two motivations – 1) we like our Whiskey Egg-Nog, and 2) it confirms a belief we already hold.

However mistaken I may be about how it is mediated, I can say with confidence that confirmation bias is a part of our psychology. You can see it in action just about anywhere you turn (and yes, if you draw too large a conclusion from this fact of observation you are guilty of confirmation bias). But I would be remiss in any case if I failed to give the classical example of confirmation bias, and to remark that I know people who actually hold the incorrect belief based on their indulgence in it in just this way. I also know plenty that know it is a classical example of confirmation bias and who manage to avoid the false belief – and I am among them. That doesn’t mean that I or any of these people are immune. In fact, I have some concrete and heinous examples in mind where I went through long periods of time with worse incorrect beliefs largely held because of my own failures to avoid confirmation bias. So, the classical example, changing the names to protect the guilty-as-hell:

Lucy: Must be a full moon tonight. They’re acting crazy at work.
Me: Ha ha!
Lucy: I don’t know what it is about a full moon.
Me: Ummm. You know that isn’t true, right?
Lucy: I know it is right. I don’t know how many times I’ve noticed how crazy people were being and I checked and it was a full moon.
Me: Yeah, but…
Lucy: Oh shut up, and let me tell you about the HIV needles people put in the change slots in those pay phones everywhere.

So, Lucy has discovered a fact of nature, by noticing on a number of occasions the correlation between craziness and the moon. She knows enough to realize that one needs a fair amount of data to support such a correlation, but she figures she’s noticed it enough that doing the math isn’t all that important. But she’s wrong. If she only knew about confirmation bias, then she would realize that she as only “documenting” in her memory the cases in which 1) she checked, 2) and it was a full moon. She doesn’t remember the times that she didn’t check (which would include a fair number of full moons), naturally. More importantly, she doesn’t remember the times when she said, “well, huh – no full moon tonight after all. Must be some other kind of craziness”. Because that isn’t nearly as interesting as the times when she gets to say “hey! it’s happening again!” That’s the confirmation bias.

Of course other stuff plays into it. “Everybody knows” is a fallacious piece of evidence she might appeal to if she hadn’t told me to shut up & we’d had a debate about it. It isn’t only confirmation bias involved when the moon was *almost* full, so she counted it. Maybe she didn’t pay enough attention to realize that it was actually only gibbous. Or maybe she realized it was not quite completely full, but decided that it was close enough. Yes, in part it was confirmation bias that caused that, but it was also just sloppy and poorly quantified research.

So, that’s confirmation bias.

Now, I wrote all that to say this: science as it is done in the modern world (and when it is done right) is, among a few other important things, very much the process of eliminating mistakes that arise because of effects like confirmation bias. Researchers share their methodologies when they write papers for a number of good reasons – so that experiments can be repeated and verified for instance. But one of the reasons is so that editors, referees, or other scientists can glare over them and discover the fact if there is room for error to slip in through a poorly designed experiment.

I don’t think you can exaggerate the importance of this distinction between what happens (or usually does) in science and what happens with non-scientific styles of discourse and thinking. Without strict methodological checks on how ideas can be confirmed or denied – checks that require repeatable observations which are carefully controlled and quantified to be certain that they mean only and exactly what we think they mean – just about any idea can be justified, confirmed, and believed deeply.

There’s another post in here. It’s about how science education should work when, of necessity, people who aren’t rigorously trained in the methods are tasked with communicating the results to the public who also aren’t situated with the leisure to closely examine the data and methods. It’s about how scientific debate is not a mass media enterprise. It’s about how people with a political stake shouldn’t try to create a shadow debate in the mass media, and how the mass media and their audiences shouldn’t let them. But I’m probably not the person to write it. At least not today.

*RW – not sure if confirmation bias had anything to do with me posting the link to CJ’s flub on Breitbart & Hansen. I didn’t trouble myself to check that context thoroughly – which is just as bad, if not part of the process of confirmation bias. Thanks for setting me straight on that. And, I have noticed that CJ has started to get a little wild-eyed himself. Which is weird, since his divorce from the right was largely participated by the same stuff from them.