Log in

Previous Entry | Next Entry

When David and I arrived at work last Wednesday, our HR manager was in a pretty foul mood. When David asked how she was, she answered "scared. We've just voted in a Muslim terrorist as President."

Now, Barack Obama is neither a terrorist nor a Muslim; in point of fact, he's a Christian and a long-time member of a Christian church affiliated with the United Church of Christ. But that's not really what I came here to talk about; in fact, I'm not really here to talk politics at all. I'm here to talk about what makes people believe outlandish things.

There's a really interesting two-part essay over on Slactivist about an enduring urban legend surrounding Proctor & Gamble, the company that makes laundry detergent and soap and whatnot. According to the urban legend, an unnamed officer of Proctor & Gamble appeared on some television talk show some years ago and announced that the company donates part of its profits every year to the Church of Satan.

As with all urban legends, the details are fuzzy and change over time. Sometimes, it was the president of the company; in other tellings, it was the CEO. Sometimes it was Oprah; sometimes, Phil Donahue. The name of the person who appeared on the show and the date the show aired are, of course, never given.

The interesting thing about this urban legend is its total absurdity. It's trivial to disprove; it can be demonstrated conclusively beyond even a single atom of doubt that it just plain never happened. Moreover, its utter absurdity would seem to suggest that no reasonable person could believe it.

The two-part essay is worth reading; you can find part one here and part two here.

The essay asserts that, in a nutshell, the folks who repeat this tale, which has been circulating since at least 1980 and possibly before, don't believe it's true; instead, they willingly pass on a story they know to be false, and only pretend to believe it's true. The author asserts:

Those spreading this rumor can be divided into two categories: Those who know it to be false, but spread it anyway, and those who suspect it might be false, but spread it anyway. The latter may be dupes, but they are not innocent. We might think of them as complicit dupes. The former group, the deliberate liars, are making an explicit choice to spread what they know to be lies. The complicit dupes are making a subtler choice -- choosing to ignore their suspicion that this story just doesn't add up and then choosing to pass it along anyway because confirming that it's not true would be somehow disappointing and would prevent them from passing it along without explicitly becoming deliberate liars, which would make them uncomfortable.

What I want to explore here is why anyone would make either of those choices. In both cases, the spreading of this rumor seems less an attempt to deceive others than a kind of invitation to participate in deception. The enduring popularity of this rumor shows that many people see this invitation as something attractive and choose to accept it, so I also want to explore why anyone would choose to do that."

I think this is a very interesting argument--that those who pass on the story know it to be false, since it would seem that the story is so prima facie ridiculous that nobody could really believe it.

But I don't think that's the case. I don't believe that the people who pass on this story know it to be false, but pass it on anyway, Instead, I think the real answer can be found in a comment posted after the end of the first part of the essay, and I've been chewing on it for weeks now. It offers, I think, a very useful insight into fallacy of all sorts.

The important bit, which caused something of an epiphany in my own understanding of the human condition, is this bit:

When an untrue story circulates, it's generally because it expresses some kind of social unease. There may not be razors in the Halloween apples, but it's a way of expressing the concern that your precious children are going out knocking on the doors of people who may not wish them well. There may not be rat poison in the Mars bars, but it's a way of expressing the sense that they're definitely not good for you. Not every Bridezilla story may be true, but it's a way of expressing the sense that the wedding industry is too high-pressured and perfectionistic. There may not be Satanic abuse going on at day care centres, but it's a way of expressing a sense of discomfort at women going to work and leaving their children in the care of others. And so on.

Human beings are inherently irrational. We carry around with us a kind of internal model of the way we make decisions: we are posed with a question, we think about the question, we evaluate evidence to support or refute each of the available options, and then we come to a conclusion. But that isn't how it works at all.

More often, the decision is made emotionally, on a subconscious level, long before we ever start thinking about it. After the decision has been made deep within the bowels of our emotional lizard brains, our higher-order, monkey-brain reason is invoked--not to evaluate the decision, but to justify it.

One consequence of this emotion-first decision-making process is confirmation bias, a process of selective evaluation where we tend to exaggerate the value of anything which seems to support our ideas, and to devalue or discard anything which contradicts our ideas. It's a powerful process that ends up making the decisions we've already made and the things we already believe all but immune to the light of disproof, no matter how compelling the disproof may be.

I've written before about how the brain is really not an organ of thought so much as an apparatus for forming beliefs, and how it has been shaped by adaptive pressure to b remarkably resilient at forming, and holding on to, beliefs about the physical world.

The adaptive pressures that gave rise to the belief engine within our heads don't necessarily select in favor of organisms that generate correct beliefs, for reasons I talked about there. But the beliefs that we form do serve a purpose, and sometimes, it's an emotional purpose.

The things a person believes can reveal a lot about that person's underlying emotional processes. Beliefs often reflect, in a garbled and twisted way, the buried perceptions and the emotional landscape of those who profess them. Even the most outrageous, clearly absurd beliefs can be quite sincere, and otherwise sane, rational people will adopt insane, irrational beliefs if those beliefs serve some emotional function.

Looked at in this way, a lot of patently absurd beliefs begin to make a kind of sense. They're distorted funhouse mirror projections of an underlying emotion, twisted out of all rational shape, and clung to through a powerful set of mental processes that make them seem attractive, even obvious.

The idea that Obama is a Muslim terrorist, ridiculous as it is, is an expression of an emotional state: "I do not identify with this person, he seems alien and strange to me, and I am afraid that he will not make decisions that reflect my needs."

The notion, sometimes seen in a few of the more extremist corners of radical feminism, that all heterosexual sex is always rape becomes an expression of an emotional state: "When I have had sex, I have felt disempowered and violated by the experience."

The idea that the government staged the attacks on the World Trade Center is a twisted-up, garbled expression of an emotional state: "I am afraid that my nation's government is corrupt and evil, and is willing to resort to any means, however extreme, to achieve its own ends."

This is why these beliefs are so vigorously resistant to debunking, even when the evidence against them is overwhelming. They are not assertions of fact in the way that many other statements are; they are assertions of emotional identity.

And they can not be treated as assertions of fact, even though on their face that's what they look like.

If someone says "New York city is the capital of New York State," that's an assertion of fact. It's easily countered; you can easily show him a map, or point him to Wikipedia, and say "No, the capital of New York State is Albany." And, if he's not mentally ill in some way, he'll probably say "Really? I didn't know that. Cool!"

But if someone asserts that Proctor & Gamble donates money to the Church of Satan, and you contradict him, he's likely to respond with anger--in a way that he won't if you tell him that Albany is New York's capital. That's because you're not contradicting his assertion of fact; you're telling him that his emotional identity is wrong.

And it's important to understand that even if a particular belief is wrong, the emotional landscape supporting that belief might not be. Proctor & Gamble does not donate money to the Church of Satan, but what is that belief an expression of? One likely possibility is that the emotional state beneath it looks something like "I do not trust large, faceless corporations to have my interests at heart, and I am afraid that a society dominated by large, faceless corporations may not be responsive to my needs and my values."

And you know what? That is a perfectly reasonable feeling to have. There very well indeed may be some truth to that idea, even though the specific beliefs that grow from this soil are twisted and misshapen.

Any attempt to debunk these ideas will never succeed if the debunking does not separate the idea from its emotional foundation. Furthermore, the fact that an idea grows from and is nourished by some kind of underlying emotional reality means that even the most otherwise skeptical, rational person can become attached to ideas that are patently false, and that person's own tools of rational skepticism may not be able to evaluate, or even see, those ideas.

The challenges this notion poses to skeptics and rationalists is worthy of a post in its own right, and will be the subject of Part 2 of this essay.


( 46 comments — Leave a comment )
Nov. 10th, 2008 05:52 pm (UTC)
Nice essay.

I'll offer this as supporting,corroborating study: http://www.physorg.com/news145461143.html
Nov. 10th, 2008 05:55 pm (UTC)
oh, well done.

imma gonna have to go away and think about this. i'll be baaack :)
Nov. 10th, 2008 06:16 pm (UTC)
Very well said.
Nov. 10th, 2008 06:38 pm (UTC)
Very interesting. Thank you for this.
Nov. 10th, 2008 06:43 pm (UTC)

Marvelous post, and one that I'd like some folks I know to read and think about. Linking it in my LJ so that they have a chance to see it.
Nov. 10th, 2008 06:51 pm (UTC)
Provocative. This is tasty food for thought, I admit that in my rush to be right I often (always?) skip over the emotional landscape, and am often not heard.
Nov. 10th, 2008 06:58 pm (UTC)
I have this problem too. I need to think on this some more.
Nov. 10th, 2008 09:55 pm (UTC)
Yes, me too!

Getting better about it, but I need lots of reminders like this one.
Nov. 10th, 2008 07:17 pm (UTC)
Extremely insightful -- thank you.

It's worth considering not only how much we fall into this trap ourselves, but how our failing to account for it in others leads to misunderstanding and ineffectiveness in trying to bring them around to our point of view.
Nov. 10th, 2008 07:17 pm (UTC)
She thinks he's a muslin terrorist?

I'm wondering if that's different from a silk terrorist, or a wool terrorist.

*is silly*
Nov. 10th, 2008 07:22 pm (UTC)
Gaah! Good catch. Fixed. :)
Nov. 10th, 2008 07:39 pm (UTC)
You're welcome.

Although now I want to run a game called "Muslin Terrorists."
Nov. 10th, 2008 09:32 pm (UTC)
no is true!!!!

I seed it on tha intarwebz!!!
Nov. 10th, 2008 08:36 pm (UTC)
"Muslin terrorist"
That'd be the silverfish I keep finding near my closet.
Wool terrorist
Moths, or rather their larvae.
Nov. 10th, 2008 07:21 pm (UTC)

Part of the problem with humans, too, is that they seek validation and look for wisdom in people that we consider leaders.

One of the reasons Obama has had such an incredible effect on people is that we look at him when he speaks and he comes across as wise, knowledgeable, literate and smart. The people that hold him in esteem become even more impressed with him as time goes one.

Many (not all, but many) republicans look to their leaders for the same thing. Consider, though, what those folks are saying. Limbaugh is a fear monger of the worst sort. Sarah Palin and her "he pals around with Terrorists" comments - and "he's a socialist" comments. All of those types of personal attacks target people who already have fears and expand them. Fear becomes the national drug of choice - and the past-time of people who LIKE our country in a war somewhere. So the republicans who are sensitive to emotional blackmail and pressure feel this constant weight of doom on their heads from their "wise leaders". And that's really sad. :(

Nov. 10th, 2008 07:51 pm (UTC)
Greta Christina just wrote a post about how the religious fallacy of Argument From Authority is one of the main reasons why Prop 8 and others passed, and I think that's related to your comment. She argues that there are those who blindly follow their leaders' words with no sense of reason or logic, and no interest in the evidence (which I'm sure we've all seen a lot of lately).

And those are the people who heard Palin's socialist terrorist comments and took them at face value, without even being interested in the evidence to the contrary.
Nov. 10th, 2008 07:31 pm (UTC)
"When an untrue story circulates, it's generally because it expresses some kind of social unease."

The examples that follow this statement are similar to the theories behind the origins of children's nursery rhymes, which if you've ever noticed are chock full of death, crime, pain and any number of very bad things that will potentially befall children if they are not moral and proper and minding of their parents. Social fears vs social mores expressed in an easy to remember parable or rhyme.
Nov. 10th, 2008 08:01 pm (UTC)
It's kind of amazing. For much of my existence, I have always been the "voice of reason", the logical, rational one who tries to explain concepts to people who insist on holding onto their emotional reactions and turn away from the evidence. Even when I was wrong, this was still the method of argument.

In the last handful of years, I have been intentionally surrounding myself with other rational, logical, analytical thinkers. Many of whom, like tacit, understand the importance of emotions in the decision-making process. But a couple of them are so rational and analytical, that 2 things happen. 1) They often don't even realize when they, themselves, make a decision based on an emotion and 2) I find myself in the position of trying to explain to them why they need to be more sensitive to their audience's feelings when trying to convince someone of something.

That is a very hard position for me to take because I don't always understand it myself. But it has forced me to recognize the importance of not ignoring the fact that people do make their decisions from an emotional base and to understand that if I want to be heard, simply yelling at them that I'm right while waving the facts in their face won't do it.

Now, sometimes I don't care - like my journal, for instance. That's a place for *me*, not for others, but others who want to hear me, can. But in conversation, in actual dialogs where I do want to change someone's mind, I have had to learn more about this emotional-base thinking process. And in watching others I know struggle to make people understand a concept that *I also* believe is an important one and that *I also* want others to hear, I've found myself in the unique position of "translator", trying to explain why one should change one's approach if one wants to be heard and understood, and that approach is to learn more about the emotional landscape from which the audience is listening.

It's an uncomfortable position to be in, but, I think, ultimately a valuable one.
Nov. 10th, 2008 11:21 pm (UTC)
Good observation. In a similar vein, I constantly have to remind myself of this quote:

"Never appeal to a man's better nature; he may not have one. Appealing to his self-interest gives you more leverage."

-- Robert Heinlein

This, of course, forces me to figure out what others consider to be in their self-interest.
Nov. 11th, 2008 04:40 am (UTC)
If you want to spend the effort. I just ask.

If they tell me, then they get an ally in trying to get it.

If they don't tell me or they lie, the deserve what they get.
Nov. 10th, 2008 08:42 pm (UTC)

Incidentally, this is the exactly the kind of insight that (*good*) therapy attempts to bring to the forefront of the consciousness of its clients. Some examples that I know of off the top of my head are the Imago relationship therapy developed by Harville Hendricks, who was in turn a client of John Pierakkos, who founded Core Energetics, which also worked to help clients discover their own "Lower Selves" in order to address the underlying fears that cause them to behave in these irrational ways. I've been out of the loop of conventional therapy for so long, that I'm not sure if it really digs deep enough to do the same. Admittedly, I have some disdain for conventional therapy now, because of this lack of depth, which it had when I was a participant/scholar of it almost 20 years ago, and it's my fondest hope that it's coming around.

Anyway, thanks for putting this out there.
Nov. 10th, 2008 09:36 pm (UTC)
Have you read the book "Religion Explained" by Pascal Boyer? I found it to be terribly compelling, and when you say "the brain is really not an organ of thought so much as an apparatus for forming beliefs" I think you would find a lot to agree with in it.

It starts by debunking every "standard" explanation of where religion comes from, and goes on to build in detail a fairly convincing case based on specific research in psychology, social psychology, and evolution. I learned more from it than any three other books I've read recently.
Nov. 10th, 2008 09:54 pm (UTC)
That book and the book 'god is not Great' by Christopher Hitchins were key to helping me understand religion (not having been raised in anything except some very loose Native Canadian mythology/spirituality)
Nov. 10th, 2008 09:47 pm (UTC)
Aye, lucid as always. Thanks for the insight - while I know it to be true, it's often hard to hold onto in a debate

'They are not assertions of fact in the way that many other statements are; they are assertions of emotional identity.'
Nov. 10th, 2008 09:52 pm (UTC)
Interestingly, this also applies to those who recently voted to defeat the California same sex marriage vote, eh? People are voting with their beliefs, not necessarily facts. The protestors are protesting with their beliefs, backed (sometimes) by facts.

Reflecting on all this.. well.. it's a rather sad thought.
(Deleted comment)
Nov. 10th, 2008 10:05 pm (UTC)
Even worse...why should it matter if he's a Muslim or not, anyway? Don't we supposedly have freedom of religion in this country? Why is there this underlying assumption that Muslim = not fit for presidency?
(Deleted comment)
Nov. 14th, 2008 02:04 am (UTC)
You don't. In many places, an atheist is banned from holding public office.

However, there is an oath that one can take without swearing on the Bible. I forget the details, but we do have in our justice system an alternate method.
Nov. 10th, 2008 11:20 pm (UTC)
I first heard the rumours about Proctor and Gamble back in the 1960's when peace, love and rock and roll were all the rage, only it was the "moonies" not the church of Satan. Maybe it's the same thing??
Nov. 10th, 2008 11:50 pm (UTC)
I first heard the Proctor and Gamble thing from Amway uplines in about 1986. Which was weird, because P&G don't exist as a brandname here in Australia.

We were told to comment on it and how they own lots of household product brands. [wink][wink][nudge][nudge]

Not long after that I gave up on Amway.
Nov. 10th, 2008 11:26 pm (UTC)
Great post.

And they can not be treated as assertions of fact, even though on their face that's what they look like.

This would be why, while my brother painstakingly debunks each email forwarded on by my lunatic family, I redirect them all to my spam folder. He's probably the better person in this, but there's only so much crazy I can take.
Nov. 10th, 2008 11:31 pm (UTC)
This is funny as I was discussing that legend just the other day. Though we were talking specifically about Jif peanut butter. I remembered first hearing this when I was in highschool so late 80s early 90s.
Nov. 11th, 2008 12:05 am (UTC)
Interesting post.

I haven't thought about it in decades, but I remember my friend Dave coming over all excited and rummaging through the medicine chest looking for P&G products after seeing one of those big Xian shows (PTL? 700 Club?) do a full hour on the P&G symbol. (With only one TV in a double-wide, what Dad watches everyone watches.) Some big haired evangelist associated the moon and stars with pagan worship, paying close attention to the unlucky number of stars, 13.

P&G had a PR guy answer the allegations a few days later, saying the moon was just a popular symbol when the company was founded and that the stars referred to the 13 original colonies -- but that the symbol meant nothing to them except as a tradition and that they would change it to quell fears.

I never heard (or don't remember) the bit about the CEO confessing the finances; I wonder if that rumor started later.
Nov. 11th, 2008 05:13 am (UTC)
You should have told her he is also going to usher in White slavery and watch her hit the roof. Sometimes it's just fun to fuck with people who are that out of it. I know it's wrong but wtf, I'm imperfect.
Nov. 11th, 2008 06:05 am (UTC)
From James Harvey Robinson's 'The Mind In The Making' (quoted from memory) "...most of our so-called reasoning consists of finding ways to go on believing what we have become accustomed to as being true" When I read that more than 25 years ago I had what you would describe as "something of an epiphany in my own understanding of the human condition" Now I understand WHY people are that way. P.S. Bravo! I remember a time when you had no understanding of the human condition. TMM
Nov. 11th, 2008 08:28 am (UTC)
Great post.
Nov. 11th, 2008 03:46 pm (UTC)
Excellent post, thanks!

A funny coincidence: only today my friend forwarded me a chain email that warned us about 7 women who died from smelling free perfume samples that they received by mail. The email also added that the government tries to suppress the matter, because they suspect it was a terrorist act, and don't want panic to ensue (!) The email was supposedly signed by Dr. * * (full name was given), the head of the pediatric department of a * hospital.

As you see, it's even more ridiculous than the P&G rumor; so it was very interesting, how could my friend, (who is a CS PhD student -- super-geek, very smart and intelligent) still fall for it!
And the short answer is that she wasn't using her brains. The long answer has to do with what you wrote here, but even more with what you wrote in the previous essay on the subject: the consequences of believing this particular rumor if it is false are much less severe than the consequences of not believing it, if it is true. And even more than that, she felt it was her social duty to pass the rumor on, even when she was uncertain on its veracity. It was a click-whirr! reaction on her part -- "I must warn my friends, or bad things might happen to them". And she only needed to press two buttons to do it.

The author of the Slacktivist essay seems to believe that it is immoral to forward any rumor, unless you are reasonably certain that it is true. But most people I know would wholeheartedly disagree with this. They believe that it is their moral duty to forward any warning, unless they are certain that it is false. After all, what if it's really true?! And you know what -- it is really difficult to explain to these people what is the harm in their actions.

In Russian, there even is an expression that is used to preface the retelling of a story without any fact checking: "За что купил, за то и продаю". It translates, literally, to: "I sell it for what I bought it". Meaning: I heard it, I have no idea if it's true, but I'm retelling it in the hope that someone, somewhere, sometime does the fact checking, informs us, and then we'll all know for sure. The all-powerful Someone Else will do the thinking -- I'm just a delivery boy. After all, one cannot check everything -- knowledge is a collective effort... The existence of this expression goes on to show just how widespread such behavior is.

- Ola
Nov. 14th, 2008 02:10 am (UTC)
Today, I got in my inbox a "warning" from "John Hopkins University" all the ways that microwaving various products cause cance. Both the claims it makes and the organization it purports to be from are false.

I used to have a pre-written response that I would respond with, with blanks for the specific Snopes page that referred to that specific hoax. In it, I detailed why forwarding hoaxes were bad, including pointing out that I now had everyone's private email address that was not deleted from the headers or not BCC'd to.

People would get really upset that I had their email, those I didn't know personally. They would angrily reply asking me to take them off my mailing list or demand to know how I got their email addy. So I explained again that this is the consequence for participating in email forwards and for not stopping their friends and family from participating.
Nov. 14th, 2008 04:31 pm (UTC)
Oh yes, I'm pretty sure that spammers send such letters into circulation, in the hope of getting them back with lots of email addresses attached.

What were the other reasons you cited in your email for why forwarding chain letters is bad?
Nov. 11th, 2008 10:29 pm (UTC)
"The idea that the government staged the attacks on the World Trade Center is a twisted-up, garbled expression of an emotional state: "I am afraid that my nation's government is corrupt and evil, and is willing to resort to any means, however extreme, to achieve its own ends."

This is why these beliefs are so vigorously resistant to debunking, even when the evidence against them is overwhelming. "

Sorry? Overwhelming? If you really think a bunch of box-cutter wielding thugs overpowered several planes and flew them into several different places, then I recommend you take a look at Loose Change.

And if you STILL think so, after watching it, then I have one question for you...what happened to two 6 ton turbofan engines which supposedly flew into the Pentagon?

Where are the two six ton engines? And don't try to tell me that the small turbo fan they found was all that was left, it wasn't even a piece of them...it didn't even go to that plane.

It may not have been the government who did it, but you can bet that they were in cahoots.

And finally, you're welcome. You can hold me up as an example of proof to all your other intellectual friends who "know" that it wasn't the government, despite the "overwhelming evidence". As for me, I'll continue to refer to them as "Vezzini"...those and the ones who willingly selected Obama.

Christ...a taxpayer voting for Obama is like a cow voting for Burger King.
Nov. 12th, 2008 03:19 pm (UTC)
The "mystery" of what happened to the turbofans isn't a mystery at all.

In fact, it's a great example of confirmation bias. You read about the the attacks and accept factual assertions made by a person who wasn't even there (in fact,t he first 9/11 conspiracy theories were written by a Frenchman who wasn't even on the same continent when it happened), and don't question those assertions. He says "there was no sign of the engines on the Pentagon lawn; what happened to the engines?" and his readers accept the idea that the engines were never found.

In fact, there's no mystery here at all; the engines were recovered. They tore through the inner walls of the Pentagon and punched clear through to the atrium inside, where they came to rest--leaving incredible damage in their wake. The hole punched through to the atrium by the engines acted as a vent for the ensuing fire, intensifying it tremendously:

On the inside wall of the second ring of the Pentagon, a nearly circular hole, about 12-feet wide, allows light to pour into the building from an internal service alley. An aircraft engine punched the hole out on its last flight after being broken loose from its moorings on the plane. The result became a huge vent for the subsequent explosion and fire. Signs of fire and black smoke now ring the outside of the jagged-edged hole.

The engines later had to be removed by crane.
Nov. 12th, 2008 08:46 pm (UTC)
One hole?

Two 6-ton masses, moving at over 400 mile an hour, only punch one hole?

How come two engines didn't punch two holes?
Nov. 13th, 2008 03:08 am (UTC)
Read the Snopes article about it.

Your comments don't give the impression that you're emotionally detached from your arguments; rather the opposite.
Nov. 13th, 2008 03:49 pm (UTC)
Your reply doesn't answer my questions.

And yes, I am emotionally attached.
Nov. 13th, 2008 03:56 pm (UTC)
The Snopes explanation explains nothing.

It certainly doesn't explain where the two six ton engines went.
Nov. 13th, 2008 04:12 pm (UTC)
The two six ton engines went to the NTSB. After they were pulled from the wreckage. There's no mystery here.

This argument kind of reminds me of the Tupac Shakur conspiracy theories. There's a whole elaborate conspiracy theory that asks "If he really was shot, why was there no autopsy and no coroner's report?" This in spite of the fact that there was an autopsy, and the coroner's report is a matter of public record.

"Why were there no engines?" Same deal.
Jan. 9th, 2015 03:50 am (UTC)
Wow, I don't know where to start, but I digress. I came to your site upon seeing the sexual experience map, or what ever the exact name you have for it. I like it and enjoyed reading your various other written work & photos.
( 46 comments — Leave a comment )