February
27, 2017 "Information
Clearing House"
- "New
Yorker"
- In 1975, researchers at Stanford invited a
group of undergraduates to take part in a study
about suicide. They were presented with pairs of
suicide notes. In each pair, one note had been
composed by a random individual, the other by a
person who had subsequently taken his own life.
The students were then asked to distinguish
between the genuine notes and the fake ones.
Some
students discovered that they had a genius for
the task. Out of twenty-five pairs of notes,
they correctly identified the real one
twenty-four times. Others discovered that they
were hopeless. They identified the real note in
only ten instances.
As is
often the case with psychological studies, the
whole setup was a put-on. Though half the notes
were indeed genuine—they’d been obtained from
the Los Angeles County coroner’s office—the
scores were fictitious. The students who’d been
told they were almost always right were, on
average, no more discerning than those who had
been told they were mostly wrong.
In the
second phase of the study, the deception was
revealed. The students were told that the real
point of the experiment was to gauge their
responses to thinking they were right
or wrong. (This, it turned out, was also a
deception.) Finally, the students were asked to
estimate how many suicide notes they had
actually categorized correctly, and how many
they thought an average student would get right.
At this point, something curious happened. The
students in the high-score group said that they
thought they had, in fact, done quite
well—significantly better than the average
student—even though, as they’d just been told,
they had zero grounds for believing this.
Conversely, those who’d been assigned to the
low-score group said that they thought they had
done significantly worse than the average
student—a conclusion that was equally unfounded.
“Once
formed,” the researchers observed dryly,
“impressions are remarkably perseverant.”
A few
years later, a new set of Stanford students was
recruited for a related study. The students were
handed packets of information about a pair of
firefighters, Frank K. and George H. Frank’s bio
noted that, among other things, he had a baby
daughter and he liked to scuba dive. George had
a small son and played golf. The packets also
included the men’s responses on what the
researchers called the Risky-Conservative Choice
Test. According to one version of the packet,
Frank was a successful firefighter who, on the
test, almost always went with the safest option.
In the other version, Frank also chose the
safest option, but he was a lousy firefighter
who’d been put “on report” by his supervisors
several times. Once again, midway through the
study, the students were informed that they’d
been misled, and that the information they’d
received was entirely fictitious. The students
were then asked to describe their own beliefs.
What sort of attitude toward risk did they think
a successful firefighter would have? The
students who’d received the first packet thought
that he would avoid it. The students in the
second group thought he’d embrace it.
Even after
the evidence “for their beliefs has been totally
refuted, people fail to make appropriate
revisions in those beliefs,” the researchers
noted. In this case, the failure was
“particularly impressive,” since two data points
would never have been enough information to
generalize from.
The
Stanford studies became famous. Coming from a
group of academics in the nineteen-seventies,
the contention that people can’t think straight
was shocking. It isn’t any longer. Thousands of
subsequent experiments have confirmed (and
elaborated on) this finding. As everyone who’s
followed the research—or even occasionally
picked up a copy of Psychology Today—knows,
any graduate student with a clipboard can
demonstrate that reasonable-seeming people are
often totally irrational. Rarely has this
insight seemed more relevant than it does right
now. Still, an essential puzzle remains: How did
we come to be this way?
In a new
book, “The Enigma of Reason” (Harvard), the
cognitive scientists Hugo Mercier and Dan
Sperber take a stab at answering this question.
Mercier, who works at a French research
institute in Lyon, and Sperber, now based at the
Central European University, in Budapest, point
out that reason is an evolved trait, like
bipedalism or three-color vision. It emerged on
the savannas of Africa, and has to be understood
in that context.
Stripped
of a lot of what might be called
cognitive-science-ese, Mercier and Sperber’s
argument runs, more or less, as follows: Humans’
biggest advantage over other species is our
ability to coöperate. Coöperation is difficult
to establish and almost as difficult to sustain.
For any individual, freeloading is always the
best course of action. Reason developed not to
enable us to solve abstract, logical problems or
even to help us draw conclusions from unfamiliar
data; rather, it developed to resolve the
problems posed by living in collaborative
groups.
“Reason is
an adaptation to the hypersocial niche humans
have evolved for themselves,” Mercier and
Sperber write. Habits of mind that seem weird or
goofy or just plain dumb from an
“intellectualist” point of view prove shrewd
when seen from a social “interactionist”
perspective.
Consider
what’s become known as “confirmation bias,” the
tendency people have to embrace information that
supports their beliefs and reject information
that contradicts them. Of the many forms of
faulty thinking that have been identified,
confirmation bias is among the best catalogued;
it’s the subject of entire textbooks’ worth of
experiments. One of the most famous of these was
conducted, again, at Stanford. For this
experiment, researchers rounded up a group of
students who had opposing opinions about capital
punishment. Half the students were in favor of
it and thought that it deterred crime; the other
half were against it and thought that it had no
effect on crime.
The
students were asked to respond to two studies.
One provided data in support of the deterrence
argument, and the other provided data that
called it into question. Both studies—you
guessed it—were made up, and had been designed
to present what were, objectively speaking,
equally compelling statistics. The students who
had originally supported capital punishment
rated the pro-deterrence data highly credible
and the anti-deterrence data unconvincing; the
students who’d originally opposed capital
punishment did the reverse. At the end of the
experiment, the students were asked once again
about their views. Those who’d started out
pro-capital punishment were now even more in
favor of it; those who’d opposed it were even
more hostile.
If reason
is designed to generate sound judgments, then
it’s hard to conceive of a more serious design
flaw than confirmation bias. Imagine, Mercier
and Sperber suggest, a mouse that thinks the way
we do. Such a mouse, “bent on confirming its
belief that there are no cats around,” would
soon be dinner. To the extent that confirmation
bias leads people to dismiss evidence of new or
underappreciated threats—the human equivalent of
the cat around the corner—it’s a trait that
should have been selected against. The fact that
both we and it survive, Mercier and Sperber
argue, proves that it must have some adaptive
function, and that function, they maintain, is
related to our “hypersociability.”
Mercier
and Sperber prefer the term “myside bias.”
Humans, they point out, aren’t randomly
credulous. Presented with someone else’s
argument, we’re quite adept at spotting the
weaknesses. Almost invariably, the positions
we’re blind about are our own.
A recent
experiment performed by Mercier and some
European colleagues neatly demonstrates this
asymmetry. Participants were asked to answer a
series of simple reasoning problems. They were
then asked to explain their responses, and were
given a chance to modify them if they identified
mistakes. The majority were satisfied with their
original choices; fewer than fifteen per cent
changed their minds in step two.
In step
three, participants were shown one of the same
problems, along with their answer and the answer
of another participant, who’d come to a
different conclusion. Once again, they were
given the chance to change their responses. But
a trick had been played: the answers presented
to them as someone else’s were actually their
own, and vice versa. About half the participants
realized what was going on. Among the other
half, suddenly people became a lot more
critical. Nearly sixty per cent now rejected the
responses that they’d earlier been satisfied
with.
This
lopsidedness, according to Mercier and Sperber,
reflects the task that reason evolved to
perform, which is to prevent us from getting
screwed by the other members of our group.
Living in small bands of hunter-gatherers, our
ancestors were primarily concerned with their
social standing, and with making sure that they
weren’t the ones risking their lives on the hunt
while others loafed around in the cave. There
was little advantage in reasoning clearly, while
much was to be gained from winning arguments.
Among the
many, many issues our forebears didn’t worry
about were the deterrent effects of capital
punishment and the ideal attributes of a
firefighter. Nor did they have to contend with
fabricated studies, or fake news, or Twitter.
It’s no wonder, then, that today reason often
seems to fail us. As Mercier and Sperber write,
“This is one of many cases in which the
environment changed too quickly for natural
selection to catch up.”
Steven
Sloman, a professor at Brown, and Philip
Fernbach, a professor at the University of
Colorado, are also cognitive scientists. They,
too, believe sociability is the key to how the
human mind functions or, perhaps more
pertinently, malfunctions. They begin their
book, “The Knowledge Illusion: Why We Never
Think Alone” (Riverhead), with a look at
toilets.
Virtually
everyone in the United States, and indeed
throughout the developed world, is familiar with
toilets. A typical flush toilet has a ceramic
bowl filled with water. When the handle is
depressed, or the button pushed, the water—and
everything that’s been deposited in it—gets
sucked into a pipe and from there into the
sewage system. But how does this actually
happen?
In a study
conducted at Yale, graduate students were asked
to rate their understanding of everyday devices,
including toilets, zippers, and cylinder locks.
They were then asked to write detailed,
step-by-step explanations of how the devices
work, and to rate their understanding again.
Apparently, the effort revealed to the students
their own ignorance, because their
self-assessments dropped. (Toilets, it turns
out, are more complicated than they appear.)
Sloman and
Fernbach see this effect, which they call the
“illusion of explanatory depth,” just about
everywhere. People believe that they know way
more than they actually do. What allows us to
persist in this belief is other people. In the
case of my toilet, someone else designed it so
that I can operate it easily. This is something
humans are very good at. We’ve been relying on
one another’s expertise ever since we figured
out how to hunt together, which was probably a
key development in our evolutionary history. So
well do we collaborate, Sloman and Fernbach
argue, that we can hardly tell where our own
understanding ends and others’ begins.
“One
implication of the naturalness with which we
divide cognitive labor,” they write, is that
there’s “no sharp boundary between one person’s
ideas and knowledge” and “those of other
members” of the group.
This
borderlessness, or, if you prefer, confusion, is
also crucial to what we consider progress. As
people invented new tools for new ways of
living, they simultaneously created new realms
of ignorance; if everyone had insisted on, say,
mastering the principles of metalworking before
picking up a knife, the Bronze Age wouldn’t have
amounted to much. When it comes to new
technologies, incomplete understanding is
empowering.
Where it
gets us into trouble, according to Sloman and
Fernbach, is in the political domain. It’s one
thing for me to flush a toilet without knowing
how it operates, and another for me to favor (or
oppose) an immigration ban without knowing what
I’m talking about. Sloman and Fernbach cite a
survey conducted in 2014, not long after Russia
"annexed" the Ukrainian territory of Crimea.
Respondents were asked how they thought the U.S.
should react, and also whether they could
identify Ukraine on a map. The farther off base
they were about the geography, the more likely
they were to favor military intervention.
(Respondents were so unsure of Ukraine’s
location that the median guess was wrong by
eighteen hundred miles, roughly the distance
from Kiev to Madrid.)
Surveys on
many other issues have yielded similarly
dismaying results. “As a rule, strong feelings
about issues do not emerge from deep
understanding,” Sloman and Fernbach write. And
here our dependence on other minds reinforces
the problem. If your position on, say, the
Affordable Care Act is baseless and I rely on
it, then my opinion is also baseless. When I
talk to Tom and he decides he agrees with me,
his opinion is also baseless, but now that the
three of us concur we feel that much more smug
about our views. If we all now dismiss as
unconvincing any information that contradicts
our opinion, you get, well, the Trump
Administration.
“This is
how a community of knowledge can become
dangerous,” Sloman and Fernbach observe. The two
have performed their own version of the toilet
experiment, substituting public policy for
household gadgets. In a study conducted in 2012,
they asked people for their stance on questions
like: Should there be a single-payer health-care
system? Or merit-based pay for teachers?
Participants were asked to rate their positions
depending on how strongly they agreed or
disagreed with the proposals. Next, they were
instructed to explain, in as much detail as they
could, the impacts of implementing each one.
Most people at this point ran into trouble.
Asked once again to rate their views, they
ratcheted down the intensity, so that they
either agreed or disagreed less vehemently.
Sloman and
Fernbach see in this result a little candle for
a dark world. If we—or our friends or the
pundits on CNN—spent less time pontificating and
more trying to work through the implications of
policy proposals, we’d realize how clueless we
are and moderate our views. This, they write,
“may be the only form of thinking that will
shatter the illusion of explanatory depth and
change people’s attitudes.”
One way to
look at science is as a system that corrects for
people’s natural inclinations. In a well-run
laboratory, there’s no room for myside bias; the
results have to be reproducible in other
laboratories, by researchers who have no motive
to confirm them. And this, it could be argued,
is why the system has proved so successful. At
any given moment, a field may be dominated by
squabbles, but, in the end, the methodology
prevails. Science moves forward, even as we
remain stuck in place.
In
“Denying to the Grave: Why We Ignore the Facts
That Will Save Us” (Oxford), Jack Gorman, a
psychiatrist, and his daughter, Sara Gorman, a
public-health specialist, probe the gap between
what science tells us and what we tell
ourselves. Their concern is with those
persistent beliefs which are not just
demonstrably false but also potentially deadly,
like the conviction that vaccines are hazardous.
Of course, what’s hazardous is not
being vaccinated; that’s why vaccines were
created in the first place. “Immunization is one
of the triumphs of modern medicine,” the Gormans
note. But no matter how many scientific studies
conclude that vaccines are safe, and that
there’s no link between immunizations and
autism, anti-vaxxers remain unmoved. (They can
now count on their side—sort of—Donald Trump,
who has said that, although he and his wife had
their son, Barron, vaccinated, they refused to
do so on the timetable recommended by
pediatricians.)
The
Gormans, too, argue that ways of thinking that
now seem self-destructive must at some point
have been adaptive. And they, too, dedicate many
pages to confirmation bias, which, they claim,
has a physiological component. They cite
research suggesting that people experience
genuine pleasure—a rush of dopamine—when
processing information that supports their
beliefs. “It feels good to ‘stick to our guns’
even if we are wrong,” they observe.
The
Gormans don’t just want to catalogue the ways we
go wrong; they want to correct for them. There
must be some way, they maintain, to convince
people that vaccines are good for kids, and
handguns are dangerous. (Another widespread but
statistically insupportable belief they’d like
to discredit is that owning a gun makes you
safer.) But here they encounter the very
problems they have enumerated. Providing people
with accurate information doesn’t seem to help;
they simply discount it. Appealing to their
emotions may work better, but doing so is
obviously antithetical to the goal of promoting
sound science. “The challenge that remains,”
they write toward the end of their book, “is to
figure out how to address the tendencies that
lead to false scientific belief.”
“The
Enigma of Reason,” “The Knowledge Illusion,” and
“Denying to the Grave” were all written before
the November election. And yet they anticipate
Kellyanne Conway and the rise of “alternative
facts.” These days, it can feel as if the entire
country has been given over to a vast
psychological experiment being run either by no
one or by Steve Bannon. Rational agents would be
able to think their way to a solution. But, on
this matter, the literature is not reassuring.
Elizabeth Kolbert has been a
staff writer at
The New Yorker
since 1999. She won the 2015 Pulitzer Prize for
general nonfiction for “The Sixth Extinction: An
Unnatural History.”
This
article appears in other versions of the
February 27, 2017, issue, with the headline
“That’s What You Think.”
The
views expressed in this article are solely those of the author and do not necessarily
reflect the opinions of Information Clearing
House.
It is unacceptable to slander, smear or engage in personal attacks on authors of articles posted on ICH.
Those engaging in that behavior will be banned from the comment section.
In accordance
with Title 17 U.S.C. Section 107, this material
is distributed without profit to those who have
expressed a prior interest in receiving the
included information for research and educational
purposes. Information Clearing House has no
affiliation whatsoever with the originator of
this article nor is Information ClearingHouse
endorsed or sponsored by the originator.)