The Knowledge Illusion: Why We Never Think Alone

Steven Sloman and Philip Fernbach “The Knowledge Illusion: Why We Never Think Alone,” Riverhead Books, 2017, 265 pp.

As I have read in other books, this book explains the two minds we all have, these authors label them intuitive thinking and deliberative thinking.  Other authors have labeled them thinking slow and thinking fast, system 1 and system 2, etc.  Our gut reaction involves intuitive thinking, no deliberation is involved; deliberation takes conscious effort and is stridently avoided by many.  However, after discussing this two-mind subject, the book progresses onto a more novel supposition about how little any of us as individuals actually know whether we are thinking fast or more deliberatively. 

“You know more than I do, and you know next to nothing yourself.” That’s a finding of recent generations of neuroscientific researchers, who, as the authors note, is given to addressing a large question: “How is thinking possible?” One answer is that much of our thinking relies on the thinking of others—and, increasingly, on machine-others (robots, computers, etc.) that aid the process.

One thought experiment the authors propose is to produce from your mind everything you can say about how zippers work, a sobering exercise that quickly reveals the superficiality of much of what we carry inside our heads. We think we know, and then, on deliberative analysis, we discover we don’t.

The authors propose that what matters is what the individual mind contributes to the collectivity. All the same, the authors maintain, “intelligence is no longer a person’s ability to reason and solve problems; it’s how much the person contributes to a group’s reasoning and problem-solving process.”   

My Notes:

Pg. 4:  The field of cognitive science emerge in the 1950s in a noble effort to understand the working of the human mind. How is thinking possible? What goes on inside the head that allows sentient beings to do math, understand their mortality, act virtuously and (sometimes) selflessly, and even do simple things, like eat with a knife and fork”  No machine, and probably no other animal, is capable of these acts.  Yet, individual knowledge is remarkably shallow, only scratching the surface of the true complexity of the world.

Pg. 5:  The human mind is not like a desktop computer, designed to hold reams of information.  The mind is a flexible problem solver that evolved to extract only the most useful information to guide decisions in new situations.  Individuals store very little detailed information about the world in their heads.  In that sense, people are like bees and society a beehive: Our intelligence resides not in individual brains but in the collective mind.  Human thought is a product of a community, not of any individual alone.

Pg. 13:  We would not be such competent thinkers if we had to rely only on the limited knowledge stored in our heads and our facility for causal reasoning.  The secret to our success is that we live in a world in which knowledge is all around us.  It is in the things we make, in our bodies and workspaces, and in other people.  We live in a community of knowledge.  So the study of the mind cannot be reduced to the study of the brain.  Cognitive science is not the same as neuroscience.

Pg. 27:  Deliberation is only a tiny part of what goes on when we think.  MOs of cognition consists of intuitive thought that occurs below the surface of consciousness.  It involves processing huge quantities of information in parallel.  When people search for a word, for example, we don’t consider one word at a time sequentially.  We search our mental dictionary simultaneously, and the word we’re looking for usually rises to the top. 

Pg. 48:  If we had evolved in an environment that favored other kinds of capacities rather than the ability to choose effective actions, the human mind would probably follow a different kind of logic than it does.  If we evolved in a world that rewarded gambling on games of chance, we would probably be able to reason flawlessly about probability distributions and the laws of statistics.  Instead, we evolved in a world ruled by the logic of action, and that is why this kind of thought is so central to what makes us human and why we believed so long that the world is flat and the sun goes around the earth, it sure seems evident and logical.  The hypothesis that only humans are capable of true diagnostic reasoning—causal reasoning from effect to cause—is still defensible, even if the result is not always correct.

Pg. 67:  It is no coincidence that storytelling, the most natural mode of human discourse, depends on causal knowledge—a very central human trait.

Pg. 80:  People think they understand causal systems better than they in fact do.  The illusion is a product of the intuitive mind: we think about how things work automatically and effortlessly.  But when we deliberate about our knowledge, our illusion is often shattered.  This helps to explain why not everybody falls for the illusion as in the following.  Here’s a test: A bat and a ball cost $1.10.  The bat costs one dollar more than the ball.  How much does the ball cost?  Do you think the answer is $.10?

Pg. 103:  It is much easier to spell a word or do arithmetic if you write down what you are thinking.  In general, the fact that thought is more effective when it is done in conjunction with the physical world suggests that thought is not a disembodied process that takes place on a stage inside the head.  Mental activities do not simply occur in the brain.  Rather, the brain is only one part of a processing system that also includes the body and other aspects of the world.

Pg. 108:  People and bees have an important property in common: We both harness the power of multiple entities working together to generate massive intelligence.  Humans are the most complex and powerful species ever, not just because of what happens in individual brains, but because of how communities of brains work together.

Pg. 111:  The evolution of modern humans from other species of hominids—primate species related to people—was extremely rapid on an evolutionary time scale.  It began with the emergence of the genus Homo on the African Savannah 2 to 3 million years ago, with modern humans emerging about 200,000 years ago.  The great leap that humanity made during that period was cognitive.  The brain mass of modern humans is about three times that of our early hominid ancestors.  It did not evolve in the context of individuals sitting alone solving problems.  It evolved in the context of group collaboration, and our thinking evolved interdependently, to operate in conjunction with the thinking of others.  Much like a beehive, the group intelligence that emerges is more than the sum of its parts.

Pg. 128:  Because we live inside a hive mind, relying heavily on others and the environment to store our knowledge, most of what is in our heads is quite superficial.  We get by because a division of cognitive labor exists that divides responsibility for different aspects of knowledge across a community.  This ability to share knowledge across a community is what has allowed us to go to the moon, to build cars and freeways, etc…

Pg. 150:  The super-intelligence that is changing the world is in the community of knowledge.  The great advances in technology aren’t to be found in creating machines with superhuman horsepower; instead, they’ll come from helping information to flow smoothly through ever-bigger communities of knowledge and by making collaboration easier.  Intelligent technology is not replacing people so much as connecting them.  The web is demonstrating that real superintelligence resides in the community.

Pg. 157:  In 1979 the National Science Board began surveying Americans about scientific knowledge.  Here is a table of the results:

	Question	Percent Correct
1	True or false: The center of the Earth is very hot?	84
2	True or false: The continents have been moving their location for millions of years and will continue to move.	80
3	Does the Earth go around the Sun, or does the Sun go around the Earth?	73
4	True or false: All radioactivity is man-made.	67
5	True or false: Electrons are smaller than atoms.	51
6	True or false: Lasers work by focusing sound waves.	47
7	True or false: The universe began with a huge explosion.	38
8	True or false: The cloning of living things produces genetically identical copies.	80
9	True or false: It is the father’s gene that decides whether the baby is a boy or a girl.	61
10	True or false: Ordinary tomatoes do not contain genes, while genetically modified tomatoes do.	47
11	True or false: Antibiotics kill viruses as well as bacteria.	50
12	True or false: Human beings, as we know them today, developed from earlier species of animals.	47
	Answers: 1. T; 2 T; 3 Earth around Sun; 4. F; 5. T; 6. F; 7. T; 8. T; 9. T; 10. F; 11. F; 12. T.

If people were just guessing, they would get 50 percent correct.  Note that respondents from other countries including China, Russia, the EU, India, Japan, and South Korea do no better and most do a little worse. (Note: when the result above is less than 50%, religious belief is probably involved).

Pg. 162:  Our beliefs are not our own.  They are shared with our community.  And this makes them really hard to change.  We typically don’t know enough individually to form knowledgeable, nuanced views about new technologies and scientific developments.  We simply have no choice but to adopt the positions of those we trust. 

Pg. 165:  Genetically modified foods are highly controversial, yet according to the American Association for the Advancement of Science, the science is clear: ‘crop improvement by the modern molecular techniques of biotechnology is safe.’  Also, decades of testing foods exposed to high-energy radiation to kill pathogens have shown these foods to be safe and effective for decreasing food-borne illness and improving shelf life.  But public opposition has hardened due in part to confusion between radiation and radioactivity.

Pg. 181:  Careful analysis of the consequences of many political policies in shaping people’s attitudes may matter a lot less than you might think and that the community one is embedded in matters a lot more.  Also, there are certain values that we hold sacred, and no amount of discussion of consequences is likely to change them (think abortion, or assisted suicide: either side).  Causal explanation of an issue to moderate opinion applies only to certain issues, issues that elicit opinions based on outcomes as opposed to opinions based on values.  However, that leaves us with plenty of issues—most opinions are driven by a consideration of outcomes.  Opinions about everything from whether society should support nuclear power to education and health care are, for most people, a matter of how to achieve the best outcomes.  But if people’s positions are not consequentialist but based on sacred values, then shattering the illusion won’t matter.

Pg. 186:  Americans’ attitudes toward gay marriage have undergone a tectonic shift in recent years.  According to the Pew Research Center, 60 percent of Americans opposed gay marriage in 2004; in 2015 55 percent were in favor.  During this period, the debate shifted from being values-based (gay marriage is wrong) to a more consequence-based debate (everyone has the right to marry).  Whether an issue is framed in terms of consequences or sacred values influences the likelihood of achieving compromise in negotiation.  Sacred values perspectives are inviting because they simplify the field.  All that nasty detailed causal analysis can be avoided.  And sacred values can be correct.  For instance, who are we to dispute the Golden Rule?  Sacred values have their place, but their place should not be to prevent causal reasoning about the consequences of social policy.

Pg. 204:  All cognitive tests tend to be positively correlated, a fact that has been recognized since 1904 due to seminal work by Charles Spearman.  People who do well on one task are at least a little more likely than others to do well on another task.   Spearman called this commonality general intelligence.

Pg. 212:  Once you accept that we live in a community of knowledge, it becomes clear that most researchers have been looking in the wrong place for a definition of intelligence.  Intelligence is not a property of an individual; it’s a property of a team.  (Testing is much more accurate by observing an individual in a team environment.)

Pg. 248:  Behavioral science has found it is much more effective to structure the default choices for people that yield the most desirable results, called Nudging.  Participation in 401k plans should be the default, you have to opt out not in.  Likewise, organ donor checkoff on driver’s license should require you to opt out, not in.  The big lesson of the nudge approach is that it is easier and more effective to change the environment than it is to change the person.