THE WISDOM OF CROWDS:

Why the Many Are Smarter than the Few and How Collective

Wisdom Shapes Business, Economies, Societies and Nations

By James Surowiecki

 

The Wisdom of Crowds is the latest in a recent spate of counter-intuitive books, like Blink and The Tipping Point, that fly in the face of conventional wisdom.  The basic thesis of this book is that all of us are smarter than any of us and a group of ordinary people who pool their knowledge effectively can often obtain better results than an expert or even a group of experts.  In other words, large groups of people are smarter than an elite few, no matter how brilliant.  They are better at solving problems, fostering innovation, coming to wise decisions and predicting the future.  Surowiecki writes:

 

“The idea of the wisdom of crowds is not that a group will always give you the right answer, but that it will consistently come up with a better answer than any individual can provide.”

 

He continues:

 

“The fact that cognitive diversity matters does not mean that if you assemble a group of diverse but thoroughly uninformed people, their collective wisdom will be smarter than an expert’s.  But if you can assemble a diverse group of people who possess varying degrees of knowledge and insight, you’re better off entrusting it with major decisions rather than leaving them in the hands of one or two people, no matter how smart those people are.”

 

The author examines three kinds of problems: cognition problems, coordination problems and cooperation problems.  Two simple examples of cognition problems are guessing how many jelly beans there are in a jar or the weight of an ox.  He cites cases of these two examples where the average estimates of a large crowd were better than any individual estimate.

 

One example of a cognition problem where the wisdom of the crowd is very difficult to surpass is the point spread in sports contests, such as NFL games.  Bookmakers establish the initial point spreads for the 14 NFL games that are played each weekend, but then the spreads fluctuate according to how people bet.  For example let’s assume that Philadelphia is initially favored to beat Washington by 7 points, but then a large number of bets come in on Washington.  This indicates that the crowd is saying that the point spread is too high, so it is lowered until roughly the same amount of money is being bet on  Philadelphia as on Washington. 

 

In a recent movie (Two for the Money), a former college football star whose pro career is shattered by injury seeks to make a living by advising people how to beat the spread.  In the beginning, he beats the spread consistently, but the law of averages soon catches up with him and he ends up causing his clients to lose big-time.  Surowiecki would say that the movie is realistic, because it is virtually impossible to beat the wisdom of the crowd if the crowd is sufficiently diverse and there is a free flow of information throughout the crowd.  In sports betting, this is definitely the case, because the spread is continually adjusted to reflect the betting.

 

A simple example of a coordination problem is a crowded sidewalk in Manhattan.  Despite the fact that Manhattan sidewalks are very often seriously overcrowded, most people manage to navigate through the crowd with a minimum of collisions.  The crowd manages to coordinate the movements of individual pedestrians in an amazing way.  People somehow manage to discern one another’s intentions and adjust their movements accordingly.  Ironically, automobile traffic in Manhattan or any other large city is a very different story.

 

Two examples of extraordinary coordination in nature are a flock of starlings and a school of fish.  On the other hand, an example of poor coordination is the U.S. intelligence community.  Surowiecki devotes almost an entire chapter to this, because it illustrates that the wisdom of a crowd only works when there is a free flow of information throughout the crowd.  Because individual intelligence agencies like the CIA, the FBI, the NSA the DIA, etc. normally do not share information with one another, the intelligence community “crowd” often performs poorly.

 

The wisdom of crowds suggests that decentralization should make things run more smoothly.  So why didn’t decentralization work in the U.S. intelligence community?  The author writes:

 

“Decentralization’s great strength is that it encourages independence and specialization on the one hand while still allowing people to coordinate their activities and solve difficult problems on the other.  Decentralization’s great weakness is that there’s no guarantee that valuable information which is uncovered in one part of the system will find its way through the rest of the system.  Sometimes valuable information never gets disseminated, making it less useful than it otherwise would be….A decentralized system can only produce genuinely intelligent results if there is a means of aggregating the information in the system.”

 

Coordination problems can be solved even if each individual is single-mindedly pursuing his self-interest – in fact in the case of prices, that’s what coordination seems to require.  To solve cooperation problems - which include things like keeping the sidewalks free of snow, paying taxes and curbing pollution – the members of a group or society need to do more.  They need to adopt a broader definition of self-interest than the myopic one that maximizing profits in the short term demands.  And they need to be able to trust those around them because in the absence of trust, myopic self-interest is the only strategy that makes sense.

 

In other words, cooperation requires trust and confidence that others are cooperating too.  People will not cooperate if they feel that the system is not fair.  Surowiecki cites the case of the “ultimatum game”, where two people are given $10 to divide between them.  One player (the proposer) makes a take-it-or-leave-it offer to the other person.  If the responder refuses, neither of them gets anything.  If the responder accepts, he gets what is offered and the proposer keeps the rest.  Therefore, no matter what the proposer offers, the rational solution is to accept it, because otherwise the responder gets nothing.  Surowiecki concludes:

 

“In practice, though, this rarely happens.  Instead, lowball offers – anything below $2 – are routinely rejected….People would rather have nothing than let their ’partner’ walk away with too much of the loot  They will give up free money to punish what they perceive as greedy or selfish behavior.  And the interesting thing is that the proposers anticipate this – presumably because they know they would act the same way if they were in the responder’s shoes.  As a result, the proposers don‘t make low offers in the first place.  The most common offer in the ultimatum game, in fact, is $5.

 

This impulse toward fairness leads to pro-social behavior.  Societies and organizations only work if people cooperate.  It seems reasonable that people would cooperate with people with whom they have to deal on an on-going basis, but people also cooperate with strangers by tipping in restaurants far from home or giving to charity. 

 

Tax-paying is a classic example of a cooperation problem.  Everyone reaps benefits from the services that taxes fund, such as schools, police, the military, etc., whether they pay taxes or not.  Most people will dutifully pay their taxes as long as they perceive that the system is basically fair.  If they feel that others are taking advantage of their cooperation, however, the system will break down.  No one wants to be a chump.

 

Capitalism is another example of a cooperation problem.  Capitalism only works if people cooperate, which explains why tightly-knit groups of people who tend to cooperate very well with other members of the same group tend to prosper in a capitalist society.  The Quakers, for example, ran a sizable chunk of Britain’s economy in the 18th and early 19th century.  They were prosperous because they cooperated.  The Amish are another example that comes to mind.

 

The fact that “Individual irrationality can add up to collective rationality” does not mean that the crowd always makes good decisions.  The author examines several cases where the crowd, in fact, makes bad decisions.  One example would be “bubbles” in the marketplace, such as the dotcom bubble in the 1990’s or the real estate bubble in Tokyo in the 1980’s.  In these cases, people kept bidding up the price because everyone else seemed to be doing the same thing, and it was assumed that it could continue to go up indefinitely.  The stock market is usually remarkably efficient at fixing price at the appropriate level (which he estimates should be roughly equal to the earnings of the company for the next twenty years) because of the diversity of the large number of potential buyers and sellers.  This is similar to gamblers getting the “right” point spread.  In a bubble, however, that goes out the window, because people are making dependent decisions (buying because other people are buying), thus bidding up the price well beyond what is reasonable.

 

In fact, the wisdom of the crowd depends on the crowd being a diverse group of people making independent decisions.  If either of these conditions is not met, the crowd “loses its head” so to speak, as in the case of a mob.  Mob psychology is where people do things they would never do if they were alone or in a small group because they are making dependent decisions: their behavior depends on the behavior of the other members of the crowd/mob.

 

Groups that are either homogenous or dominated by one or a small number of people can also make people dumber rather than wiser.  Small groups can make very bad decisions because influence is more direct and immediate and small-group judgments tend to be more volatile and extreme.  The author cites the example of the Columbia disaster, where the team leader did not encourage divergent views about the potential damage caused by chunks of insulation coming off during launch.  In fact, she squelched them, because she had decided ahead of time that the foam strike was inconsequential.

 

The last chapter of the book is devoted to an examination of democracy.  The basic question is how we can entrust decisions to a largely uninformed, ignorant and apathetic electorate.  He concludes that for all its inefficiency, democracy is still the best form of government.  He writes;

 

“Choosing candidates and making policy in a democracy are not, in that sense, cognitive problems and so we should not expect them to yield themselves to the wisdom of the crowd.  On the other hand, there’s no reason to think that any other political system (dictatorships, aristocracy rule by elites) will be any better at making policy and the risks built into those systems – most notably the risk of the exercise of unchecked and unreasonable power – are much greater than those in a democracy….The decisions that democracies make may not demonstrate the wisdom of the crowd.  The decision to make them democratically does.”

 

This is a very thought-provoking and challenging book.  It would be easy to jump to simplistic conclusions, but the author does an excellent job of tempering his enthusiasm for the wisdom of the crowd with the fact that the crowd is wiser only if:

 

1.      It is diverse.

2.      Its members are thinking independently.

3.      It can find an effective way of pooling its knowledge.

 

If any of these conditions is not met, the crowd or group can make dumber, not wiser decisions.  Remembering this can help us make groups and teams work more efficiently and take advantage of the wisdom of crowds.

 

John Ed Robertson

October 25, 2005

 

Surowiecki, James; The Wisdom of Crowds; Doubleday; 2004; ISBN 0-385-50386