Wisdom of the crowds; How much does that cow really weigh?

"Anyone who regularly uses Social Media already knows: the so-called wisdom of the crowd is often disappointing. Yet the idea that we are smarter collectively than individually often arises."

The classic “Wisdom of the Crowds” experiment was conducted in 1906 by Francis Galton, a cousin of Charles Darwin. He went to a village fair and presented a prize for those who could correctly guess the weight of an ox. Nobody was able to, not even the farmers,who you’d expect to know. The mean of all estimates, however, turned out to be the exact weight of the animal.

Whether this experiment relied more on luck than wisdom is unclear. This did not stop James Surowiecki from prominently stating the example in his book “Wisdom of the Crowds” in 2005. Surowiecki argues that a diverse collection of independently deciding people is likely to make certain types of decisions and predictions better than individuals or even experts. Proof provided: the masses are wise.

Surowiecki states that four conditions must be met in order for collective wisdom to occur. A diversity of opinions, that must be based on some prior knowledge, is necessary . In addition, the members of the masses are expected to think independently and not to influence each other. A mechanism to convert these individual statements into collective decisions must be in place. Finally, there needs to be sufficient trust of everyone’s good intentions within the group. Without all these preconditions, hysterical groupthink is lurking. Perhaps this explains the current climate on the Internet.

Since the publication of “Wisdom of the Crowds”, there have likely been as many studies that endorse Surowiecki’s thesis as those that undermine it. The experiment with the ox has also been repeated a number of times in this century. It turned out that farmers and butchers are much better at guessing the weight of the cow than city dwellers. So more seems to be needed than a group of people with an opinion. Some expertise cannot hurt. Disagreement and discussion can lead to better decision making, but too much communication makes the group as a whole less intelligent.

"Surowiecki argues that a diverse collection of independently deciding people is likely to make certain types of decisions and predictions better than individuals or even experts. Proof provided: the masses are wise."

Tweet

What can we do with this knowledge? What if the challenge is more complex than guessing a mammal’s weight? This can be a commercial decision that is important for the future of the company. But government and other not-for-profit organizations also face challenges that are not easy to solve. Especially in a professional environment it turns out to be smart to use the collective brainpower of the group. After all, the available knowledge forms a good basis for a better decision. An independent, diverse team increases the chance of a good decision. Pitfalls that must be avoided are groupthink and the hijacking of the decision by the too strong influence of an (in)formal leader.

Fortunately, nowadays there are technical solutions that can avoid these pitfalls, allow everyone to have a say and also formulate the collective decision. A combination of the brainpower of the masses and smart use of support software then leads to better decisions and better business results.