The great Victorian polymath, Sir Francis Galton was at a country fair in 1906, so the story goes, and came across a competition where you had to guess the weight of an ox. Once the competition was over Galton, an explorer, meteorologist, scientist and statistician, took the 787 guesses and calculated the average, which came to 1,197 pounds. The actual weight of the ox was 1,198 pounds. In effect, the crowd had provided a near perfect answer. Galton would later publish this insight in Nature.
This phenomenon, where collective wisdom is better than most, if not all of the individuals in the crowd has become known as the Wisdom of Crowds. The authoritative take on it came from James Surowiecki. A more up-to-date example is the “Ask the Audience” part of Who Wants to be a Millionaire, where the studio audience are polled and the most popular answer is the correct answer 91 percent of the time.
Even if there is a better individual guess, you face the problem of deciding which individual’s guess to select. If you choose the crowd’s guess, the decision is made for you and there is every opportunity that you will get a good answer, certainly better than choosing randomly from the other guesses. The technique has practical uses beyond the quiz show.
Understanding the Challenger Disaster
On Jan. 28, 1986 the space shuttle Challenger broke up 73 seconds after launch, killing all seven astronauts on board. The disaster has been well reported in the intervening 30 years, but one intriguing aspect of it may have passed you by.
Almost immediately after the explosion, investors started selling stocks of the four main contractors involved in the Challenger launch—Lockheed, Rockwell International, Martin Marietta and Morton Thiokol. Of the four companies Morton Thiokol fell the most, almost 12 percent by the end of trading on that day, compared to about 3 percent for the other three companies.
This was a sign that the stock market felt that Morton Thiokol was to blame for the disaster but without having any firm evidence to hand.
In any case, six months later, the market was proven to be right. The O-ring seals on the booster rockets made by Thiokol were the cause of the problem. Richard Feynman, the renowned physicist, famously presented his findings to the Rogers Commission showing how the seals had failed.
It is still not quite clear how the wisdom of crowds managed to identify the company that was to blame for the disaster within minutes of it happening. Markets always weigh up a variety of factors and it’s hard to unpick the rationales at play. It’s just about possible that a few investors caught wind of whispers from before the launch about engineers’ concerns.
Finding the Scorpion Submarine
On May 22 1968 the US navy lost one of its submarines and wanted to find the wreckage, but the intelligence it had was not able to provide an area that was small enough to effectively search. John Craven a naval officer, decided to harness the wisdom of crowds.
He asked a wide group of individuals, drawn from diverse backgrounds ranging from mathematicians to salvage experts to guess the submarine’s location. The group’s average guess was just 220 yards from the location where the Scorpion was eventually found.
What Makes It Work?
The wisdom of crowds might seem like an easy way to get answers. Simply ask a lot of people want they think, and aggregate the answers. If the method could find the Scorpion submarine, then a missing plane should be just as easy? Well, no.
As yet, nobody has been able to find the Malaysia Airlines plane MH370 that went missing in March 2014. Almost two years on and the crash site—assuming it crashed—has not been found. That’s despite a massive crowdsourcing effort to identify the location of the aircraft, which was detailed in an article on The Conversation. But this was a case of searching for pieces of debris, not making educated guesses about location. And it leads us in to the key rules to follow if you want to use the wisdom of crowds to your advantage.
Four criteria are important in making this an effective tool.
Independence: The various guesses have to be independent of one another. That is, each person must guess without the knowledge of what other people have guessed.
Diversity: It is important to have a diverse set of guesses. In the guess the weight of the ox example, the people making the guesses ranged from farmers, butchers, livestock experts, housewives etc. That is, some people would be considered experts, while others would be considered as people with just a passing interest.
Decentralisation: The people making the guesses should be able to draw on their private, local knowledge.
Aggregation: There must be some way of aggregating the guesses into a single collective guess. In the guess the weight of the ox example, this was done by taking the average guess. This is a common method, but others may also be used.
Philip Ball, in this BBC article, highlighted flaws in the theory when studies ignore the rules. Remove independence and people start to gravitate towards a consensus which veers away from the accurate answer. Reduce diversity and respondents rely on shared biases, like a room full of football fans predicting results while burdened with the knowledge of which teams are the favorites. In other words, it helps to deploy a bit of wisdom when choosing your crowd.
Graham Kendall is a professor of operations research and vice-provost at the University of Nottingham in the U.K. This article was originally published on The Conversation.
