Humans are inherently bad at predicting the future. It’s a defect all too apparent in the corporate world, and in the business of managing complex geopolitics.
But some people have better track records than others, and the ways in which they think about questions and arrive at their projections offer clues as to how the rest of us might become more successful forecasters.
A group of researchers isolated these traits in a study tied to a geopolitical forecasting tournament arranged by an R&D group run by the US director of national intelligence. Five university-backed teams competed; they were asked to predict 199 world events, such as whether Syrian president Bashar al-Assad would remain in power, and whether North Korea would conduct another successful nuclear weapons test.
The University of Pennsylvania psychologists behind the winning squad have published a fascinating paper (pdf) breaking down their approach to building a better forecasting team.
The kinds of forecasts that the US intelligence community relies on are usually made by highly trained people with access to confidential information. But the study found ways to take lay people with no special clearances and help them become impressively accurate forecasters.
The best forecasters were the brightest, both in terms of cognitive ability and political knowledge. But many other traits and behaviors mattered as well. Thinking style is important; people who are actively open-minded performed significantly better. They’re much more willing to consider unorthodox ideas or results, and to stray from the theories and beliefs they’re comfortable with.
Not all of the skills of successful forecasters are innate; many can be learned. The researchers found that being instructed to recognize and avoid bias and to use outside views had a huge impact. So did training in probabilistic approaches, like using forecasting models to average the likelihood of all possible outcomes for a given question. People who simply spent more time pondering each question also did better, as did those who habitually updated their forecasts when new information came in.
Feedback and attitude matter, too. Over the course of the tournament, team members got frequent updates on their Brier scores (a measurement of the accuracy of probabilistic predictions) and how they compared to those of other participants in the exercise. Those who saw forecasting as a skill to be developed and responded to the feedback were more accurate.
The tournament lent itself to an experiment where people could be divided into a range of different work environments. Some of the University of Pennsylvania’s 743 team members—computer scientists, mathematicians, and financial investors among them—worked on their forecasts independently. Others worked in groups of up to 15 where they could freely debate and share predictions. The groups were trained extensively in how to work well together to help their teammates produce better forecasts.
The people working in groups performed significantly better than those working alone, with forecasts that were about 10% more accurate. Working in a team boosted the effect of other positive attributes, like intelligence and open mindedness.
There are negative aspects to working on a team, like the potential to mistakenly follow a crowd, or the tendency to end up in factions. But the positive aspects—such as the opportunity for dissent to arise, the diversity of knowledge to draw on—outweighed them.