Why good forecasters become better people

Tim Harford in The FT

So, what’s going to happen next, eh? Hard to say: the future has a lotta ins, a lotta outs, a lotta what-have-yous. 

Perhaps I should be more willing to make bold forecasts. I see my peers forecasting all kinds of things with a confidence that only seems to add to their credibility. Bad forecasts are usually forgotten and you can milk a spectacular success for years. 

Yet forecasts are the junk food of political and economic analysis: tasty to consume but neither satisfying nor healthy in the long run. So why should they be any more wholesome to produce? The answer, it seems, is that those who habitually make forecasts may turn into better people. That is the conclusion suggested by a research paper from three psychologists, Barbara Mellers, Philip Tetlock and Hal Arkes. 

Prof Tetlock won attention for his 2005 book Expert Political Judgment, which used the simple method of asking a few hundred experts to make specific, time-limited forecasts such as “Will Italy’s government debt/GDP ratio be between 70 and 90 per cent in December 1998?” or “Will Saddam Hussein be the president of Iraq on Dec 31 2002?” 

It is only a modest oversimplification to summarise Prof Tetlock’s results using the late William Goldman’s aphorism: nobody knows anything. 

Yet Profs Mellers, Tetlock and Don Moore then ran a larger forecasting tournament and discovered that a small number of people seem to be able to forecast better than the rest of us. These so-called superforecasters are not necessarily subject-matter experts, but they tend to be proactively open-minded, always looking for contrary evidence or opinions. 

There are certain mental virtues, then, that make people better forecasters. The new research turns the question around: might trying to become a better forecaster strengthen such mental virtues? In particular, might it make us less polarised in our political views? 

Of course there is nothing particularly virtuous about many of the forecasts we make, which are often pure bluff, attention-seeking or cheerleading. “We are going to make America so great again” (Donald Trump, February 2016); “There will be no downside to Brexit, only a considerable upside” ( David Davis, October 2016); “If this exit poll is right . . . I will publicly eat my hat” (Paddy Ashdown, May 2015). These may all be statements about the future, but it seems reasonable to say that they were never really intended as forecasts. 

A forecasting tournament, on the other hand, rewards a good-faith effort at getting the answer right. A serious forecaster will soon be confronted by the gaps in his or her knowledge. In 2002, psychologists Leonid Rozenblit and Frank Keil coined the phrase “the illusion of explanatory depth”. If you ask people to explain how a flush lavatory actually works (or a helicopter, or a sewing machine) they will quickly find it is hard to explain beyond hand-waving. Most parents discover this when faced by questions from curious children. 

Yet subsequent work has shown that asking people to explain how the US Affordable Care Act or the European Single Market work prompts some humility and, with it, political moderation. It seems plausible that thoughtful forecasting has a similar effect. 

Good forecasters are obliged to consider different scenarios. Few prospects in a forecasting tournament are certainties. A forecaster may believe that parliament is likely to reject the deal the UK has negotiated with the EU, but he or she must seriously evaluate the alternative. Under which circumstances might parliament accept the deal instead? Again, pondering alternative scenarios and viewpoints has been shown to reduce our natural overconfidence. 

My own experience with scenario planning — a very different type of futurology than a forecasting tournament — suggests another benefit of exploring the future. If the issue at hand is contentious, it can feel safer and less confrontational to talk about future possibilities than to argue about the present. 

It may not be so surprising, then, that Profs Mellers, Tetlock and Arkes found that forecasting reduces political polarisation. They recruited people to participate in a multi-month forecasting tournament, then randomly assigned some to the tournament and some to a non-forecasting control group. (A sample question: “Will President Trump announce that the US will pull out of the Trans-Pacific Partnership during the first 100 days of his administration?”) 

At the end of the experiment, the forecasters had moderated their views on a variety of policy domains. They also tempered their inclination to presume the opposite side was packed with extremists. Forecasting, it seems, is an antidote to political tribalism. 

Of course, centrism is not always a virtue and, if forecasting tournaments are a cure for tribalism, then they are a course of treatment that lasts months. Yet the research is a reminder that not all forecasters are blowhards and bluffers. Thinking seriously about the future requires keeping an open mind, understanding what you don’t know, and seeing things as others see them. If the end result is a good forecast, perhaps we should see that as the icing on the cake.