Economists had a dreadful 2023

From The Economist 

Spare a thought for economists. Last Christmas they were an unusually pessimistic lot: the growth they expected in America over the next calendar year was the fourth-lowest in 55 years of fourth-quarter surveys. Many expected recession; The Economist added to the prognostications of doom and gloom. This year economists must swap figgy pudding for humble pie, because America has probably grown by an above-trend 3%—about the same as in boomy 2005. Adding to the impression of befuddlement, most analysts were caught out on December 13th by a doveish turn by the Federal Reserve, which sent them scrambling to rewrite their outlooks for the new year.

It is not just forecasters who have had a bad year. Economists who deal in sober empirical work have also had their conclusions challenged. Consider research on inequality. Perhaps the most famous economic studies of the past 20 years have been those by Thomas Piketty and his co-authors, who have found a rising gap between rich and poor. But in November a paper finding that after taxes and transfers American incomes are barely less equal than in the 1960s was accepted for publication by one of the discipline’s top journals. Now Mr Piketty’s faction is on the defensive, accusing its critics of “inequality denial”.

Economists have long agreed that America would be richer if it allowed more homes to be built around popular cities. There is lots of evidence to that effect. But the best-known estimate of the costs of restricting construction has been called into question. Chang-Tai Hsieh of the University of Chicago and Enrico Moretti of the University of California, Berkeley, found that easing building rules in New York, San Francisco and San Jose would have boosted American gdp in 2009 by 3.7%. Now Brian Greaney of the University of Washington claims that after correcting for mistakes the true estimated effect is just 0.02%. If builders disagreed as wildly about roof measurements, the house would collapse.

Think social mobility in America is lower than it was in the freewheeling 19th century, when young men could go West? Think again, according to research by Zachary Ward of Baylor University. He has updated estimates of intergenerational mobility between 1850 and 1940 to account for the fact that past studies tended to look only at white people, as well as correcting other measurement errors. It now looks as if there is more equality of opportunity today than in the past (albeit only because the past was worse than was thought).

A rise in suicides, overdoses and liver disease has reduced life expectancy for white Americans. Angus Deaton and Anne Case of Princeton University popularised the idea that these are “deaths of despair”, rooted in grimmer life prospects for those without college degrees. But economists have been losing faith in the idea that overdoses, which are probably the biggest killer of Americans aged 18-49, have much to do with changes in the labour market. New research has instead blamed the carnage on simple proximity to smuggled fentanyl, a powerful opioid.

Other findings are also looking shaky. The long decline in the prestige of the once-faddish field of behavioural economics, which studies irrationality, continued in 2023. In June Harvard Business School said it believed, after an investigation, that some of the results in four papers co-written by Francesca Gino, a behavioural scientist and phd economist, were “invalid”, owing to “alterations of the data”. (Ms Gino, who has written a book about why it pays to break rules, is suing for defamation the university and the bloggers who exposed the alleged fiddling.)

What lessons should be drawn from economists’ tumultuous year? One is that for all their intellectual discipline they are still human. Replicating existing studies and checking them for errors is crucial work.

Another lesson is that disdain for economic theory in favour of the supposed realism of empirical studies may have gone too far. After the global financial crisis of 2007-09, commentators heaped opprobrium on theorists’ common assumption that people make rational predictions about the world; gibes about an unrealistic, utility-maximising Homo economicus helped raise the status of behavioural economics. Yet rational-expectations models allow for the possibility that inflation can fall rapidly without a recession—exactly the scenario that caught out forecasters in 2023.

A last lesson is that economists should cheer up. The research that has been called into question this year inspired much pessimism about the state of modern capitalism. But a dodged recession, flatter inequality trends and less despair would all be good news. Perhaps the dismal science should be a little less so. ■

What the “superforecasters” predict for major events in 2024

 The experts at Good Judgment weigh in on the coming year thanks to The Economist

Nov 13th 2023

Journalists and commentators often make predictions about the future using ambiguous, carefully chosen words. Other forecasters prefer the more precise language of numbers. Good Judgment, a forecasting firm, has recruited many such people to its team of superforecasters, who work together to provide detailed, specific forecasts. Here are their predictions for events in 2024.

image: the economist

image: the economist

image: the economist

Forecasting winner

Congratulations to Zane Stucker, a legal professional based in the New York metro area, who has won The World Ahead 2023 forecasting challenge organised in collaboration with Good Judgment. Like previous winners, he has been invited to join Good Judgment’s professional superforecasting team.

Could you be a superforecaster, too? Test your own prediction skills in our 2024 forecasting challenge, which runs until October 2024 at gjopen.com/economist

Is Raghuram Rajan wrong about his economic forecasts for India?

 

A level Economics: How to Improve Economic Forecasting

 Nicholas Gruen in The FT 

Today’s four-day weather forecasts are as accurate as one-day forecasts were 30 years ago. Economic forecasts, on the other hand, aren’t noticeably better. Former Federal Reserve chair Ben Bernanke should ponder this in his forthcoming review of the Bank of England’s forecasting. 

There’s growing evidence that we can improve. But myopia and complacency get in the way. Myopia is an issue because economists think technical expertise is the essence of good forecasting when, actually, two things matter more: forecasters’ understanding of the limits of their expertise and their judgment in handling those limits. 

Enter Philip Tetlock, whose 2005 book on geopolitical forecasting showed how little experts added to forecasting done by informed non-experts. To compare forecasts between the two groups, he forced participants to drop their vague weasel words — “probably”, “can’t be ruled out” — and specify exactly what they were forecasting and with what probability.  

That started sorting the sheep from the goats. The simple “point forecasts” provided by economists — such as “growth will be 3.0 per cent” — are doubly unhelpful in this regard. They’re silent about what success looks like. If I have forecast 3.0 per cent growth and actual growth comes in at 3.2 per cent — did I succeed or fail? Such predictions also don’t tell us how confident the forecaster is. 

By contrast, “a 70 per cent chance of rain” specifies a clear event with a precise estimation of the weather forecaster’s confidence. Having rigorously specified the rules of the game, Tetlock has since shown how what he calls “superforecasting” is possible and how diverse teams of superforecasters do even better.  

What qualities does Tetlock see in superforecasters? As well as mastering necessary formal techniques, they’re open-minded, careful, curious and self-critical — in other words, they’re not complacent. Aware, like Socrates, of how little they know, they’re constantly seeking to learn — from unfolding events and from colleagues. 

Superforecasters actively resist the pull to groupthink, which is never far away in most organisations — or indeed, in the profession of economics as a whole, as practitioners compensate for their ignorance by keeping close to the herd. The global financial crisis is just one example of an event that economists collectively failed to warn the world about. 

There are just five pages referencing superforecasting on the entire Bank of England website — though that’s more than other central banks. 

Bernanke could recommend that we finally set about the search for economic superforecasters. He should also propose that the BoE lead the world by open sourcing economic forecasting.  

In this scenario, all models used would be released fully documented and a “prediction tournament” would focus on the key forecasts. Outsiders would be encouraged to enter the tournament — offering their own forecasts, their own models and their own reconfiguration or re-parameterisation of the BoE’s models. Prizes could be offered for the best teams and the best schools and universities.  

The BoE’s forecasting team(s) should also compete. The BoE could then release its official forecasts using the work it has the most confidence in, whether it is that of its own team(s), outsiders or some hybrid option. Over time, we’d be able to identify which ones were consistently better.  

Using this formula, I predict that the Bank of England’s official forecasts would find their way towards the top of the class — in the UK, and the world.

The Only Function of Economic Forecasting Is To Make Astrology Look Respectable

 Tim Harford in The FT 

Economist Ezra Solomon once quipped that “the only function of economic forecasting is to make astrology look respectable”. I’m not sure if the astrologer “Mystic Meg” was ever respectable, but she was certainly much loved. “Britain’s most famous astrologer by a million miles,” said her agent, after her recent death prompted an outpouring of affectionate recollections about her campy image and her arch forecasts about the National Lottery, praised both for their brilliant accuracy and sheer absurdity. 

It seems hard to imagine that an economic forecaster will ever earn such valedictions. But many economic pundits seem to have been taking lessons from astrologers. Consider this horoscope: “The balance of risks remains tilted to the downside, but adverse risks have moderated . . . On the upside, a stronger boost from pent-up demand in numerous economies or a faster fall in inflation are plausible. On the downside, severe health outcomes in China could hold back the recovery . . . ” 

That pretty much covers everything: good news, bad news, more inflation, disinflation. In case you’re wondering, it’s the latest World Economic Outlook from the IMF. But that sort of “rainbow forecast” is typical of the genre. 

Forecasting expert Philip E Tetlock, in his 2005 book Expert Political Judgement, noted that expert pundits had a tendency to make vague forecasts, and to excuse error as “erring on the side of caution” or being wrong only on timing. 

If so, those experts are treading a well-worn path. Consider the following statements: “You have a great need for other people to like and admire you.” “You have a tendency to be critical of yourself.” “While you have some personality weaknesses, you are generally able to compensate for them.” 

They sound like the kind of thing a clairvoyant might say after gazing into a crystal ball, but these statements are from an academic paper, “The Fallacy of Personal Validation”, published in 1949 by psychologist Bertram Forer. 

After getting his students to fill out a diagnostic questionnaire, Forer handed each of them a written assessment of their traits. The students believed the assessments were uniquely tailored on the basis of the questionnaire. But, in fact, each student got the same list of 13 statements, including the three above. The students felt the diagnostic had done an excellent job, and the vast majority agreed with at least 10 out of 13 statements. When the deception was revealed, wrote Forer, “they burst into laughter”. These “Forer statements” — also sometimes called “Barnum statements” after showman PT Barnum — can feel uncannily specific. Most people don’t realise that they are almost universal. 

In defence of economic forecasters, including the IMF, Barnumesque verbiage is traditionally accompanied by specific falsifiable numerical predictions. Surely, the real incorrigibles are the economics columnists. We’ll blithely hand-wave about risks and opportunities which may or may not manifest. And like Mystic Meg, we’re kept around only because people find our prognostications entertaining. 

The parallels should be no surprise. Walter Friedman’s history of economic forecasting, Fortune Tellers, explains that clairvoyants and economic forecasters started from a similar place. Evangeline Adams and Roger Babson were near contemporaries, born in the US in 1868 and 1875 respectively. Both offered investment advice in general and stock market forecasts in particular. Both were in high demand, and both died rich. The chief difference was that Adams was an astrologer, while Babson offered data-driven forecasts inspired by ideas from physics. 

Babson’s forecasting ideas look very strange today. He was a huge fan of Isaac Newton: he purchased and moved the parlour of Newton’s house from London to Massachusetts, funded research into antigravity, and his forecasting ideas are full of misappropriated Newtonian physics. His “Babsonchart” was built around the Newtonish idea that each boom above the trend was followed by an equal and opposite bust below. With hindsight, this was true by definition when Babson plotted the trend line in the right place. Alas, it offered little predictive power beyond generalities. 

Still, generalities will get you a long way. Babson’s reputation as a forecaster was secured when, on September 5 1929, a few weeks before the great crash, he opined, “sooner or later a crash is coming which will . . . cause a decline of from 60 to 80 points in the Dow-Jones Barometer”. Impressive. 

What is less impressive is that those gloomy forecasts began years earlier, in 1926, after which the Dow more than doubled. The crash was vastly bigger than Babson had predicted, and it continued long after Babson started predicting a recovery. 

No matter. Shortly after the crash began, Babson ran an advert in The New York Times announcing that “Babson clients were prepared” and he still gets credit for predicting the crash. Aficionados of clairvoyancy will recognise some similarities here. If you want to be admired for your forecasts, temper your bold claims with vagueness and be sure to trumpet the successes and downplay the failures. 

No sooner had Mystic Meg’s death been announced than The Sun, which published her column, was explaining that her final horoscope was a “sweet prediction” that she would be reunited in the afterlife with an old flame who died in a car crash in 1977. “Leo: It can be the most routine of routine journeys that takes you towards your soulmate.” 

For those readers willing to swallow the idea that death itself is “the most routine of routine journeys”, it’s a startling piece of prescience. For the rest of us, it’s audacious silliness. Mystic Meg would have been proud.

Unreliable Macroeconomic Forecasts and Corporate Business Plans

Anne-Sylvaine Chassany in The FT

Top Ikea executive Jesper Brodin says he is not usually one to indulge in nostalgia. But at a pre-Christmas gathering for senior managers that used to work at the Swedish furniture group, he could not help but join with the chorus of those who said they missed the old times — when the world seemed relatively stable, trends were predictable and this could be translated into a more or less credible multiyear business plan. 

“We always debate whether it was better before. I used to always argue it is better now. This time we tended to agree it was better before,” he said. “The risks, the uncertainty, everything that used to be in a ‘risk matrix file’ is more or less happening . . . We laugh about the time when we were doing one-year budgets, and how we would be right or wrong by 0.3 per cent.” 

Brodin’s reflections resonate across the corporate world. CEOs are struggling to make sense of confusing macroeconomic signals. In Europe and the US, an economic downturn is combined with record low unemployment and labour shortages. Consumer behaviour is a mystery: up until recently people have kept spending even though the price of almost everything has gone up. 

The worst predictions of economic crisis and energy shortages from last year have not materialised. But it feels uniquely hard to predict the path ahead at the moment. On both sides of the Atlantic, little consensus is heard about where the economy is going, and for listed businesses, delivering guidance to the market is more difficult than ever. In the UK, auditing firms worry that the forecasts their corporate clients submit to them for sign-off are impossible to assess. 

In the game of adjusting to these new forms of chaos, some are better placed than others. Generally pressure is less on privately owned companies that do not have to publish profit targets. 

Ikea, for instance, has changed tack. Instead of setting out specific goals for the year, it has a set of “scenarios” to give the business wiggle room as the outlook changes. It means acknowledging that widely different outcomes are possible. “It’s teaching us agility in how we operate,” said Brodin. 

A year ago, the 54-year-old firm expected customers to cut spending because of high energy bills and mortgage rates. That did not happen. Meanwhile, supply chain disruptions improved more quickly than anticipated, leaving the group with more inventory and, in turn, the need to lower the prices of some of its products. 

“We are celebrating that things are going in the right direction,” said Brodin, “but we have no concept of predicting with precision what’s going to happen in 6 to 12 months.” 

For Ikea, input costs are the trickiest to forecast. Transport prices have fallen. But Brodin did not expect that greater demand for wood to burn as fuel would make some of the company’s materials more expensive. 

It is not just the traditional variables of financial modelling such as inflation and consumer spending that have become harder to predict. The past few years have also provided some unexpected lessons on how business and society cope with shocks and uncertainty. 

“Look at what people have gone through: the pandemic, the economic damage, the tragedy of war, energy prices,” Brodin said. “What people might have underestimated is human resilience.”

Covid is forcing economists to look at other disciplines for recovery clues

Larry Elliot in The Guardian

Three times a week an update on new Covid-19 cases is published by the economics consultancy Pantheon. Vaccination rates are monitored by the Swiss bank UBS. The scientists advising the government are in regular contact with the Bank of England’s monetary policy committee – the body that sets interest rates.

Richard Nixon may or may not have said “we are all Keynesians now” after the US broke its link with gold in 1971 but one thing is for sure: all economists are epidemiologists now. And there’s a downside and an upside to that.

The downside is that economic forecasting is currently even more of a mug’s game than usual because even the real (as opposed to the amateur) epidemiologists don’t really know what is going to happen next. Are there going to be new mutations of the virus? Assuming there are, will they be less susceptible to vaccines? Will Covid-19 go away in the summer only to return again as the days get shorter, as happened last year? Nobody really knows the answers to those questions.

The upside is that the pandemic has forced economists to look beyond their mechanical models and embrace thinking from other disciplines, of which epidemiology is just one.

For a start, it is hard to estimate how people are going to react to the easing of lockdown restrictions without some help from psychologists. It is possible that there will be an explosion of spending as consumers, in the words of Andrew Bailey, “go for it”, but it is also possible that the second wave of infection will make them a lot more cautious than they were last summer, when there was still hope that Covid-19 was a fleeting phenomenon.

An individual’s behaviour is also not entirely driven by their own economic circumstances. It can be strongly affected by what others are doing. If your peer group decides after having the vaccine that it is safe to go to the pub, that will probably affect your decision about whether to join your mates for a drink, even if you are slightly nervous. Sociology has a part to play in economic forecasting.

As does history, if only to a limited extent, because there are not a lot of comparable episodes to draw upon. A century has passed since the last truly global pandemic and there is only so much that can be learned from the outbreak of Spanish flu after the first world war. But when Andy Haldane, the chief economist of the Bank of England, says the economy is like a coiled spring waiting to be unleashed, that’s because he thinks there are lessons to be learned from the rapid recovery seen last summer. Back then, the economy followed a near 19% collapse in the second quarter of 2020 with a 16% jump in the third quarter.

Naturally, economics has a part to play in judging what happens next. Millions of people (mostly the better off) have remained in work on full pay for the past year but have struggled to find anything to spend their money on. Millions of others – those furloughed on 80% of their normal wages or self-employed people who have slipped through the Treasury’s safety net – are less well-off than they were a year ago and may fear for their job prospects.

In an ideal world, the better-off would decide that the amount of money saved during lockdown was far in excess of what they needed and would then go on a spending spree: heading out for meals, taking weekend breaks, buying new cars; having their homes re-decorated. That would provide jobs and incomes for those on lower incomes.

But it might not work out like that. If the better-off leave their accumulated savings (or most of them, at least) in the bank, that means higher unemployment for those working in consumer-facing services jobs – such as hotels and restaurants – and an economy with a dose of long Covid.

There are two conclusions to be drawn from all of this. The first is that precise forecasts of what is going to happen to the economy over the next year, or even the next few months, should be treated with caution. Assuming the vaccination programme continues to go well, assuming that there are no further waves of infection, assuming restrictions are lifted steadily from early March onwards, and assuming that people come out of hibernation rapidly and in numbers, then the economy will start to recover in the second quarter. But there are a heck of a lot of assumptions in there: it might take until the third quarter for the bounce back to begin; the recovery might prove weaker or stronger than the consensus currently expects.

The second conclusion is equally obvious. If, as is clearly the case, the existence of so many imponderables makes precision forecasting more difficult than normal, it makes sense for economic policy makers to act with caution. For the Bank of England, that means no dash to embrace negative interest rates, which won’t be necessary if Haldane’s bullishness proves to be justified; and for the Treasury it means extending financial support and ignoring calls for higher taxes, especially those that might lead businesses to collapse or cut back on investment.

It would appear that Rishi Sunak has reached the same conclusion. There has been far less talk from the chancellor recently about the need to reduce the UK’s budget deficit, a process that has now been delayed until the second budget of 2021 in the autumn. By that stage, it might well once again by Sunak rather than the epidemiologists running the economy. Well, perhaps.

Economists should learn lessons from meteorologists

Weather forecasters make hypotheses and test them daily writes Tim Harford in The FT


The UK’s national weather service, the Met Office, is to get a £1.2bn computer to help with its forecasting activities. That is a lot of silicon. My instinctive response was: when do we economists get one? 

People may grumble about the weather forecast, but in many places we take its accuracy for granted. When we ask our phones about tomorrow’s weather, we act as though we are gazing through a window into the future. Nobody treats the latest forecasts from the Bank of England or the IMF as a window into anything. 

That is partly because politics gets in the way. On the issue of Brexit, for example, extreme forecasts from partisans attracted attention, while independent mainstream forecasters have proved to be pretty much on the money. Few people stopped to praise the economic bean-counters. 

Economists might also protest that nobody asks them to forecast economic activity tomorrow or even next week; they are asked to describe the prospects for the next year or so. True, some almanacs offer long-range weather forecasts based on methods that are secret, arcane, or both — but the professionals regard such attempts as laughable. 

Enough excuses; economists deserve few prizes for prediction. Prakash Loungani of the IMF has conducted several reviews of mainstream forecasts, finding them dismally likely to miss recessions. Economists are not very good at seeing into the future — to the extent that most argue forecasting is simply none of their business. The weather forecasters are good, and getting better all the time. Could we economists do as well with a couple of billion dollars’ worth of kit, or is something else lacking? 

The question seemed worth exploring to me, so I picked up Andrew Blum’s recent book, The Weather Machine, to understand what meteorologists actually do and how they do it. I realised quickly that a weather forecast is intimately connected to a map in a way that an economic forecast is not. 

Without wishing to oversimplify the remarkable science of meteorology, one part of the game is straightforward: if it’s raining to the west of you and the wind is blowing from the west, you can expect rain soon. Weather forecasts begin with weather observations: the more observations, the better. 

In the 1850s, the Smithsonian Institution in Washington DC used reports from telegraph operators to patch together local downpours into a national weather map. More than a century and a half later, economists still lack high-definition, high-frequency maps of the economic weather, although we are starting to see how they might be possible, tapping into data from satellites and digital payments. 

An example is an attempt — published in 2012 — by a large team of economists to build a simulation of the Washington DC housing market as a complex system. It seems a long way from a full understanding of the economy, but then the Smithsonian’s paper map was a long way from a proper weather forecast, too. 

Weather forecasters could argue that they have a better theory of atmospheric conditions than economists have of the economy. It was all sketched out in 1904 by the Norwegian mathematician Vilhelm Bjerknes, who published “The problem of weather prediction”, an academic paper describing the circulation of masses of air. If you knew the density, pressure, temperature, humidity and the velocity of the air in three dimensions, and plugged the results into Bjerknes’s formulas, you would be on the way to a respectable weather forecast — if only you could solve those computationally-demanding equations. The processing power to do so was to arrive many decades later. 

The missing pieces, then: much better, more detailed and more frequent data. Better theory too, perhaps — although it is striking that many critiques of the economic mainstream seem to have little interest in high-resolution, high frequency data. Instead, they propose replacing one broad theory with another broad theory: the latest one I have seen emphasises “the energy cost of energy”. I am not sure that is the path to progress. 

The weather forecasters have another advantage: a habit of relentless improvement in the face of frequent feedback. Every morning’s forecast is a hypothesis to be tested. Every evening that hypothesis has been confirmed or refuted. If the economy offered similar daily lessons, economists might be quicker to learn. All these elements are linked. If we had more detailed data we might formulate more detailed theories, building an economic map from the bottom up rather than from the top down. And if we had more frequent feedback, we could test theories more often, making economics more empirical and less ideological. 

And yet — does anyone really want to spend a billion pounds on an economic simulation that will accurately predict the economic weather next week? Perhaps the limitations of economic forecasting reflect the limitations of the economics profession. Or perhaps the problem really is intractable.

Why good forecasters become better people

Tim Harford in The FT

So, what’s going to happen next, eh? Hard to say: the future has a lotta ins, a lotta outs, a lotta what-have-yous. 

Perhaps I should be more willing to make bold forecasts. I see my peers forecasting all kinds of things with a confidence that only seems to add to their credibility. Bad forecasts are usually forgotten and you can milk a spectacular success for years. 

Yet forecasts are the junk food of political and economic analysis: tasty to consume but neither satisfying nor healthy in the long run. So why should they be any more wholesome to produce? The answer, it seems, is that those who habitually make forecasts may turn into better people. That is the conclusion suggested by a research paper from three psychologists, Barbara Mellers, Philip Tetlock and Hal Arkes. 

Prof Tetlock won attention for his 2005 book Expert Political Judgment, which used the simple method of asking a few hundred experts to make specific, time-limited forecasts such as “Will Italy’s government debt/GDP ratio be between 70 and 90 per cent in December 1998?” or “Will Saddam Hussein be the president of Iraq on Dec 31 2002?” 

It is only a modest oversimplification to summarise Prof Tetlock’s results using the late William Goldman’s aphorism: nobody knows anything. 

Yet Profs Mellers, Tetlock and Don Moore then ran a larger forecasting tournament and discovered that a small number of people seem to be able to forecast better than the rest of us. These so-called superforecasters are not necessarily subject-matter experts, but they tend to be proactively open-minded, always looking for contrary evidence or opinions. 

There are certain mental virtues, then, that make people better forecasters. The new research turns the question around: might trying to become a better forecaster strengthen such mental virtues? In particular, might it make us less polarised in our political views? 

Of course there is nothing particularly virtuous about many of the forecasts we make, which are often pure bluff, attention-seeking or cheerleading. “We are going to make America so great again” (Donald Trump, February 2016); “There will be no downside to Brexit, only a considerable upside” ( David Davis, October 2016); “If this exit poll is right . . . I will publicly eat my hat” (Paddy Ashdown, May 2015). These may all be statements about the future, but it seems reasonable to say that they were never really intended as forecasts. 

A forecasting tournament, on the other hand, rewards a good-faith effort at getting the answer right. A serious forecaster will soon be confronted by the gaps in his or her knowledge. In 2002, psychologists Leonid Rozenblit and Frank Keil coined the phrase “the illusion of explanatory depth”. If you ask people to explain how a flush lavatory actually works (or a helicopter, or a sewing machine) they will quickly find it is hard to explain beyond hand-waving. Most parents discover this when faced by questions from curious children. 

Yet subsequent work has shown that asking people to explain how the US Affordable Care Act or the European Single Market work prompts some humility and, with it, political moderation. It seems plausible that thoughtful forecasting has a similar effect. 

Good forecasters are obliged to consider different scenarios. Few prospects in a forecasting tournament are certainties. A forecaster may believe that parliament is likely to reject the deal the UK has negotiated with the EU, but he or she must seriously evaluate the alternative. Under which circumstances might parliament accept the deal instead? Again, pondering alternative scenarios and viewpoints has been shown to reduce our natural overconfidence. 

My own experience with scenario planning — a very different type of futurology than a forecasting tournament — suggests another benefit of exploring the future. If the issue at hand is contentious, it can feel safer and less confrontational to talk about future possibilities than to argue about the present. 

It may not be so surprising, then, that Profs Mellers, Tetlock and Arkes found that forecasting reduces political polarisation. They recruited people to participate in a multi-month forecasting tournament, then randomly assigned some to the tournament and some to a non-forecasting control group. (A sample question: “Will President Trump announce that the US will pull out of the Trans-Pacific Partnership during the first 100 days of his administration?”) 

At the end of the experiment, the forecasters had moderated their views on a variety of policy domains. They also tempered their inclination to presume the opposite side was packed with extremists. Forecasting, it seems, is an antidote to political tribalism. 

Of course, centrism is not always a virtue and, if forecasting tournaments are a cure for tribalism, then they are a course of treatment that lasts months. Yet the research is a reminder that not all forecasters are blowhards and bluffers. Thinking seriously about the future requires keeping an open mind, understanding what you don’t know, and seeing things as others see them. If the end result is a good forecast, perhaps we should see that as the icing on the cake.

I can make one confident prediction: my forecasts will fail

Tim Harford in The Financial Times 

I am not one of those clever people who claims to have seen the 2008 financial crisis coming, but by this time 10 years ago I could see that the fallout was going to be bad. Banking crises are always damaging, and this was a big one. The depth of the recession and the long-lasting hit to productivity came as no surprise to me. I knew it would happen. 

Or did I? This is the story I tell myself, but if I am honest I do not really know. I did not keep a diary, and so must rely on my memory — which, it turns out, is not a reliable servant. 

In 1972, the psychologists Baruch Fischhoff and Ruth Beyth conducted a survey in which they asked for predictions about Richard Nixon’s imminent presidential visit to China and Russia. How likely was it that Nixon and Mao Zedong would meet? What were the chances that the US would grant diplomatic recognition to China? Professors Fischhoff and Beyth wanted to know how people would later remember their forecasts. Since their subjects had taken the unusual step of writing down a specific probability for each of 15 outcomes, one might have hoped for accuracy. But no — the subjects flattered themselves hopelessly. The Fischhoff-Beyth paper was titled, “I knew it would happen”. 

This is a reminder of what a difficult task we face when we try to make big-picture macroeconomic and geopolitical forecasts. To start with, the world is a complicated place, which makes predictions challenging. For many of the subjects that interest us, there is a substantial delay between the forecast and the outcome, and this delayed feedback makes it harder to learn from our successes and failures. Even worse, as Profs Fischhoff and Beyth discovered, we systematically misremember what we once believed. 

Small wonder that forecasters turn to computers for help. We have also known for a long time — since work in the 1950s by the late psychologist Paul Meehl — that simple statistical rules often outperform expert intuition. Meehl’s initial work focused on clinical cases — for example, faced with a patient suffering chest pains, could a two or three-point checklist beat the judgment of an expert doctor? The experts did not fare well. However, Meehl’s rules, like more modern machine learning systems, require data to work. It is all very well for Amazon to forecast what impact a price drop may have on the demand for a book — and some of the most successful hedge funds use algorithmically-driven strategies — but trying to forecast the chance of Italy leaving the eurozone, or Donald Trump’s impeachment, is not as simple. Faced with an unprecedented situation, machines are no better than we are. And they may be worse. 

Much of what we know about forecasting in a complex world, we know from the research of the psychologist Philip Tetlock. In the 1980s, Prof Tetlock began to build on the Fischhoff-Beyth research by soliciting specific and often long-term forecasts from a wide variety of forecasters — initially hundreds. The early results, described in Prof Tetlock’s book Expert Political Judgement, were not encouraging. Yet his idea of evaluating large numbers of forecasters over an extended period of time has blossomed, and some successful forecasters have emerged. 

The latest step in this research is a “Hybrid Forecasting Tournament”, sponsored by the US Intelligence Advanced Research Projects Activity, designed to explore ways in which humans and machine learning systems can co-operate to produce better forecasts. We await the results. If the computers do produce some insight, it may be because they can tap into data that we could hardly have imagined using before. Satellite imaging can now track the growth of crops or the stockpiling of commodities such as oil. Computers can guess at human sentiment by analysing web searches for terms such as “job seekers allowance”, mentions of “recession” in news stories, and positive emotions in tweets. 

And there are stranger correlations, too. A study by economists Kasey Buckles, Daniel Hungerman and Steven Lugauer showed that a few quarters before an economic downturn in the US, the rate of conceptions also falls. Conceptions themselves may be deducible by computers tracking sales of pregnancy tests and folic acid. 

Back in 1991, a psychologist named Harold Zullow published research suggesting that the emotional content of songs in the Billboard Hot 100 chart could predict recessions. Hits containing “pessimistic rumination” (“I heard it through the grapevine / Not much longer would you be mine”) tended to predict an economic downturn. 

His successor is a young economist named Hisam Sabouni, who reckons that a computer-aided analysis of Spotify streaming gives him an edge in forecasting stock market movements and consumer sentiment. Will any of this prove useful for forecasting significant economic and political events? Perhaps. But for now, here is an easy way to use a computer to help you forecast: open up a spreadsheet, note down what you believe today, and regularly revisit and reflect. The simplest forecasting tip of all is to keep score.

Analysts caught off guard by 41% Capita share drop

Cat Rutter Pooley in The Financial Times

There may be some red-faced analysts across the City this morning. 

Only two out of 16 analysts polled by Bloomberg had a sell rating on Capita before today, when its shares plummeted 41 per cent on a profit warning and planned £700m rights issue. 

Of the rest, 11 had a hold rating and three a buy rating. 

One of those buy recommendations came from Numis, which issued its note on the company two weeks ago. 

Then, Numis described a meeting with the new Capita chief executive as “positive”, noting that: 

 It is easy to be critical of the past, but his observations on some of the structural and cultural issues at Capita highlighted some fundamental problems, but also material opportunities. We were encouraged by [Jonathan Lewis’s] comments on the need for great focus, cost reductions (whilst also re-investing for growth), and need to focus on cash. 

Numis declined to comment immediately on whether it was reviewing the recommendation in light of the company’s update. 

Jefferies, which has also had a ‘buy’ recommendation on the stock, characterised Wednesday’s announcement as a “kitchen sinking”, or effort to cram all the bad news out at once. The revelations could generate a 40 per cent decline in earnings expectations for the full year, it said, adding that the revenue environment remained “lacklustre”. 

Shares are current trading around 210p, down 40 per cent. 

Meanwhile, the ripples from Capita’s share price drop are leaking across the outsourcing industry. Serco slipped 3 per cent, and Mitie was down 2.4 per cent at pixel time.

From climate change to robots: what politicians aren’t telling us

Simon Kuper in The Financial Times

On US television news this autumn, wildfires and hurricanes have replaced terrorism and — mostly — even mass shootings as primetime content. Climate change is making natural disasters more frequent, and more Americans now live in at-risk areas. But meanwhile, Donald Trump argues on Twitter about what he supposedly said to a soldier’s widow. So far, Trump is dangerous less because of what he says (hot air) or does (little) than because of the issues he ignores. 

He’s not alone: politics in many western countries has become a displacement activity. Most politicians bang on about identity while ignoring automation, climate change and the imminent revolution in medicine. They talk more about the 1950s than the 2020s. This is partly because they want to distract voters from real problems, and partly because today’s politicians tend to be lawyers, entertainers and ex-journalists who know less about tech than the average 14-year-old. (Trump said in a sworn deposition in 2007 that he didn’t own a computer; his secretary sent his emails.) But the new forces are already transforming politics. 

Ironically, given the volume of American climate denial, the US looks like becoming the first western country to be hit by climate change. Each new natural disaster will prompt political squabbles over whether Washington should bail out the stricken region. At-risk cities such as Miami and New Orleans will gradually lose appeal as the risks become uninsurable. If you buy an apartment on Miami Beach now, are you confident it will survive another 30 years undamaged? And who will want to buy it from you in 2047? Miami could fade as Detroit did. 

American climate denial may fade too, as tech companies displace Big Oil as the country’s chief lobbyists. Already in the first half of this year, Amazon outspent Exxon and Walmart on lobbying. Facebook, now taking a kicking over fake news, will lobby its way back. Meanwhile, northern Europe, for some years at least, will benefit from its historical unique selling point: its mild and rainy climate. Its problem will be that millions of Africans will try to move there. 

On the upside, many Africans will soon, for the first time ever, have access to energy (thanks to solar panels) and medical care (as apps monitor everything from blood pressure to sugar levels, and instantly prescribe treatment). But as Africa gets hotter, drier and overpopulated, people will struggle to feed themselves, says the United Nations University. So they will head north, in much greater numbers than Syrians have, becoming the new bogeymen for European populists. Patrolling robots — possibly with attack capabilities — will guard Fortress Europe. 

Everywhere, automation will continue to eat low-skilled jobs. That will keep people angry. Carl Benedikt Frey of Oxford university’s Martin School recalls workers smashing up machines during the British industrial revolution, and says: “There was a machinery riot last year: it was the US presidential election.” American workers hit by automation overwhelmingly voted Trump, even though he doesn’t talk about robots. 

Soon, working-class men will lose driving jobs to autonomous vehicles. They could find new jobs servicing rich people as cleaners (a profession that’s surprisingly hard to automate), carers or yoga teachers. Young men will develop new notions of masculinity and embrace this traditionally feminine work. But older working-class men will probably embrace politicians like Trump. 

The most coveted good of all — years of life — will become even more unfairly distributed. The lifespans of poor westerners will continue to stagnate or shorten, following the worldwide surge in obesity since the 1980s. Many poorer people will work into their seventies, then die, skipping the now standard phase of retirement. Meanwhile, from the 2020s the rich will live ever longer as they start buying precision medicine. They will fix their faulty DNA and edit their embryos, predicts Vivek Wadhwa, thinker on technology. (I heard him and Frey at this month’s excellent Khazanah Megatrends Forum in Malaysia.) Even if governments want to redress inequality, they won’t be able to, given that paying tax has become almost voluntary for global companies. 

The country hit hardest by automation could be China (though Germany could suffer too, especially if its carmakers fail to transform). China’s model of exploiting cheap factory labour without environmental regulations has run its course, says Wadhwa. “I don’t think we need Chinese robots.” Even if China’s economy keeps growing, low-skilled men won’t find appealing careers, and they won’t even have the option of electing a pretend system-smasher like Trump. The most likely outcome: China’s regime joins the populist trend and runs with aggressive nationalism. 

Troubled regimes will also ratchet up surveillance. Now they merely know what you say. In 10 years, thanks to your devices, they will know your next move even before you do. Already, satellites are monitoring Egypt’s wheat fields, so as to predict the harvest, which predicts the chance of social strife. Meanwhile, western politicians will probably keep obsessing over newsy identity issues. My prediction for the 2020s: moral panics over virtual-reality sex.