Sunday, 29 January 2017

‘Trump makes sense to a grocery store owner’ N N Taleb

Suhasini Haider in The Hindu

Economist-mathematician Nassim Nicholas Taleb contends that there is a global riot against pseudo-experts

After predicting the 2008 economic crisis, the Brexit vote, the U.S. presidential election and other events correctly, Nassim Nicholas Taleb, author of the Incerto series on global uncertainties, which includes The Black Swan: The Impact of the Highly Improbable, is seen as something of a maverick and an oracle. Equally, the economist-mathematician has been criticised for advocating a “dumbing down” of the economic system, and his reasoning for U.S. President Donald Trump and global populist movements. In an interview in Jaipur, Taleb explains why he thinks the world is seeing a “global riot against pseudo-experts”.

I’d like to start by asking about your next book, Skin in the Game, the fifth of the Incerto series. You do something unusual with your books: before you launch, you put chapters out on your website. Why is that?

Putting my work online motivates me to go deeper into a subject. I put it online and it gives some structure to my thought. The only way to judge a book is by something called the Lindy effect, and that is its survival. My books have survived. I noticed that The Black Swan did well because it was picked up early online, long before the launch. I also prefer social media to interviews in the mainstream media as many journalists don’t do their research, and ‘zeitgeist’ updates [Top Ten lists] pass for journalism.

The media is not one organisation or a monolithic entity.

Well, I’m talking about the United States where I get more credible news from the social media than the mainstream media. But I am very impressed with the Indian media that seems to present both sides of the story. In the U.S., you only get either the official, bureaucratic or the academic side of the story.

In Skin in the Game, you seem to build on theories from The Black Swan that give a sense of foreboding about the world economy. Do you see another crisis coming?

Oh, absolutely! The last crisis [2008] hasn’t ended yet because they just delayed it. [Barack] Obama is an actor. He looks good, he raises good children, he is respectable. But he didn’t fix the economic system, he put novocaine [local anaesthetic] in the system. He delayed the problem by working with the bankers whom he should have prosecuted. And now we have double the deficit, adjusted for GDP, to create six million jobs, with a massive debt and the system isn’t cured. We retained zero interest rates, and that hasn’t helped. Basically we shifted the problem from the private corporates to the government in the U.S. So, the system remains very fragile.

You say Obama put novocaine in the system. How will the Trump administration be able to address this?

Of course. The whole mandate he got was because he understood the economic problems. People don’t realise that Obama created inequalities when he distorted the system. You can only get rich if you have assets. What Trump is doing is put some kind of business sense in the system. You don’t have to be a genius to see what’s wrong. Instead of Trump being elected, if you went to the local souk [bazaar] in Aleppo and brought one of the retail shop owners, he would do the same thing Trump is doing. Like making a call to Boeing and asking why are we paying so much.

You’re seen as something of an oracle, given that you saw the 2008 economic crash coming, you predicted the Brexit vote, the outcome of the Syrian crisis. You said the Islamic State would benefit if Bashar al-Assad was pushed out and you predicted Trump’s win. How do you explain it?

Not the Islamic State, but al-Qaeda at the time, and I said the U.S. administration was helping fund them. See, you have to have courage to say things others don’t. I was lucky financially in life, that I didn’t need to work for a living and can spend all my time thinking. When Trump was running for election, I said what he says makes sense to a grocery store owner. Because the grocery guy can say Trump is wrong because he can see where he is wrong. But with Obama, he can’t understand what he’s saying, so the grocery man doesn’t know where he is wrong.

Is it a choice between dumbing down versus over-intellectualisation, then?

Exactly. Trump never ran for archbishop, so you never saw anything in his behaviour that was saintly, and that was fine. Whereas Obama behaved like the Archbishop of Canterbury, and was going to do good but people didn’t feel their lives were better. As I said, if it was a shopkeeper from Aleppo, or a grocery store owner in Mumbai, people would have liked them as much as Trump. What he says makes common sense, asking why are we paying so much for this rubbish or why do we need these complex taxes, or why do we want lobbyists. You can call Trump’s plain-speaking what you like. But the way intellectuals treat people who don’t agree with them isn’t good either. I remember I had an academic friend who supported Brexit, and he said he knew what it meant to be a leper in the U.K. It was the same with supporting Trump in the U.S.

But there were valid reasons for people to be worried about Trump too.

Well, if you’re a businessman, for example, what Trump said didn’t bother you. The intellectual class of no more than 2,00,000 people in the U.S. don’t represent everyone upset with Trump. The real problem is the ‘faux-expert problem’, one who doesn’t know what he doesn’t know, and assumes he knows what people think. An electrician doesn’t have that problem.

Is the election of Trump part of a global phenomena? You have commented on the similarity to the election of Narendra Modi in India.

Well, with Trump, Modi, Brexit, and now France, there are some similar problems in those countries. What you are hearing is people getting fed up with the ruling class. This is not fascism. It has nothing to do with fascism. It has to do with the faux-experts problem and a world with too many experts. If we had a different elite, we may not see the same problem.

There are other similarities, to quote from studies of populist movements worldwide: these leaders are majoritarian, they build on resentment, they use social media for direct access to their voters, and they can take radical decisions.
I often say that a mathematician thinks in numbers, a lawyer in laws, and an idiot thinks in words. These words don’t amount to anything. I think you have to draw the conclusion that there is a global riot against pseudo-experts. I saw it with Brexit, and Nigel Farage [leader of the U.K. Independence Party], who was a trader for 15 years, said the problem with the government was that none of them had ever had a proper job. Being a bureaucrat is not a proper job.

As a businessperson, you have a point about experts and pseudo-experts who you say are ‘left-wing’. How do you explain the other parts to the phenomenon that aren’t economic: the xenophobia, Islamophobia, misogyny, etc.?

I don’t understand how a left-wing person can defend Salafism, or religious extremism. In a democracy, you can allow people to have any view, but they can’t come with a message to destroy democracy. Why should people who come to the West come with a message to finish the West? This is where the discourse goes haywire. So in Yemen, the [Saudi] intervention is good, but the intervention [by Russia] in Aleppo shouldn’t be allowed. I don’t think Trump was racist when he said Mexican criminals shouldn’t be allowed into the U.S.; he was targeting criminals. If you are Naziphobic, you are not against Germans. If I oppose Salafism, I am not an Islamophobe. Obama also deported Mexicans and refused to accept immigrants.

Is anti-globalisation a part of this sentiment?

I am not anti-globalisation, but I am against big global corporations. One of the reasons is what they cost. Today, every project sees cost overruns because these projects have to factor in global risks as well. In nature there is an ‘island effect’. The number of species on an island drops significantly when you go to the mainland. Similarly, when you open up your small economies, you lose some of your ethnicity or diversity. Artisans are being killed by globalisation. Think of the effect on so many artists who have been put out of work while people are buying wrinkle-free shirts and cheap mobile phones. I’m a localist. The problem is globalisation comes through large global corporates that are predatory, and so we want to counter its ill-effects.

Where do you see the world moving now? Further right, or will it revert to the centre?

I don’t think it will go left or right, and I don’t know about the short term. But I think in the long term, the world can only survive if it lives like nature does. Many smaller units of governance, and a collection of super islands with some separation, quick decision-making, and visible implementation. Lots of Switzerlands, that’s what we need. What we need is not leaders, we don’t need them. We just need someone at the top who doesn’t mess the system up.

Fateh ka Fatwa - Episode 4

On Triple Talaq

Saturday, 21 January 2017

Art of the one-liner: wit and grit for a deadly hit

Gary Nunn in The Guardian

For a while, I had the perfect tagline on my dating profile, and it was all thanks to the wit of Carrie Fisher: “Instant gratification takes too long.”
It alerted me to the dimmer bulbs in the chandelier. “Your tagline makes no sense. What could be quicker than instant?” Blocked! 
But: “Great Postcards from the Edge quote there” = date request! 
Alas, that led to little success so I’m again taking Fisher’s advice, as echoed by Meryl Streep this month: “Take your broken heart, make it into art.” The art I’ve decided to make is to discover the world’s best one-liner. This one’s for you, Carrie:
Some one-liners are so great, they’ve become their own cliches. Some characters deserve their own category for speaking almost exclusively in them - take a bow, Oscar Wilde, Dorothy Parker, Groucho Marx, Mae West, Mark Twain, Maya Angelou.
What makes a great one-liner? Certainly not an inspirational quote in an infuriatingly pretty font over an evocative filtered landscape. They are more likely to be bawdy, rambunctious and not always kind. The edge makes them memorable, although that’s not to say they can’t be profound.
Straight-up humour isn’t enough: the funniest one-liners have a sardonic, sarcastic or even bitchy undertone. The dreamier ones need a tinge of sadness or bitterness. Those offering guidance need to insinuate it’s advice the author of the phrase wistfully - or bitterly - wishes they’d taken themselves. Concision is essential. 
Some wordplay will make the ‘inspirational’ one liner forgivable for its linguistic merit. Don’t state the bleeding obvious: tell us something counterintuitive, or something that reveals the grit of your struggle and how you’ve mastered words as your response.
Retorts are out; if you need someone to rack up a line for you to knock down, then strictly speaking that isn’t a one-liner. It should include all its wit, grit and tips in that standalone line. Metaphors, self-deprecation and genuine poignancy are in.
With those criteria in mind, here’s my - unapologetically subjective - stab at the shortlist of the world’s greatest one-liners of all time:
“It costs a lot of money to look this cheap” - Dolly Parton
“I used to be Snow White, but I drifted” - Mae West
“My face looks like a wedding cake left out in the rain” - WH Auden
“I can resist everything, except temptation” - Oscar Wilde
“Better to be looked over than overlooked” - Mae West
“There are two kinds of people in the world: Those who believe there are two kinds of people in the world and those who don’t” - Robert Benchley
“If you can’t explain it to a six-year-old, you don’t understand it yourself” – Albert Einstein
“Faith: not wanting to know what is true” - Friedrich Nietzsche
“Anything that is too stupid to be spoken is sung” - Voltaire
“Copy from one, it’s plagiarism; copy from two, it’s research” - Wilson Mizner
“Moral indignation is jealousy with a halo” - HG Wells
“I’ve had a perfectly wonderful evening, but this wasn’t it” - Groucho Marx
“Every love’s the love before in a duller dress” - Dorothy Parker
“War is God’s way of teaching Americans geography” - Ambrose Bierce
“If you haven’t got anything nice to say about anybody, come sit next to me” - Alice Roosevelt Longworth (and Shirley MacLaine in Steel Magnolias, of course)
“Happiness makes up in height what it lacks in length” - Robert Frost
“If you think education is expensive, try ignorance” - Derek Bok
Just brilliant
“You can lead a horticulture, but you can’t make her think” - Dorothy Parker
“There is only one thing in the world worse than being talked about, and that is not being talked about” - Oscar Wilde
“When someone shows you who they are believe them; the first time” - Maya Angelou
“When you’re going through hell, keep going” - Winston Churchill
“Before you leave the house, look in the mirror and take one thing off” - Coco Chanel
“Sometimes the questions are complicated and the answers are simple” - Dr Seuss
“At 18 our convictions are hills from which we look; at 45 they are caves in which we hide” - F Scott Fitzgerald
“We are all in the gutter, but some of us are looking at the stars” - Oscar Wilde
“It is never too late to be what you might have been” - George Eliot
“No one can make you feel inferior without your consent” – Eleanor Roosevelt
“If everything is under control, you are going too slow” - Mario Andretti
I can relate
“Being a writer is like having homework every night for the rest of your life” - Lawrence Kasdan
I’d best wrap this up because:
“He that uses many words for explaining any subject, doth, like the cuttlefish, hide himself for the most part in his own ink” - John Ray
And finally, take all these with a pinch of salt because:
“The aphorism is a personal observation inflated into a universal truth, a private posing as a general” - Stefan Kanfer
What do you think is the world’s best one-liner?

Might cricket ban close-in fielders some day?

Michael Jeh in Cricinfo

Following Matt Renshaw's concussion injury, a respected cricket writer posed the question: will we ever get to the point where short leg, bat pad and silly mid-off are banned in international cricket?

In junior cricket in Australia that is already the case. I'm not sure if it is the same in places like India, where the art of spin bowling (and batting against it) will be poorer for such rules. More realistically, given the litigious climate we inhabit, can a fielder refuse the captain's instructions to field in a position that compromises his or her safety? Especially in professional cricket, where livelihoods are at stake, what are the health and safety implications of deliberately putting an employee in a dangerous position, knowing full well that serious injury is a possible outcome?

Barely 40 years on from when Tony Greig wore a motorcycle helmet while batting, it is almost as rare to now see a first-class cricketer batting in a hat or cap.

I have seen the helmet policy change radically - from wearing one being optional, to having to sign an indemnity form if you didn't wear one, to it now being a case of "no helmet, no play" at my local cricket club. This transformation has taken place in the time it has taken my son to progress from Under-8s to U-13s, accelerated no doubt by the Phillip Hughes accident (even though Hughes was wearing a helmet at the time).

In first-class cricket, the rules are so ridiculous that you are allowed to bat in a cap, but if you wear a helmet, it has to meet certain design specs.

Can a fielder refuse the captain's instructions to field in a position that compromises his or her safety?

I remain convinced that this blind faith in helmets is breeding a generation of cricketers who are sometimes technically inept, attempting to pull off the front foot instead of getting inside the line of the ball, or trying to play shots when ducking may have been wiser. In the last two weeks, at least four international batsmen have been hit in the head in Australia and New Zealand. Musfiqur Rahim was the most serious of these cases.

Even bizarre accidents can sometimes be predictable. Umpiring in an U-9 game recently, I refused to allow a batsman to face up because he was not wearing gloves. The opposition coach (also a parent) took exception to my decision, arguing that his son was prepared to take the risk. My counter-argument was that I was not prepared to put my fielders at risk if the bat flew from his hand on a hot, sweaty Brisbane morning. The acid burn of the wrath I incurred hurts less today as I view the replay of Peter Nevill's injury in the Big Bash.

A few years ago Queensland Cricket, in a noble but futile attempt to improve the "spirit of cricket" on the grade-cricket scene, ran a workshop where every captain of every club in every grade was forced to attend an event that tried to encourage a less abrasive, more sportsmanlike atmosphere. If a captain did not attend this workshop, his team lost points if he subsequently captained that season.

On the night in question, when each group was given a different hypothetical situation to mull over (for example: what do you do if an overnight not-out batsman turned up ten minutes late the next day because he was tending to his sick child?) I raised the issue of bowlers and fielders making threats against the batsman along the lines of "I'm going to f***ing kill you." My point was that even if it was not meant literally but taken to signal they were going to bowl aggressively at the batsman's body, once those words were said, if the batsman was actually killed (or badly injured), would there be a case to answer for premeditated assault occasioning bodily harm or worse? Would witnesses (fielders, umpires, non-striker) be asked to testify, under oath, as to whether they actually heard that threat being made, regardless of whether they thought it was meant literally or not?

Gautam Gambhir leaps to avoid getting hit by a shot from Michael Clarke in Delhi, 2008 © AFP

On hearing my question, the first-grade captain of another club stood up in disgust and said that if the evening was going to descend into complete farce with questions of this nature, he was taking leave forthwith. And that was the general consensus in the room: ridiculous question, it will never happen, can we move on to more realistic scenarios please? The hypothetical question I posed was never addressed. Many in the room thought I had pushed credibility too far.

Sadly, vindication is often a dish served cold, but it has a sour aftertaste. It wasn't long before we had the coronial inquest into the death of Hughes, and many of those same questions were posed by the coroner, Michael Barnes. We never quite got to the bottom of the matter, but the coroner was sufficiently unconvinced to note: "The repeated denials of any sledging having occurred in the game in which Phillip Hughes was injured were difficult to accept. Hopefully the focus on this unsavoury aspect of the incident may cause those who claim to love the game to reflect upon whether the practice of sledging is worthy of its participants. An outsider is left to wonder why such a beautiful game would need such an ugly underside."

So what's next? Players (employees) taking legal action against selectors for unfair recruitment policies? Suing your cricket board for making you play while injured? Been there, done that, thanks Nathan Bracken!

Can a batsman who has scored more runs in first-class cricket (Callum Ferguson, for example) make a case for unfair dismissal or discrimination if they jettison him after just one Test for an X-factor cricketer (Nic Maddinson) whose numbers don't quite match up and who is given three Tests? Ridiculous? Yes. Possible? Yes.

No bat pad? No leg slip? It might be a bridge too far. It will change the face of cricket forever, of course. But it won't be the first time that an outlandish suggestion morphs into an everyday reality.

Thursday, 19 January 2017

If you were an elephant

Charles Foster in The Guardian

If you were an elephant living wild in a western city, you’d be confused and disgusted.

You’d have one two-fingered hand swinging from your face – a hand as sensitive as tumescent genitals, but which could smash a wall or pick a cherry. With that hand you’d explore your best friends’ mouths, just for the sake of friendship. With that hand you’d smell water miles away and the flowers at your feet. You’d sift it all, triaging. Category 1: immediate danger. Category 2: potential threat. Category 3: food and water. Category 4: weather forecasts – short and long range. Category 5: pleasure.

Grumbles from trucks and cabs would shudder through the toxic ground, tickle the lamellar corpuscles in your feet and ricochet up your bones. You’d hear with your feet, and your femurs would be microphones. As you walked 10 miles for your breakfast you’d chatter with your friends in 10 octaves. A nearby human would throb like a bodhran as subsonic waves bounced around her chest.

Even if it swayed with grass instead of being covered in concrete and dog shit, the city would be far, far too small for you. You’d feel the ring roads like a corset. You’d smell succulent fields outside, and be wistful. But you’d make the most of what you had. You’d follow a labyrinth of old roads, relying on the wisdom of long-dead elephants, now passed down to your matriarch. You’d have the happiest kind of political system, run by wise old women, appointed for their knowledge of the world and their judgment, uninterested in hierarchy for hierarchy’s sake, and seeking the greatest good for the greatest number.

No room here for the infantile phallocentric Nietzscheanism that is destroying modern human culture. If you were a boy you’d be on the margins, drifting between family groups (but never allowed to disrupt them) or shacked up with your bachelor pals in the elephant equivalent of an unswept bedsit (though usually your behaviour would be gentler, more convivial and more urbane than cohabiting human males). Your function would be to inseminate, and that’s all. Government would be the business of the females.

‘You’d hear with your feet, and your femurs would be microphones.’ Photograph: Bruno Guerreiro/Getty Images/EyeEm

You’d be a communitarian. Relationality would be everything. It’s not that you couldn’t survive alone, although there would certainly be a survival benefit from being a member of a community, just as humans live longer if they are plugged into a church, a mosque or a bowling club. Yes, at some level your altruism might be reciprocal altruism, where you scratch my back if I scratch yours, or kin selection, where you are somehow persuaded to sacrifice yourself if your death or disadvantage will preserve a gene in a sufficiently closely related gene-bearer. But at a much more obvious and important level you’d be relational – joyously shouldering the duties that come with community – because it made you happy. Why do elephants seek out other elephants? Not primarily for sex, or for an extra arsenal of receptors to pick up the scent of poachers, or because they assume that the others will have found particularly nutritious food, but because they like other elephants.

This should be terribly unsurprising. Yet many humans will be surprised. That shows how fully we’ve fallen for the anthropocentric lie that only humans have minds and real emotions. The lie is the high-water mark of scientific fundamentalism. Fortunately it’s going out of fashion now, but for years it paralysed the study of animal behaviour.

As an elephant, you’d have a mind. You would, no doubt at all, be conscious. All the evidence agrees. None – absolutely none – disagrees. You’d have a sense of yourself as distinct from other things. When you looked out contemptuously at humans, wondering why they ate obviously contaminated food, opted to be miserable and alone, or wasted energy on pointless aggression and anxiety, it would be your contempt, as opposed to generic elephantine contempt, or reflexive contempt that bypassed your cerebral cortex, or the contempt of your sister. It would be you looking out, and you’d know it was you.

‘You’d have a mind. You would, no doubt at all, be conscious.’ Photograph: Palani Mohan/Getty Images

The American ecologist Carl Safina argues that elephant X can understand the relationship that elephant Y has with elephant Z – whether it is a kin relationship or simple friendship. Just think about that. Think about what it entails for X’s knowledge of itself; for X’s ability to think itself into the head of another, and for the way that X must articulate to itself the concept of a third-party relationship. Perhaps elephants are explaining the world to themselves by formulating, evaluating and selecting propositions – a faculty we tend to think of as uniquely ours.

That will be too much for most. Indeed, it’s a mistake to assume that in order to have a mind one has to have a mind that is like human minds. So let’s just say that, according to the evidence, it’s not obviously ridiculous to invite you, the human, to imagine yourself as an elephant. There’s some biological justification for what sounds like a whimsical, sentimental literary device. You and the elephant both have minds, wrought from the same stuff. And your minds engage with the world using the same devices. Your neurological hardware differs only in sensitivity: sodium and potassium surge in the same way through the same molecular gates when you and the elephant step on a nail; the same ancient hormones mediate pleasure, anger and stress. “If you prick us,” ask the elephants (using a chromatic orchestra of sounds, and well over 100 distinct body movements), “do we not bleed?” Indeed they do.

We can be cautiously Beatrix-Pottery with elephants. When the temporal glands near their eyes stream in circumstances that, for us, would be emotional, they’re crying. When a bereaved elephant mother carries her dead baby round on her tusks, or trails miserably behind the herd for weeks, her head hanging down, she’s grieving. When other elephants sit for hours around the body of a dead elephant, they’re mourning. When they cover an elephant corpse with soil or vegetation, or move elephant bones, they’re being reverential. When they cover a dead human, or build a protective wall of sticks around a wounded human, they’re showing an empathic acknowledgment of our shared destiny that we’d do well to learn. These, dear reductionists, are, as you would put it, the most parsimonious hypotheses.

 ‘You’d smell water miles away and the flowers at your feet.’ Photograph: Simon Eeman/Alamy Stock Photo

If elephants have minds, and minds (as seems likely) can extend beyond the brains in which we conventionally assume they’re situated, we’d expect them to tune into distant elephants, and perhaps into the minds of other species too. There are some tantalising hints that they can. Safina was told by a keeper at a Kenyan elephant sanctuary that the resident elephants knew, from distances well beyond the reach of ordinary senses, that other elephants were on the way – just as Kalahari bushmen know, from 50 miles away, just what a hunting party has killed, and when it will return. When the elephant whisperer Lawrence Anthony died, two groups of elephants that he’d rescued came to his house on two consecutive days. They hadn’t visited for a year.

Perhaps one of the reasons we’re so keen to deny non-human creatures minds, consciousness and personhood is that, if they’re people, they’re embarrassingly better people than we are. They build better communities; they live at peace with themselves and aren’t, unlike us, actively psychopathic towards other species. They know, and take account of, a great deal more information about the natural world than we do.

Back to the shamanic fantasy: you’re a city elephant. You’ll inhabit the city much more intensely and satisfactorily than most of its human denizens. All your senses will be turned fully on. You won’t, like most woefully unsensual humans, use only your eyes, and then translate the visual images into self-referential abstractions with only a slight and dysfunctional relationship to the real world. You’ll be much more properly local than any cockney, New Yorker or MadrileƱo, though you call Africa your home. You’ll know far more of the city than any geographer, historian, zoologist, botanist, policeman or lover. By trying to become an elephant, you might become a much more thriving human.

Be careful, though. You’re likely to end up dead because someone wants a couple of your teeth.

How statistics lost their power

William Davies in The Guardian

In theory, statistics should help settle arguments. They ought to provide stable reference points that everyone – no matter what their politics – can agree on. Yet in recent years, divergent levels of trust in statistics has become one of the key schisms that have opened up in western liberal democracies. Shortly before the November presidential election, a study in the US discovered that 68% of Trump supporters distrusted the economic data published by the federal government. In the UK, a research project by Cambridge University and YouGov looking at conspiracy theories discovered that 55% of the population believes that the government “is hiding the truth about the number of immigrants living here”.

Rather than diffusing controversy and polarisation, it seems as if statistics are actually stoking them. Antipathy to statistics has become one of the hallmarks of the populist right, with statisticians and economists chief among the various “experts” that were ostensibly rejected by voters in 2016. Not only are statistics viewed by many as untrustworthy, there appears to be something almost insulting or arrogant about them. Reducing social and economic issues to numerical aggregates and averages seems to violate some people’s sense of political decency.

Nowhere is this more vividly manifest than with immigration. The thinktank British Future has studied how best to win arguments in favour of immigration and multiculturalism. One of its main findings is that people often respond warmly to qualitative evidence, such as the stories of individual migrants and photographs of diverse communities. But statistics – especially regarding alleged benefits of migration to Britain’s economy – elicit quite the opposite reaction. People assume that the numbers are manipulated and dislike the elitism of resorting to quantitative evidence. Presented with official estimates of how many immigrants are in the country illegally, a common response is to scoff. Far from increasing support for immigration, British Future found, pointing to its positive effect on GDP can actually make people more hostile to it. GDP itself has come to seem like a Trojan horse for an elitist liberal agenda. Sensing this, politicians have now largely abandoned discussing immigration in economic terms.

All of this presents a serious challenge for liberal democracy. Put bluntly, the British government – its officials, experts, advisers and many of its politicians – does believe that immigration is on balance good for the economy. The British government did believe that Brexit was the wrong choice. The problem is that the government is now engaged in self-censorship, for fear of provoking people further.

This is an unwelcome dilemma. Either the state continues to make claims that it believes to be valid and is accused by sceptics of propaganda, or else, politicians and officials are confined to saying what feels plausible and intuitively true, but may ultimately be inaccurate. Either way, politics becomes mired in accusations of lies and cover-ups.

The declining authority of statistics – and the experts who analyse them – is at the heart of the crisis that has become known as “post-truth” politics. And in this uncertain new world, attitudes towards quantitative expertise have become increasingly divided. From one perspective, grounding politics in statistics is elitist, undemocratic and oblivious to people’s emotional investments in their community and nation. It is just one more way that privileged people in London, Washington DC or Brussels seek to impose their worldview on everybody else. From the opposite perspective, statistics are quite the opposite of elitist. They enable journalists, citizens and politicians to discuss society as a whole, not on the basis of anecdote, sentiment or prejudice, but in ways that can be validated. The alternative to quantitative expertise is less likely to be democracy than an unleashing of tabloid editors and demagogues to provide their own “truth” of what is going on across society.

Is there a way out of this polarisation? Must we simply choose between a politics of facts and one of emotions, or is there another way of looking at this situation? One way is to view statistics through the lens of their history. We need to try and see them for what they are: neither unquestionable truths nor elite conspiracies, but rather as tools designed to simplify the job of government, for better or worse. Viewed historically, we can see what a crucial role statistics have played in our understanding of nation states and their progress. This raises the alarming question of how – if at all – we will continue to have common ideas of society and collective progress, should statistics fall by the wayside.

In the second half of the 17th century, in the aftermath of prolonged and bloody conflicts, European rulers adopted an entirely new perspective on the task of government, focused upon demographic trends – an approach made possible by the birth of modern statistics. Since ancient times, censuses had been used to track population size, but these were costly and laborious to carry out and focused on citizens who were considered politically important (property-owning men), rather than society as a whole. Statistics offered something quite different, transforming the nature of politics in the process.

Statistics were designed to give an understanding of a population in its entirety,rather than simply to pinpoint strategically valuable sources of power and wealth. In the early days, this didn’t always involve producing numbers. In Germany, for example (from where we get the term Statistik) the challenge was to map disparate customs, institutions and laws across an empire of hundreds of micro-states. What characterised this knowledge as statistical was its holistic nature: it aimed to produce a picture of the nation as a whole. Statistics would do for populations what cartography did for territory.

Equally significant was the inspiration of the natural sciences. Thanks to standardised measures and mathematical techniques, statistical knowledge could be presented as objective, in much the same way as astronomy. Pioneering English demographers such as William Petty and John Graunt adapted mathematical techniques to estimate population changes, for which they were hired by Oliver Cromwell and Charles II.

The emergence in the late 17th century of government advisers claiming scientific authority, rather than political or military acumen, represents the origins of the “expert” culture now so reviled by populists. These path-breaking individuals were neither pure scholars nor government officials, but hovered somewhere between the two. They were enthusiastic amateurs who offered a new way of thinking about populations that privileged aggregates and objective facts. Thanks to their mathematical prowess, they believed they could calculate what would otherwise require a vast census to discover.

There was initially only one client for this type of expertise, and the clue is in the word “statistics”. Only centralised nation states had the capacity to collect data across large populations in a standardised fashion and only states had any need for such data in the first place. Over the second half of the 18th century, European states began to collect more statistics of the sort that would appear familiar to us today. Casting an eye over national populations, states became focused upon a range of quantities: births, deaths, baptisms, marriages, harvests, imports, exports, price fluctuations. Things that would previously have been registered locally and variously at parish level became aggregated at a national level.

New techniques were developed to represent these indicators, which exploited both the vertical and horizontal dimensions of the page, laying out data in matrices and tables, just as merchants had done with the development of standardised book-keeping techniques in the late 15th century. Organising numbers into rows and columns offered a powerful new way of displaying the attributes of a given society. Large, complex issues could now be surveyed simply by scanning the data laid out geometrically across a single page.

These innovations carried extraordinary potential for governments. By simplifying diverse populations down to specific indicators, and displaying them in suitable tables, governments could circumvent the need to acquire broader detailed local and historical insight. Of course, viewed from a different perspective, this blindness to local cultural variability is precisely what makes statistics vulgar and potentially offensive. Regardless of whether a given nation had any common cultural identity, statisticians would assume some standard uniformity or, some might argue, impose that uniformity upon it.

Not every aspect of a given population can be captured by statistics. There is always an implicit choice in what is included and what is excluded, and this choice can become a political issue in its own right. The fact that GDP only captures the value of paid work, thereby excluding the work traditionally done by women in the domestic sphere, has made it a target of feminist critique since the 1960s. In France, it has been illegal to collect census data on ethnicity since 1978, on the basis that such data could be used for racist political purposes. (This has the side-effect of making systemic racism in the labour market much harder to quantify.)

Despite these criticisms, the aspiration to depict a society in its entirety, and to do so in an objective fashion, has meant that various progressive ideals have been attached to statistics. The image of statistics as a dispassionate science of society is only one part of the story. The other part is about how powerful political ideals became invested in these techniques: ideals of “evidence-based policy”, rationality, progress and nationhood grounded in facts, rather than in romanticised stories.

Since the high-point of the Enlightenment in the late 18th century, liberals and republicans have invested great hope that national measurement frameworks could produce a more rational politics, organised around demonstrable improvements in social and economic life. The great theorist of nationalism, Benedict Anderson, famously described nations as “imagined communities”, but statistics offer the promise of anchoring this imagination in something tangible. Equally, they promise to reveal what historical path the nation is on: what kind of progress is occurring? How rapidly? For Enlightenment liberals, who saw nations as moving in a single historical direction, this question was crucial.

The potential of statistics to reveal the state of the nation was seized in post-revolutionary France. The Jacobin state set about imposing a whole new framework of national measurement and national data collection. The world’s first official bureau of statistics was opened in Paris in 1800. Uniformity of data collection, overseen by a centralised cadre of highly educated experts, was an integral part of the ideal of a centrally governed republic, which sought to establish a unified, egalitarian society.

From the Enlightenment onwards, statistics played an increasingly important role in the public sphere, informing debate in the media, providing social movements with evidence they could use. Over time, the production and analysis of such data became less dominated by the state. Academic social scientists began to analyse data for their own purposes, often entirely unconnected to government policy goals. By the late 19th century, reformers such as Charles Booth in London and WEB Du Bois in Philadelphia were conducting their own surveys to understand urban poverty.

Illustration by Guardian Design

To recognise how statistics have been entangled in notions of national progress, consider the case of GDP. GDP is an estimate of the sum total of a nation’s consumer spending, government spending, investments and trade balance (exports minus imports), which is represented in a single number. This is fiendishly difficult to get right, and efforts to calculate this figure began, like so many mathematical techniques, as a matter of marginal, somewhat nerdish interest during the 1930s. It was only elevated to a matter of national political urgency by the second world war, when governments needed to know whether the national population was producing enough to keep up the war effort. In the decades that followed, this single indicator, though never without its critics, took on a hallowed political status, as the ultimate barometer of a government’s competence. Whether GDP is rising or falling is now virtually a proxy for whether society is moving forwards or backwards.

Or take the example of opinion polling, an early instance of statistical innovation occurring in the private sector. During the 1920s, statisticians developed methods for identifying a representative sample of survey respondents, so as to glean the attitudes of the public as a whole. This breakthrough, which was first seized upon by market researchers, soon led to the birth of the opinion polling. This new industry immediately became the object of public and political fascination, as the media reported on what this new science told us about what “women” or “Americans” or “manual labourers” thought about the world.

Nowadays, the flaws of polling are endlessly picked apart. But this is partly due to the tremendous hopes that have been invested in polling since its origins. It is only to the extent that we believe in mass democracy that we are so fascinated or concerned by what the public thinks. But for the most part it is thanks to statistics, and not to democratic institutions as such, that we can know what the public thinks about specific issues. We underestimate how much of our sense of “the public interest” is rooted in expert calculation, as opposed to democratic institutions.

As indicators of health, prosperity, equality, opinion and quality of life have come to tell us who we are collectively and whether things are getting better or worse, politicians have leaned heavily on statistics to buttress their authority. Often, they lean too heavily, stretching evidence too far, interpreting data too loosely, to serve their cause. But that is an inevitable hazard of the prevalence of numbers in public life, and need not necessarily trigger the type of wholehearted rejections of expertise that we have witnessed recently.

In many ways, the contemporary populist attack on “experts” is born out of the same resentment as the attack on elected representatives. In talking of society as a whole, in seeking to govern the economy as a whole, both politicians and technocrats are believed to have “lost touch” with how it feels to be a single citizen in particular. Both statisticians and politicians have fallen into the trap of “seeing like a state”, to use a phrase from the anarchist political thinker James C Scott. Speaking scientifically about the nation – for instance in terms of macroeconomics – is an insult to those who would prefer to rely on memory and narrative for their sense of nationhood, and are sick of being told that their “imagined community” does not exist.

On the other hand, statistics (together with elected representatives) performed an adequate job of supporting a credible public discourse for decades if not centuries. What changed?

The crisis of statistics is not quite as sudden as it might seem. For roughly 450 years, the great achievement of statisticians has been to reduce the complexity and fluidity of national populations into manageable, comprehensible facts and figures. Yet in recent decades, the world has changed dramatically, thanks to the cultural politics that emerged in the 1960s and the reshaping of the global economy that began soon after. It is not clear that the statisticians have always kept pace with these changes. Traditional forms of statistical classification and definition are coming under strain from more fluid identities, attitudes and economic pathways. Efforts to represent demographic, social and economic changes in terms of simple, well-recognised indicators are losing legitimacy.

Consider the changing political and economic geography of nation states over the past 40 years. The statistics that dominate political debate are largely national in character: poverty levels, unemployment, GDP, net migration. But the geography of capitalism has been pulling in somewhat different directions. Plainly globalisation has not rendered geography irrelevant. In many cases it has made the location of economic activity far more important, exacerbating the inequality between successful locations (such as London or San Francisco) and less successful locations (such as north-east England or the US rust belt). The key geographic units involved are no longer nation states. Rather, it is cities, regions or individual urban neighbourhoods that are rising and falling.

The Enlightenment ideal of the nation as a single community, bound together by a common measurement framework, is harder and harder to sustain. If you live in one of the towns in the Welsh valleys that was once dependent on steel manufacturing or mining for jobs, politicians talking of how “the economy” is “doing well” are likely to breed additional resentment. From that standpoint, the term “GDP” fails to capture anything meaningful or credible.

When macroeconomics is used to make a political argument, this implies that the losses in one part of the country are offset by gains somewhere else. Headline-grabbing national indicators, such as GDP and inflation, conceal all sorts of localised gains and losses that are less commonly discussed by national politicians. Immigration may be good for the economy overall, but this does not mean that there are no local costs at all. So when politicians use national indicators to make their case, they implicitly assume some spirit of patriotic mutual sacrifice on the part of voters: you might be the loser on this occasion, but next time you might be the beneficiary. But what if the tables are never turned? What if the same city or region wins over and over again, while others always lose? On what principle of give and take is that justified?

In Europe, the currency union has exacerbated this problem. The indicators that matter to the European Central Bank (ECB), for example, are those representing half a billion people. The ECB is concerned with the inflation or unemployment rate across the eurozone as if it were a single homogeneous territory, at the same time as the economic fate of European citizens is splintering in different directions, depending on which region, city or neighbourhood they happen to live in. Official knowledge becomes ever more abstracted from lived experience, until that knowledge simply ceases to be relevant or credible.

The privileging of the nation as the natural scale of analysis is one of the inbuilt biases of statistics that years of economic change has eaten away at. Another inbuilt bias that is coming under increasing strain is classification. Part of the job of statisticians is to classify people by putting them into a range of boxes that the statistician has created: employed or unemployed, married or unmarried, pro-Europe or anti-Europe. So long as people can be placed into categories in this way, it becomes possible to discern how far a given classification extends across the population.

This can involve somewhat reductive choices. To count as unemployed, for example, a person has to report to a survey that they are involuntarily out of work, even if it may be more complicated than that in reality. Many people move in and out of work all the time, for reasons that might have as much to do with health and family needs as labour market conditions. But thanks to this simplification, it becomes possible to identify the rate of unemployment across the population as a whole.

Here’s a problem, though. What if many of the defining questions of our age are not answerable in terms of the extent of people encompassed, but the intensity with which people are affected? Unemployment is one example. The fact that Britain got through the Great Recession of 2008-13 without unemployment rising substantially is generally viewed as a positive achievement. But the focus on “unemployment” masked the rise of underemployment, that is, people not getting a sufficient amount of work or being employed at a level below that which they are qualified for. This currently accounts for around 6% of the “employed” labour force. Then there is the rise of the self-employed workforce, where the divide between “employed” and “involuntarily unemployed” makes little sense.

This is not a criticism of bodies such as the Office for National Statistics (ONS), which does now produce data on underemployment. But so long as politicians continue to deflect criticism by pointing to the unemployment rate, the experiences of those struggling to get enough work or to live on their wages go unrepresented in public debate. It wouldn’t be all that surprising if these same people became suspicious of policy experts and the use of statistics in political debate, given the mismatch between what politicians say about the labour market and the lived reality.

The rise of identity politics since the 1960s has put additional strain on such systems of classification. Statistical data is only credible if people will accept the limited range of demographic categories that are on offer, which are selected by the expert not the respondent. But where identity becomes a political issue, people demand to define themselves on their own terms, where gender, sexuality, race or class is concerned.

Opinion polling may be suffering for similar reasons. Polls have traditionally captured people’s attitudes and preferences, on the reasonable assumption that people will behave accordingly. But in an age of declining political participation, it is not enough simply to know which box someone would prefer to put an “X” in. One also needs to know whether they feel strongly enough about doing so to bother. And when it comes to capturing such fluctuations in emotional intensity, polling is a clumsy tool.

Statistics have faced criticism regularly over their long history. The challenges that identity politics and globalisation present to them are not new either. Why then do the events of the past year feel quite so damaging to the ideal of quantitative expertise and its role in political debate?

In recent years, a new way of quantifying and visualising populations has emerged that potentially pushes statistics to the margins, ushering in a different era altogether. Statistics, collected and compiled by technical experts, are giving way to data that accumulates by default, as a consequence of sweeping digitisation. Traditionally, statisticians have known which questions they wanted to ask regarding which population, then set out to answer them. By contrast, data is automatically produced whenever we swipe a loyalty card, comment on Facebook or search for something on Google. As our cities, cars, homes and household objects become digitally connected, the amount of data we leave in our trail will grow even greater. In this new world, data is captured first and research questions come later.

In the long term, the implications of this will probably be as profound as the invention of statistics was in the late 17th century. The rise of “big data” provides far greater opportunities for quantitative analysis than any amount of polling or statistical modelling. But it is not just the quantity of data that is different. It represents an entirely different type of knowledge, accompanied by a new mode of expertise.

First, there is no fixed scale of analysis (such as the nation) nor any settled categories (such as “unemployed”). These vast new data sets can be mined in search of patterns, trends, correlations and emergent moods. It becomes a way of tracking the identities that people bestow upon themselves (such as “#ImwithCorbyn” or “entrepreneur”) rather than imposing classifications upon them. This is a form of aggregation suitable to a more fluid political age, in which not everything can be reliably referred back to some Enlightenment ideal of the nation state as guardian of the public interest.

Second, the majority of us are entirely oblivious to what all this data says about us, either individually or collectively. There is no equivalent of an Office for National Statistics for commercially collected big data. We live in an age in which our feelings, identities and affiliations can be tracked and analysed with unprecedented speed and sensitivity – but there is nothing that anchors this new capacity in the public interest or public debate. There are data analysts who work for Google and Facebook, but they are not “experts” of the sort who generate statistics and who are now so widely condemned. The anonymity and secrecy of the new analysts potentially makes them far more politically powerful than any social scientist.

A company such as Facebook has the capacity to carry quantitative social science on hundreds of billions of people, at very low cost. But it has very little incentive to reveal the results. In 2014, when Facebook researchers published results of a study of “emotional contagion” that they had carried out on their users – in which they altered news feeds to see how it affected the content that users then shared in response – there was an outcry that people were being unwittingly experimented on. So, from Facebook’s point of view, why go to all the hassle of publishing? Why not just do the study and keep quiet?

What is most politically significant about this shift from a logic of statistics to one of data is how comfortably it sits with the rise of populism. Populist leaders can heap scorn upon traditional experts, such as economists and pollsters, while trusting in a different form of numerical analysis altogether. Such politicians rely on a new, less visible elite, who seek out patterns from vast data banks, but rarely make any public pronouncements, let alone publish any evidence. These data analysts are often physicists or mathematicians, whose skills are not developed for the study of society at all. This, for example, is the worldview propagated by Dominic Cummings, former adviser to Michael Gove and campaign director of Vote Leave. “Physics, mathematics and computer science are domains in which there are real experts, unlike macro-economic forecasting,” Cummings has argued.

Figures close to Donald Trump, such as his chief strategist Steve Bannon and the Silicon Valley billionaire Peter Thiel, are closely acquainted with cutting-edge data analytics techniques, via companies such as Cambridge Analytica, on whose board Bannon sits. During the presidential election campaign, Cambridge Analytica drew on various data sources to develop psychological profiles of millions of Americans, which it then used to help Trump target voters with tailored messaging.

This ability to develop and refine psychological insights across large populations is one of the most innovative and controversial features of the new data analysis. As techniques of “sentiment analysis”, which detect the mood of large numbers of people by tracking indicators such as word usage on social media, become incorporated into political campaigns, the emotional allure of figures such as Trump will become amenable to scientific scrutiny. In a world where the political feelings of the general public are becoming this traceable, who needs pollsters?

Few social findings arising from this kind of data analytics ever end up in the public domain. This means that it does very little to help anchor political narrative in any shared reality. With the authority of statistics waning, and nothing stepping into the public sphere to replace it, people can live in whatever imagined community they feel most aligned to and willing to believe in. Where statistics can be used to correct faulty claims about the economy or society or population, in an age of data analytics there are few mechanisms to prevent people from giving way to their instinctive reactions or emotional prejudices. On the contrary, companies such as Cambridge Analytica treat those feelings as things to be tracked.

But even if there were an Office for Data Analytics, acting on behalf of the public and government as the ONS does, it is not clear that it would offer the kind of neutral perspective that liberals today are struggling to defend. The new apparatus of number-crunching is well suited to detecting trends, sensing the mood and spotting things as they bubble up. It serves campaign managers and marketers very well. It is less well suited to making the kinds of unambiguous, objective, potentially consensus-forming claims about society that statisticians and economists are paid for.

In this new technical and political climate, it will fall to the new digital elite to identify the facts, projections and truth amid the rushing stream of data that results. Whether indicators such as GDP and unemployment continue to carry political clout remains to be seen, but if they don’t, it won’t necessarily herald the end of experts, less still the end of truth. The question to be taken more seriously, now that numbers are being constantly generated behind our backs and beyond our knowledge, is where the crisis of statistics leaves representative democracy.

On the one hand, it is worth recognising the capacity of long-standing political institutions to fight back. Just as “sharing economy” platforms such as Uber and Airbnb have recently been thwarted by legal rulings (Uber being compelled to recognise drivers as employees, Airbnb being banned altogether by some municipal authorities), privacy and human rights law represents a potential obstacle to the extension of data analytics. What is less clear is how the benefits of digital analytics might ever be offered to the public, in the way that many statistical data sets are. Bodies such as the Open Data Institute, co-founded by Tim Berners-Lee, campaign to make data publicly available, but have little leverage over the corporations where so much of our data now accumulates. Statistics began life as a tool through which the state could view society, but gradually developed into something that academics, civic reformers and businesses had a stake in. But for many data analytics firms, secrecy surrounding methods and sources of data is a competitive advantage that they will not give up voluntarily.

A post-statistical society is a potentially frightening proposition, not because it would lack any forms of truth or expertise altogether, but because it would drastically privatise them. Statistics are one of many pillars of liberalism, indeed of Enlightenment. The experts who produce and use them have become painted as arrogant and oblivious to the emotional and local dimensions of politics. No doubt there are ways in which data collection could be adapted to reflect lived experiences better. But the battle that will need to be waged in the long term is not between an elite-led politics of facts versus a populist politics of feeling. It is between those still committed to public knowledge and public argument and those who profit from the ongoing disintegration of those things.

Libor scandal: the bankers who fixed the world’s most important number

Liam Vaughan and Gavin Finch in The Guardian

At the Tokyo headquarters of the Swiss bank UBS, in the middle of a deserted trading floor, Tom Hayes sat rapt before a bank of eight computer screens. Collar askew, pale features pinched, blond hair mussed from a habit of pulling at it when he was deep in thought, the British trader was even more dishevelled than usual. It was 15 September 2008, and it looked, in Hayes’s mind, like the end of the world.

Hayes had been woken up at dawn in his apartment by a call from his boss, telling him to get to the office immediately. In New York, Lehman Brothers was hurtling towards bankruptcy. At his desk, Hayes watched the world processing the news and panicking. As each market opened, it became a sea of flashing red as investors frantically dumped their holdings. In moments like this, Hayes entered an almost unconscious state, rapidly processing the tide of information before him and calculating the best escape route.

Hayes was a phenomenon at UBS, one of the best the bank had at trading derivatives. So far, the mounting financial crisis had actually been good for him. The chaos had let him buy cheaply from those desperate to get out, and sell high to the unlucky few who still needed to trade. While most dealers closed up shop in fear, Hayes, with a seemingly limitless appetite for risk, stayed in. He was 28, and he was up more than $70 million for the year.

Now that was under threat. Not only did Hayes have to extract himself from every deal he had done with Lehman, he had also made a series of enormous bets that in the coming days interest rates would remain stable. The collapse of Lehman Brothers, the fourth-largest investment bank in the US, would surely cause those rates, which were really just barometers of risk, to spike. As Hayes examined his trading book, one rate mattered more than any other: the London interbank offered rate, or Libor, a benchmark that influences $350 trillion of securities and loans around the world. For traders such as Hayes, this number was the Holy Grail. And two years earlier, he had discovered a way to rig it.

Libor was set by a self-selected, self-policing committee of the world’s largest banks. The rate measured how much it cost them to borrow from each other. Every morning, each bank submitted an estimate, an average was taken, and a number was published at midday. The process was repeated in different currencies, and for various amounts of time, ranging from overnight to a year. During his time as a junior trader in London, Hayes had got to know several of the 16 individuals responsible for making their bank’s daily submission for the Japanese yen. His flash of insight was realising that these men mostly relied on inter-dealer brokers, the fast-talking middlemen involved in every trade, for guidance on what to submit each day.

Brokers are the middlemen in the world of finance, facilitating deals between traders at different banks in everything from Treasury bonds to over-the-counter derivatives. If a trader wants to buy or sell, he could theoretically ring all the banks to get a price. Or he could go through a broker who is in touch with everyone and can find a counter-party in seconds. Hardly a dollar changes hands in the cash and derivatives markets without a broker matching the deal and taking his cut. In the opaque, over-the-counter derivatives market, where there is no centralised exchange, brokers are at the epicentre of information flow. That puts them in a powerful position. Only they can get a picture of what all the banks are doing. While brokers had no official role in setting Libor, the rate-setters at the banks relied on them for information on where cash was trading.

Most traders looked down on brokers as second-class citizens, too. Hayes recognised their worth. He saw what no one else did because he was different. His intimacy with numbers, his cold embrace of risk and his unusual habits were more than professional tics. Hayes would not be diagnosed with Asperger’s syndrome until 2015, when he was 35, but his co-workers, many of them savvy operators from fancy schools, often reminded Hayes that he wasn’t like them. They called him “Rain Man”.

By the time the market opened in London, Lehman’s demise was official. Hayes instant-messaged one of his trusted brokers in the City to tell him what direction he wanted Libor to move. Typically, he skipped any pleasantries. “Cash mate, really need it lower,” Hayes typed. “What’s the score?” The broker sent his assurances and, over the next few hours, followed a well-worn routine. Whenever one of the Libor-setting banks called and asked his opinion on what the benchmark would do, the broker said – incredibly, given the calamitous news – that the rate was likely to fall. Libor may have featured in hundreds of trillions of dollars of loans and derivatives, but this was how it was set: conversations among men who were, depending on the day, indifferent, optimistic or frightened. When Hayes checked the official figures later that night, he saw to his relief that yen Libor had fallen.

Hayes was not out of danger yet. Over the next three days, he barely left the office, surviving on three hours of sleep a night. As the market convulsed, his profit and loss jumped around from minus $20 million to plus $8 million in just hours, but Hayes had another ace up his sleeve. ICAP, the world’s biggest inter-dealer broker, sent out a “Libor prediction” email each day at around 7am to the individuals at the banks responsible for submitting Libor. Hayes messaged an insider at ICAP and instructed him to skew the predictions lower. Amid the chaos, Libor was the one thing Hayes believed he had some control over. He cranked his network to the max, offering his brokers extra payments for their cooperation and calling in favours at banks around the world.

By Thursday, 18 September, Hayes was exhausted. This was the moment he had been working towards all week. If Libor jumped today, all his puppeteering would have been for nothing. Libor moves in increments called basis points, equal to one one-hundredth of a percentage point, and every tick was worth roughly $750,000 to his bottom line.

For the umpteenth time since Lehman faltered, Hayes reached out to his brokers in London. “I need you to keep it as low as possible, all right?” he told one of them in a message. “I’ll pay you, you know, $50,000, $100,000, whatever. Whatever you want, all right?”

“All right,” the broker repeated.

“I’m a man of my word,” Hayes said.

“I know you are. No, that’s done, right, leave it to me,” the broker said.

Hayes was still in the Tokyo office at 8pm when that day’s Libors were published. The yen rate had fallen 1 basis point, while comparable money market rates in other currencies continued to soar. Hayes’s crisis had been averted. Using his network of brokers, he had personally sought to tilt part of the planet’s financial infrastructure. He pulled off his headset and headed home to bed. He had only recently upgraded from the superhero duvet he’d slept under since he was eight years old.

Hayes’s job was to make his employer as much money as possible by buying and selling derivatives. How exactly he did that – the special concoction of strategies, skills and tricks that make up a trader’s DNA – was largely left up to him. First and foremost he was a market-maker, providing liquidity to his clients, who were mostly traders at other banks. From the minute he logged on to his Bloomberg terminal each morning and the red light next to his name turned green, Hayes was on the phone quoting guaranteed bid and offer prices on the vast inventory of products he traded. Hayes prided himself on always being open for business no matter how choppy the markets. It was his calling card.

Hayes likened this part of his job to owning a fruit and vegetable stall. Buy low, sell high and pocket the difference. But rather than apples and pears, he dealt in complex financial securities worth hundreds of millions of dollars. His profit came from the spread between how much he paid for a security and how much he sold it for. In volatile times, the spread widened, reflecting the increased risk that the market might move against him before he had the chance to trade out of his position.

All of this offered a steady stream of income, but it wasn’t where the big money came from. The thing that really set Hayes apart was his ability to spot price anomalies and exploit them, a technique known as relative value trading. It appealed to his lifelong passion for seeking out patterns. During quiet spells, he spent his time scouring data, hunting for unseen opportunities. If he thought that the price of two similar securities had diverged unduly, he would buy one and short the other, betting that the spread between the two would shrink.

Everywhere he worked, Hayes set up his software to tell him exactly how much he stood to gain or lose from every fraction of a move in Libor in each currency. One of Hayes’s favourite trades involved betting that the gap between Libor in different durations would widen or narrow: what’s known in the industry as a basis trade. Each time Hayes made a trade, he would have to decide whether to lay off some of his risk by hedging his position using, for example, other derivatives.

Hayes’s dealing created a constantly changing trade book stretching years into the future, which was mapped out on a vast Excel spreadsheet. He liked to think of it as a living organism with thousands of interconnected moving parts. In a corner of one of his screens was a number he looked at more than any other: his rolling profit and loss. Ask any decent trader and he will be able to give it to you to the nearest $1,000. It was Hayes’s self-worth boiled down into a single indisputable number. 
Tom Hayes was a phenomenon at UBS, one of the best the bank had at trading derivatives. Photograph: Bloomberg via Getty Images

By the summer of 2007, the mortgage crisis in the US caused banks and investment funds around the world to become skittish about lending to each other without collateral. Firms that relied on the so-called money markets to fund their businesses were paralysed by the ballooning cost of short-term credit. On 14 September, customers of Northern Rock queued for hours to withdraw their savings after the bank announced it was relying on loans from the Bank of England to stay afloat.

After that, banks were only prepared to make unsecured loans to each other for a few days at a time, and interest rates on longer-term loans rocketed. Libor, as a barometer of stress in the system, reacted accordingly. In August 2007, the spread between three-month dollar Libor and the overnight indexed swap – a measure of banks’ overnight borrowing costs – jumped from 12 basis points to 73 basis points. By December it had soared to 106 basis points. A similar pattern could be seen in sterling, euros and most of the 12 other currencies published on the website of the British Bankers’ Association each day at noon.

Everyone could see that Libor rates had shot up, but questions began to be asked about whether they had climbed enough to reflect the severity of the credit squeeze. By August 2007, there was almost no trading in cash for durations of longer than a month. In some of the smaller currencies there were no lenders for any time frame. Yet, with trillions of dollars tied to Libor, the banks had to keep the trains running. The individuals responsible for submitting Libor rates each day had no choice but to put their thumb and forefinger in the air and pluck out numbers. It was clear that their “best guesses” were unrealistically optimistic.

A game of brinkmanship had developed in which rate-setters tried to predict what their rivals would submit, and then come in slightly lower. If they guessed wrong and input rates higher than their peers, they would receive angry phone calls from their managers telling them to get back into the pack. On trading floors around the world, frantic conversations took place between traders and their brokers about expectations for Libor.

Nobody knew where Libor should be, and nobody wanted to be an outlier. Even where bankers tried to be honest, there was no way of knowing if their estimates were accurate because there was no underlying interbank borrowing on which to compare them. The machine had broken down.

Vince McGonagle, a small and wiry man with a hangdog expression, had been at the enforcement division of the Commodity Futures Trading Commission (CFTC) in Washington for 11 years, during which time his red hair had turned grey around the edges. A practising Catholic, McGonagle got his law degree from Pepperdine University, a Christian school in Malibu, California, where students are prepared for “lives of purpose, service and leadership”.

While his classmates took highly paid positions defending companies and individuals accused of corporate corruption, McGonagle opted to build a career bringing cases against them. He joined the agency as a trial attorney and was now, at 44, a manager overseeing teams of lawyers and investigators.

McGonagle closed the door to his office and settled down to read the daily news. It was 16 April, 2008, and the headline on page one of the Wall Street Journal read: “Bankers Cast Doubt on Key Rate Amid Crisis”. It began: “One of the most important barometers of the world’s financial health could be sending false signals. In a development that has implications for borrowers everywhere, from Russian oil producers to homeowners in Detroit, bankers and traders are expressing concerns that the London interbank offered rate, known as Libor, is becoming unreliable.”

The story, written at the Journal’s London office near Fleet Street, went on to suggest that some of the world’s largest banks might have been providing deliberately low estimates of their borrowing costs to avoid tipping off the market “that they’re desperate for cash”. That was having the effect of distorting Libor, and therefore trillions of dollars of securities around the world.

The journalist’s sources told him that banks were paying much more for cash than they were letting on. They feared if they were honest they could go the same way as Bear Stearns, the 85-year-old New York securities firm that had collapsed the previous month.

The big flaw in Libor was that it relied on banks to tell the truth but encouraged them to lie. When the 150 variants of the benchmark were released each day, the banks’ individual submissions were also published, giving the world a snapshot of their relative creditworthiness. Historically, the individuals responsible for making their firm’s Libor submissions were able to base their estimates on a vibrant interbank money market, in which banks borrowed cash from each other to fund their day-to-day operations. They were prevented from deviating too far from the truth because their fellow market participants knew what rates they were really being charged. Over the previous few months, that had changed. Banks had stopped lending to each other for periods of longer than a few days, preferring to stockpile their cash. After Bear Stearns there was no guarantee they would get it back.

With so much at stake, lenders had become fixated on what their rivals were inputting. Any outlier at the higher – that is, riskier – end was in danger of becoming a pariah, unable to access the liquidity it needed to fund its balance sheet. Soon banks began to submit rates they thought would place them in the middle of the pack rather than what they truly believed they could borrow unsecured cash for. The motivation for low-balling was not tied to profit – many banks actually stood to lose out from lower Libors. This was about survival.

Ironically, just as Libor’s accuracy faltered, its importance rocketed. As the financial crisis deepened, central bankers monitored Libor in different currencies to see how successful their latest policy announcements were in calming markets. Governments looked at individual firms’ submissions for clues as to who they might be forced to bail out next. If banks were lying about Libor, it was not just affecting interest rates and derivatives payments. It was skewing reality.

There was no inkling at this stage that traders such as Hayes were pushing Libor around to boost their profits, but here was a benchmark that relied on the honesty of traders who had a direct interest in where it was set. Libor was overseen by the British Bankers Association (BBA). In both cases, the body responsible for overseeing the rate had no punitive powers, so there was little to discourage firms from cheating.

When McGonagle finished reading the Wall Street Journal article, he emailed colleagues and asked them what they knew about Libor. His team put together a dossier, including some preliminary reports from within the financial community. In March, economists at the Bank for International Settlements, an umbrella group for central banks around the world, had published a paper that identified unusual patterns in Libor during the crisis, although it concluded these were “not caused by shortcomings in the design of the fixing mechanism”.

A month later, Scott Peng, an analyst at Citigroup in New York, sent his customers a research note that estimated the dollar Libor submissions of the 18 firms that set the rate were 20 to 30 basis points lower than they should have been because of a “prevailing fear” among the banks of “being perceived as a weak hand in this fragile market environment”.

While there was no evidence of manipulation by specific firms, McGonagle was coming around to the idea of launching an investigation.

In 2009, Hayes was lured away from UBS to join Citigroup. The head of Citigroup’s team in Asia, the former Lehman banker Chris Cecere, a small, goateed American with a big reputation for finding new ways to make money, had been given millions of dollars to attract the best talent – and Hayes was his round-one pick.

It wasn’t just the $3m signing bonus that had won Hayes over. The promise of a fresh start at one of the world’s biggest banks, with him at centre stage in its aggressive expansion into the Asian interest-rate derivatives market, had proved too tempting to resist. After persuading him to join, Cecere boasted to colleagues that he’d found “a real fucking animal”, who “knows everybody on the street”.

Citigroup in Canary Wharf, London Photograph: DBURKE / Alamy/Alamy

Cecere set in motion plans for Citigroup to join the Tibor (Tokyo interbank offered rate) panel which, Hayes would crow, was even easier to influence than Libor because fewer banks contributed to it. Hayes wanted to hit the ground running when he started trading, and being able to influence the two benchmarks that helped determine the profitability of the bulk of his positions was an important step. Another was bringing Citigroup’s own London-based Libor-setters on board.

On the afternoon of 8 December, Cecere was at his desk on the Tokyo trading floor. He had an office but seldom used it, preferring to be amid the action. He believed that six-month yen Libor was too high. After checking the submissions from the previous day, he was surprised to see that Citigroup had input one of the highest figures.

Cecere contacted the head of the risk treasury team in Tokyo, Stantley Tan, and asked him to find out who the yen-setter was and request that he lower his input by several basis points. It turned out the risk treasury desk in Canary Wharf was responsible for the bank’s Libor submissions.

“I spoke to our point man in London,” Tan wrote back to Cecere that afternoon. “I have asked him to consider moving quotes [lower]”.

Cecere checked the Libors again later that night and was annoyed to see that Citigroup had only reduced its six-month rate by a quarter of a basis point.

He wrote to Tan, “Can you speak with him again?”

The following day, Tan went back to the treasury desk in London as requested. He also forwarded the message chain to Andrew Thursfield, Citigroup’s head of risk treasury in London. The response he got back from his UK counterpart left little room for misinterpretation: it was a thinly veiled warning to back off.

Hayes, who sat just behind his boss, was not on the email chain, but Cecere sent it to him.

Thursfield was a straitlaced man in his forties who had spent more than 20 years in risk management at Citigroup after joining as a graduate trainee. He saw himself as the guardian of the firm’s balance sheet and didn’t take kindly to being told how to do his job by a pushy trader who knew nothing of the intricacies of bank funding.

Rather than lowering the inputs, Thursfield’s team increased its submission days later, pushing the published Libor rates higher. Hayes would have to try a different tack. On 14 December he sent an email to his London counterpart, asking him to approach the rate-setters directly.

“Do you talk to the cash desk and did we know in advance?” Hayes asked, referring to the bank’s decision to bump up its Libor submissions. “We need good dialogue with the cash desk. They can be invaluable to us. If we know ahead of time we can position and scalp the market.”

What Hayes didn’t realise was that no amount of schmoozing was going to get the rate-setters onside. Unlike some banks, Citigroup was taking the CFTC’s investigation into Libor seriously. In March 2009, Thursfield had personally delivered an 18-page presentation via video link to investigators on the rate-setting process. The cash traders weren’t about to risk their necks for someone they didn’t know who worked on the other side of the world.

It wasn’t just that they knew they were being watched. Thursfield was not only a stickler for the rules but had taken a personal dislike to Hayes when the pair had met three months earlier. It was October 2009, shortly after Hayes had accepted the job at Citigroup, and his boss had sent him to London to meet the bank’s key players.

“Good to meet you. You can help us out with Libors. I will let you know my axes,” Hayes said by way of an opening gambit when he was introduced to Thursfield.

Unshaven and dishevelled, Hayes told the Citigroup manager how the cash desk at UBS frequently skewed its submissions to suit his book. He boasted of his close relationships with rate-setters at other banks and how they would do favours for each other. Hayes was trying to charm Thursfield, but he had badly misjudged the man and the situation. The following day Thursfield called his manager, Steve Compton, and relayed his concerns.

“Once you stray on to talking about Libor fixings, I mean we just paid another $75,000 bill to the lawyer this week for the work they’re doing on the CFTC investigation,” Thursfield said. “Whoever is the desk head, or whatever, [should] have a close watch on just what he’s actually doing and how publicly. It’s all, you know, very much barrow-boy-type [behaviour].”

The knock on Hayes’s door came at 7am on a Tuesday, two weeks before Christmas 2012. Hayes padded down the bespoke pine staircase of his newly renovated home in Woldingham, Surrey, to let in more than a dozen police officers and Serious Fraud Office investigators. A year before, he had been fired from Citigroup, and shortly afterwards returned to the UK, where he married his girlfriend Sarah Tighe.

Hayes stood at his wife’s side as the officers swept through the property, gathering computers and documents into boxes and loading them into vehicles parked at the end of the gravel driveway. The couple had only moved in a fortnight before. Their infant son was upstairs in bed. Traffic was heavy by the time the former trader was led to the back of a waiting car. The 20-mile crawl from Surrey to the City of London passed in silence.

Bishopsgate police station is a grey, concrete building on one of the financial district’s busiest thoroughfares. In a formal interview, Hayes was told he had been brought in to answer questions relating to allegations that between 2006 and 2009 he had conspired to manipulate yen Libor with two of his colleagues. Hayes responded that he planned to help but would need time to consider the 112 pages of evidence so would not be answering any questions that day. It was late when he arrived back in Surrey.

In June, Barclays had become the first bank to reach a settlement with authorities, admitting to rigging the rate and agreeing to pay a then-record £290 million in fines. From the moment Barclays had settled, sparking a political firestorm that burned for weeks, Hayes’s destiny had been leading to this point. The Serious Fraud Office (SFO), which had previously resisted launching a probe into Libor rigging, was forced to reverse its position and on 6 July issued a statement announcing it would be undertaking a criminal investigation. That week the government launched its own review into the scandal. The British public and its politicians were out for scalps.

On 19 December, eight days after his arrest, Hayes was at home on his computer when a news bulletin popped up with a link to a press conference in Washington. As cameras flashed, Attorney General Eric Holder and Lanny Breuer, head of the Justice Department’s criminal division, took turns outlining the $1.5bn settlement the authorities had reached with UBS over Libor. The Swiss bank, they explained, had pleaded guilty to wire fraud at its Japanese arm. Then came the sucker punch.

“In addition to UBS Japan’s agreement to plead guilty, two former UBS traders have been charged, in a criminal complaint unsealed today, with conspiracy to manipulate Libor,” said Breuer. “Tom Hayes has also been charged with wire fraud and an antitrust violation.” Neither Tan nor Cecere has ever been charged with wrongdoing.

At that moment the full horror of the situation hit Hayes for the first time. The two most powerful lawyers in the US planned to extradite him on three separate criminal charges, each carrying a 20–30 year sentence. Less than 24 hours later, a member of Hayes’s legal team was on the phone to the SFO to discuss cutting a deal.

Fighting the charges seemed futile: the UBS settlement made reference to more than 2,000 attempts by Hayes and his colleagues to influence the rate over a four-year period. He was the star attraction, the “Jesse James of Libor”, as he would later tell it. The US authorities had yet to issue extradition papers, but it was only a matter of time.

RBS, Barclays and other banks fined in Swiss franc Libor case

So began a race to convince the SFO to take on Hayes as a sort of chief informant, who in return would receive leniency and, more importantly, an agreement that he would be dealt with in the UK.

To secure this arrangement Hayes had to agree to tell the SFO everything he knew and promise to testify against everybody involved. Crucially, he also had to plead guilty to dishonestly rigging Libor. It was not enough to admit trying to influence the rate. He had to confess that he knew it was wrong.

During two days of so-called scoping interviews to test his knowledge of the case, Hayes talked openly about his campaign to rig Libor, for the first time in his life. At the SFO’s offices near Trafalgar Square he admitted he had acted dishonestly and brought the investigators’ attention to aspects of the case they knew nothing about. The interviews covered everything from his entry into the industry and his trading strategies to how the Libor scheme began and the various individuals who helped him rig the rate. They barely had to prod to get him to talk. Hayes seemed to relish reliving moments from his past. His voice sped up when he talked about heady days piling into positions, squeezing the best prices from brokers and playing traders off against each other.

“The first thing you think is where’s the edge, where can I make a bit more money, how can I push, push the boundaries, maybe you know a bit of a grey area, push the edge of the envelope,” he said in one early interview. “But the point is, you are greedy, you want every little bit of money that you can possibly get because, like I say, that is how you are judged, that is your performance metric.”

Paper coffee cups piled up as Hayes went over the minutiae of the case. At one stage, Hayes was asked about how he viewed his attempts to move Libor around. The exchange would prove crucial.

“Well look, I mean, it’s a dishonest scheme, isn’t it?” Hayes said. “And I was part of the dishonest scheme, so obviously I was being dishonest.”

This article is adapted from The Fix: How Bankers Lied, Cheated and Colluded to Rig the World’s Most Important Number by Liam Vaughan and Gavin Finch