How algorithms secretly shape the way we behave

Algorithms, the key ingredients of all significant computer programs, have probably influenced your Christmas shopping and may one day determine how you vote

• 
• • John Naughton
  • The Observer, Sunday 16 December 2012

Program or be programmed? Schoolchildren learn to code. Photograph: Alamy

 

Keynes's observation (in his General Theory) that "practical men who believe themselves to be quite exempt from any intellectual influences, are usually the slaves of some defunct economist" needs updating. Replace "economist" with "algorithm". And delete "defunct", because the algorithms that now shape much of our behaviour are anything but defunct. They have probably already influenced your Christmas shopping, for example. They have certainly determined how your pension fund is doing, and whether your application for a mortgage has been successful. And one day they may effectively determine how you vote.

On the face of it, algorithms – "step-by-step procedures for calculations" – seem unlikely candidates for the role of tyrant. Their power comes from the fact that they are the key ingredients of all significant computer programs and the logic embedded in them determines what those programs do. In that sense algorithms are the secret sauce of a computerised world.

And they are secret. Every so often, the veil is lifted when there's a scandal. Last August, for example, a "rogue algorithm" in the computers of a New York stockbroking firm, Knight Capital, embarked on 45 minutes of automated trading that eventually lost its owners $440m before it was stopped.

But, mostly, algorithms do their work quietly in the background. I've just logged on to Amazon to check out a new book on the subject – Automate This: How Algorithms Came to Rule Our World by Christopher Steiner. At the foot of the page Amazon tells me that two other books are "frequently bought together" with Steiner's volume: Nate Silver's The Signal and the Noise and Nassim Nicholas Taleb's Antifragile. This conjunction of interests is the product of an algorithm: no human effort was involved in deciding that someone who is interested in Steiner's book might also be interested in the writings of Silver and Taleb.

But book recommendations are relatively small beer – though I suspect they will have influenced a lot of online shopping at this time of year, as people desperately seek ideas for presents. The most powerful algorithm in the world is PageRank – the one that Google uses to determine the rankings of results from web searches – for the simple reason that, if your site doesn't appear in the first page of results, then effectively it doesn't exist. Not surprisingly, there is a perpetual arms race (euphemistically called search engine optimisation) between Google and people attempting to game PageRank. Periodically, Google tweaks the algorithm and unleashes a wave of nasty surprises across the web as people find that their hitherto modestly successful online niche businesses have suddenly – and unaccountably – disappeared.

PageRank thus gives Google awesome power. And, ever since Lord Acton's time, we know what power does to people – and institutions. So the power of PageRank poses serious regulatory issues for governments. On the one hand, the algorithm is a closely guarded commercial secret – for obvious reasons: if it weren't, then the search engine optimisers would have a field day and all search results would be suspect. On the other hand, because it's secret, we can't be sure that Google isn't skewing results to favour its own commercial interests, as some people allege.

Besides, there's more to power than commercial clout. Many years ago, the sociologist Steven Lukes pointed out that power comes in three varieties: the ability to stop people doing what they want to do; the ability to compel them to do things that they don't want to do: and the ability to shape the way they think. This last is the power that mass media have, which is why the Leveson inquiry was so important.

But, in a way, algorithms also have that power. Take, for example, the one that drives Google News. This was recently subjected to an illuminating analysis by Nick Diakopoulos from the Nieman Journalism Lab. Google claims that its selection of noteworthy news stories is "generated entirely by computer algorithms without human editors. No humans were harmed or even used in the creation of this page."

The implication is that the selection process is somehow more "objective" than a human-mediated one. Diakopoulos takes this cosy assumption apart by examining the way the algorithm works. There's nothing sinister about it, but it highlights the importance of understanding how software works. The choice that faces citizens in a networked world is thus: program or be programmed.

A web of privilege supports this so-called meritocracy

On both sides of the Atlantic, the social ties that bind our political, legal and corporate forces lie exposed

• 
• • Gary Younge
  • guardian.co.uk, Sunday 6 May 2012 21.59 BST 
  • 
    

‘The meetings, lunches and visits showcase a parallel, unaccountable universe where decisions are made and deals done.' Photograph: Laurence Griffiths/Getty

Shortly after Mitt Romney's failed 2008 campaign for the Republican nomination his son Tagg set up a private equity fund with the campaign's top fundraiser. One of the first donors was his mum, Anne. Next came several of his dad's financial backers. Tagg had no experience in the world of finance, but after two years in the middle of a deep recession the company had netted $244m from just 64 investors.

Tagg insists that neither his name nor the fact that his father had made it clear he would run for the presidency again had anything to do with his success. "The reason people invested in us is that they liked our strategies,'' he told the New York Times.

Class privilege, and the power it confers, is often conveniently misunderstood by its beneficiaries as the product of their own genius rather than generations of advantage, stoutly defended and faithfully bequeathed. Evidence of such advantages is not freely available. It is not in the powerful's interest for the rest of us to know how their influence is attained or exercised. But every now and then a dam bursts and the facts come flooding forth.

The Leveson inquiry has provided one such moment. It was set up last year to look into the specific claims about phone hacking at the News of the World, alleged police corruption and the general culture and ethics of the British media. But every time it probes harder into the Murdoch empire it draws blood from the heart of our body politic, telling us a great deal about how Britain's political class in particular and ruling class in general collude, connive and corrupt both systemically and systematically.

Issues of alleged criminality will eventually be determined in the courts. But while illegality would be more damning, much of what we now know that is legal is no less corrosive. The evidence has laid bare the intimate, extensive and insidious web of social, familial and personal ties between the political, corporate and legal forces that govern a country: a patchwork of individual and institutional associations so tightly interwoven that to pick at one part is to watch the whole thing unravel. The "sit downs", pay-offs and class camaraderie on display owe more to a cross between Downton Abbey and the Sopranos than the functioning of a 21st-century democracy.

The details of the main narrative bear repeating. We now know that James Murdoch met with David Cameron 12 times between January 2006 and January 2010 – eight times for dinner, twice for breakfast, once for lunch and once for drinks. Between May 2010 and July 2011 there were also more than 60 meetings between ministers and either Rupert Murdoch, his son James, the then News International chief executive Rebekah Brooks or James Harding, the editor of the Times. That averages around one a week. We know there were more, but not all were logged as such by Downing Street.

The subplots are stunning. And the forthcoming attractions could yet overshadow the lot. Now that Brooks has agreed to hand over her text messages to Cameron, we are about to learn whether rumours that they exchanged as many as 12 texts a day are true.

Brooks was arrested both on suspicion of phone hacking and corruption last year. She was arrested again this year with her husband, Charlie Brooks, on suspicion of perverting the course of justice. Charlie went to Eton with Cameron – as did the Tory mayor of London, Boris Johnson.

Such is the incestuous nature of the British ruling class and the gene puddle from which it draws its stock. Such is their brazen venality, complicity, contempt and mendacity. Eton, Oxford, Bullingdon, Westminster – if you're looking for a tiny minority who are struggling to integrate, look no further than the cabinet.

Two things make this a matter of import as well as intrigue. The first is the lie it gives to the insistence on meritocracy at a time of acute economic crisis when benefits are slashed, the poor hammered. Cameron and his cabinet insist others pull themselves up by their bootstraps even as they themselves swan around in their parents' expensive pairs of loafers. Today almost 40% of MPs went to private school. In 1997 it was just 30%. In terms of social mobility, we are going backwards. The issue here is not class envy but class entrenchment. The fact that they were born rich is irrelevant. They had no choice in the matter. But the fact that they appear to want to give even more to those who already have a great deal while denying much to those who have little is unforgiveable.

The one job Cameron landed in the private sector was arranged by his wife's mother, Lady Astor, who was friends with Michael Green, then executive chairman of Carlton. Green gave Cameron a starting salary of £90,000. He has no more had to stand on his merits than James Murdoch had to interview for a job at News Corp.

Rocked in the cradle of power from birth so that its rhythms become second nature, these people imbibe their sense of entitlement with their mother's milk. But the personal tutors, private schools, the most expensive universities do not, somehow, suffice. As though the benefits of wealth were not enough, they apparently feel the need to game the very system they already control.

Which brings us to the manner in which these interactions mock the very notion of democracy on which the nation's illusions are based. For the meetings, lunches and visits showcase a parallel, unaccountable universe where actual decisions are made and deals are done. All these informal gatherings took place at a time when the government was supposed to be adjudicating News Corporation's bid to take over BSkyB. With the culture secretary described by Murdoch's lobbyist as a "cheerleader" for News International, it seems as if the takeover was to all intents and purposes a done deal, prevented only by the fallout from the hacking scandal. All the kinks ironed out on horseback and settled in time for the main course. Parliament would have been a mere rubber stamp. Oversight reduced to an afterthought in a House of Commons that may soon more closely resemble a house of cards.

We know that to understand politics and the peddling of influence we must follow the money. So it’s remarkable that the question of who funds the thinktanks has so seldom been asked.

Think Of A Tank

George Monbiot

 

Nadine Dorries won’t answer it. Lord Lawson won’t answer it. Michael Gove won’t answer it. But it’s a simple question, and if they don’t know it’s because they don’t want to. Where does the money come from? All are connected to groups whose purpose is to change the direction of public life. None will reveal who funds them.

When she attempted to restrict abortion counselling, Nadine Dorries MP was supported by a group called Right to Know. When other MPs asked her who funds it, she claimed she didn’t know(1,2). Lord Lawson is chairman of the Global Warming Policy Foundation, which casts doubt on climate science. It demands “openness and transparency” from scientists(3). Yet he refuses to say who pays, on the grounds that the donors “do not wish to be publicly engaged in controversy.”(4) Michael Gove was chairman of Policy Exchange, an influential conservative thinktank. When I asked who funded Policy Exchange when he ran it, his office told me “he doesn’t have that information and he won’t be able to help you.”(5)

We know that to understand politics and the peddling of influence we must follow the money. So it’s remarkable that the question of who funds the thinktanks has so seldom been asked.

There are dozens of groups in the UK which call themselves free market or conservative thinktanks, but they have a remarkably consistent agenda. They tend to oppose the laws which protect us from banks and corporations; to demand the privatisation of state assets; to argue that the rich should pay less tax; and to pour scorn on global warming. What the thinktanks call free market economics looks more like a programme for corporate power.

Some of them have a turnover of several million pounds a year, but in most cases that’s about all we know. In the US, groups claiming to be free market thinktanks have been exposed as sophisticated corporate lobbying outfits, acting in concert to promote the views of the people who fund them. In previous columns, I’ve shown how such groups, funded by the billionaire Koch brothers, built and directed the Tea Party movement(6,7).

The Kochs and the oil company Exxon have also funded a swarm of thinktanks which, by coincidence, all spontaneously decided that manmade climate change is a myth(8,9). A study in the journal Environmental Politics found that such groups, funded by economic elites and working through the media, have been “central to the reversal of US support for environmental protection, both domestically and internationally.”(10)

Jeff Judson, who has worked for 26 years as a corporate lobbyist in the US, has explained why thinktanks are more effective than other public relations agencies. They are, he says, “the source of many of the ideas and facts that appear in countless editorials, news articles, and syndicated columns.”(11) They have “considerable influence and close personal relationships with elected officials”. They “support and encourage one another, echo and amplify their messages, and can pull together … coalitions on the most important public policy issues.” Crucially, they are “virtually immune to retribution … the identity of donors to think tanks is protected from involuntary disclosure.”(12)

The harder you stare at them, the more they look like lobby groups working for big business without disclosing their interests. Yet throughout the media they are treated as independent sources of expertise. The BBC is particularly culpable. Even when the corporate funding of its contributors has been exposed by human rights or environmental groups, it still allows them to masquerade as unbiased commentators, without disclosing their interests.

For the sake of democracy, we should know who funds the organisations which call themselves thinktanks. To this end I contacted 15 groups. Eleven of them could be described as free market or conservative; four as progressive. I asked them all a simple question: “Could you give me the names of your major donors and the amount they contributed in the last financial year?”. I gave their answers a score out of five for transparency and accountability.

Three of the groups I contacted – Right to Know, the International Policy Network and Nurses for Reform – did not answer my calls or emails. Six others refused to give me any useful information. They are the Institute of Economic Affairs, Policy Exchange, the Adam Smith Institute, the TaxPayers’ Alliance, the Global Warming Policy Foundation and the Christian Medical Fellowship. They produced similar excuses, mostly concerning the need to protect the privacy of their donors. My view is that if you pay for influence, you should be accountable for it. Nul points.

Civitas did fractionally better, scoring 1. Its website names a small number of the donors to its schools(13), but it would not reveal the amount they had given or the identity of anyone else. The only rightwing thinktank that did well was Reform, which sent me a list of its biggest corporate donors: Lloyds (£50k), Novo Nordisk (£48k), Sky (£42k), General Electric (£41k) and Danone (£40k). Reform lists its other corporate sponsors in its annual review(14), and earns 4 points. If they can do it, why can’t the others?

The progressives were more accountable. Among them, Demos did least well. It sent me a list of its sponsors, but refused to reveal how much they gave. It scores 2.5. The Institute for Public Policy Research listed its donors and, after some stumbling, was able to identify the biggest of them: the European Union (a grant of E800,000) and the Esmee Fairburn Foundation(£86k). It scores 3.5. The New Economics Foundation sent me a list of all its donors and the amount each gave over the past year, earning 4 points. The biggest funders are the Network for Social Change (£173k), the department of health (£124k) and the Aim Foundation (£100k). Compass had already published a full list in its annual report(15). The biggest source by far is the Communication Workers’ Union, which gave it £78k in 2009. Compass gets 5 out of 5.

The picture we see, with the striking exception of Reform, is of secrecy among the rightwing groups, creating a powerful impression that they have something to hide. Shockingly, this absence of accountability – and the influence-peddling it doubtless obscures – does not affect their charitable status.

The funding of these groups should not be a matter of voluntary disclosure. As someone remarked in February 2010, “secret corporate lobbying, like the expenses scandal, goes to the heart of why people are so fed up with politics … it’s time we shone the light of transparency on lobbying in our country and forced our politics to come clean about who is buying power and influence.”(16) Who was this leftwing firebrand? One David Cameron.

I charge that the groups which call themselves free market thinktanks are nothing of the kind. They are public relations agencies, secretly lobbying for the corporations and multi-millionaires who finance them. If they wish to refute this claim, they should disclose their funding. Until then, whenever you hear the term free market thinktank, think of a tank, crushing democracy, driven by big business.

www.monbiot.com