Housing in the UK: The nightmare of renting started in Westminster

The human kennels we hear about didn’t spring up by accident. This broken market is the result of 30 years of bad housing policy

• 
• • Aditya Chakrabortty
  • The Guardian, Monday 25 August 2014 20.22 BST

'Report after report shows that homes in the private rental sector are far worse than either council housing or those under owner-occupation.' Illustration by Daniel Pudles

Two foot three inches isn’t much. It’s only a little higher than the world’s smallest woman, Jyoti Amge. Your average two-year-old has long rocketed past that line. Glue a couple of my school rulers together and you’re nearly there.

To get to their rented room in Hendon, on the outskirts of north London, tenants of Yaakov Marom had to crawl up a staircase with a head height as low as 2ft 3in (69cm). True, in parts they could stretch up to 3ft 11in (119cm) – which is just about enough headroom for an Ewok. For the privilege of sleeping in a human kennel, a couple were paying Marom £420 a month.

Blame it on my own frayed synapses, but I can no longer get quite so shocked by such stories. How many versions of it have we seen before? Beds in sheds; lodgers in garages; tiny studios let for huge sums. As this paper reported on Saturday, just down the road from Marom’s palace is another rental, offering a single bed suspended from the ceiling by two metal chains. This macabre cross between a hammock and a torture chamber can be yours, friends, for £760 a month, parking permit extra.

All these dispatches from bedsitland tell us two things. First, the private rental market is red-hot – otherwise, landlords wouldn’t be trying to monetise every patch. And second, the private rental market is badly broken.

Welcome to the new age of landlordism, in which the property-owner has all the power and the renter hardly any choice. This year’s English Housing Survey revealed that the number of private tenants had outstripped those in social housing for the first time in its history.

The disparity between those tenures is like the gulf between day and night, between a home and a rabbit hutch. Council tenants get security of tenure and controlled rents; shorthold tenants pay up to four times as much and under most contracts are only ever two months’ notice from getting turfed out of their homes. Yet the impossibility of first-time buying, and the scarcity of public housing, means the private rental market has taken off. The 2001 census showed 1.9m households renting privately in England and Wales – now there are 4m in England alone.

Report after report shows that homes in the private rental sector are far worse than either council housing or those under owner-occupation. One in three are officially classed as non-decent, while one in five are dangerous enough to present a category one hazard – that is, a severe threat to the health or safety of anyone who lives there. All those tenants’ tales you’ve heard or read about permanently broken boilers or mould carpeting the walls aren’t just anecdotes; they cohere into a statistical truth. One of the richest countries in history is fostering 21st-century slums.

It’s in these conditions that millions of people will live for good. Rental is no longer a stepping stone for students and young professionals; instead, it’s fast becoming a terminus. Well over a million families with children now rent. Just as lack of choice has triggered the rise of private renting, so it will keep a growing number of households stewing there. Polls suggest that around 80% of Britons would rather own a home than rent; the lack of new homes suggests that many under-35s without rich parents will be renting for decades to come.

Let me make the obvious disclaimer: not all landlords are on the take, nor are all tenants angels mindful of fixtures and fittings, and keeping the music down. Not that it matters, because a market characterised by this much demand doesn’t really reward good landlords or penalise bad ones. Because the laws give the landlord the power. Because anyone can set up as a letting agent, without qualifications or licensing – and to be one is to own a printing press of made-up fees. Because even if tenants complain, they could face a retaliatory eviction. Because unless a council is tipped off, they’ve probably been too badly hit by cuts to find a scam (Marom had already been banned by Barnet from letting out his second-floor room; but it still took nearly 18 months for officers to catch him at it).

It’s easy to look at this market, with its surveys indicating that 92% of landlords rent out property on the side, and conclude that the entire thing is an epic, ugly accident. Not so: this is Westminster’s creation. Since Margaret Thatcher – at least – successive governments have promised a property-owning democracy, all the while laying the ground for a new landlordism. Thatcher did the most, privatising council homes through right-to-buy, then bringing in the Housing Act 1988 – the big bang for the private rental sector, shredding the last vestiges of rent controls and most protections for tenants. John Major presided over the assured shorthold tenancy and the buy-to-let mortgage. Tony Blair and Gordon Brown refused to countenance the building of new public housing. David Cameron’s contribution has been the Localism Act, which requires councils to put homeless people in the private rental sector: £11bn for build-to-rent, and more right-to-buy.

Whatever the rhetoric, home ownership in England is back down to where it was in 1987. One in three former council homes are now held by private landlords. Tory, Labour and Lib Dems have all taken turns in creating a regime that – as James Meek notes in Private Island, his excellent new book on Britain’s privatisations – “puts more money into the hands of a small number of the very wealthiest people”. To underline the point: this is our money, including the £20bn we pay every year in housing benefit that swiftly goes into landlords’ hands.

That’s the backdrop against which you should judge the current promises made by all the parties to bring in a few extra protections for tenants. None talk of licensing landlords – despite the calls from town halls – let alone guaranteeing more public housing. To do so would be to attack a sector the political classes have cultivated for three decades, and has grown too powerful to hack back – a sector that includes much of the Commons: one in four Tory MPs are landlords, as are one in eight Labour MPs. As for the new housing minister, Conservative Brandon Lewis, would it really surprise you to learn that the parliamentary register has him down as a private landlord?

I envy the Scots. If only we English could also shake up our democracy

With its independence vote Scotland is able to ask big questions about its destiny. I have no idea what that feels like

• 
• • Suzanne Moore
  • theguardian.com, Monday 25 August 2014 12.41 BST

'I do not know what it is to be Scottish and feel utterly disconnected from Westminster politics. I only know what it is to be English and feel like that.' Photograph: Jeff J Mitchell/Getty Images

Imagine having a say, an actual say, in how your country is run. You might even consider big questions, like the distribution of wealth, the dividing up of resources, your tiny country’s place in the big bad world. You may go down the pub and argue with your friends about what your identity actually means, you may be already certain and slag off those who think differently on Twitter, you may be wavering on several issues. You may be just 16 and able to vote for the fist time in your life.

This may or may not be my business, depending of course which way Scotland votes in a few weeks time, but as this referendum hoves into view with another televised debate tonight, my main emotion is perhaps pathetically English. It is one of envy.

This conversation, this questioning about who you are, how you are to live and who is to be in charge, shows the essential internal organs of a political system being openly tested and examined. Here in Albion, there appears to be only stubbornly undiagnosed organ failure. I do not know what it is to be Scottish and feel utterly disconnected from Westminster politics. I only know what it is to be English and feel like that.

Every time I voice that, inevitably, I am told that somehow I should vote more. When and where exactly? Instead I see that power clusters in ever closer elites at the top of society. These closed shops hug themselves tighter than David Cameron’s wetsuit. The problem, we are told, is that Cameron surrounds himself with yes men. And the trouble with Ed Miliband is that he surrounds himself with people who are very similar to him, and so it goes on until the only real alternatives on offer are Nigel Farage or Boris Johnson. Some choice.

As I see it the Scots are always patronised by talking about independence in terms ofmarriage, divorce, and breaking up the CD collection. Yet I wonder what kind of relationship ever existed in the first place. It has long been apparent that what has gone on in the south-east, and in the City of London in particular, has clearly not been faithful to any of the values of a fairly basic social democracy. It has been about strident privatisation and the dismantling of the welfare state. Scotland at least gets the chance to reject this, English people don’t.

Because many of us have never lived in a United Kingdom. These words mean nothing to me.

Whatever the Scots decide, England will shake. And it needs shaking. If England is actually a Tory land and needs Scotland as a Labour colony, then England should face up to itself, and its demons.

Independence is also often framed in terms of maturity. Is Scotland ready for independence? Surely it is the other way round. Is England grown-up enough to see itself the way it actually is? For that, we would have to be seen and heard, reconnected to the democratic processes, and be far more honest with ourselves than we are often prepared to be.

The best debates about Scottish independence have asked big questions and often there are no easy answers. There is a truthfulness in that. Imagine if England could ask the same questions of itself instead of assuming that all was settled long ago. Imagine if we could do this without ceding to Ukip-style regression.

Scotland will do whatever it does. As I watch from afar, I cant help thinking for us English, chance would be a fine thing.

The Myth of Common Sense: Why The Social World Is Less Obvious Than It Seems

By Duncan Watts

“Mankind, it seems, makes a poorer performance of government than of almost any other human activity.”
-Barbara Tuchman, The March of Folly

“This is not rocket science”
-Bill Frist on fixing health care, The New York Times

As these two quotes illustrate, there is something strangely conflicted about contemporary views on government and policy. On one hand, many people are in apparent agreement that government frequently accomplishes less than it ought to, sometimes embarrassingly so. Yet on the other hand, many of these same people are also of the opinion that the failings of government do not imply any great difficulty of the problems themselves—that they are not rocket science, as it were.

Typically the conflict is resolved with reference to the presumed incompetence, pigheadedness, or outright corruption of our leaders. If only we elected the right people, gave them the right incentives, and—above all—if only our political leaders exhibited a little more common sense, everything would be alright. That both “we” and “they” consistently fail to follow these simple steps proves only that common sense is not nearly common enough.

There may be some truth to this attitude. But as a sociologist, I’ve also learned to be skeptical of common sense, especially when it is invoked as the solution to complex social problems.

Sociology has had to confront the criticism that it has “discovered” little that an intelligent person couldn’t have figured out on his or her own.

Sociology, of course, has its own conflicted history with common sense. For almost as long as it has existed, that is, sociology has had to confront the criticism that it has “discovered” little that an intelligent person couldn’t have figured out on his or her own.

Why is it, for example, that most social groups, from friendship circles to workplaces, are so homogenous in terms of race, education level, and even gender? Why do some things become popular and not others? How much does the media influence society? Is more choice better or worse? Do taxes stimulate the economy?

Social scientists have struggled with all these questions for generations, and continue to do so. Yet many people feel they could answer these questions themselves—simply by examining their own experience. Unlike for problems in physics and biology, therefore, where we need experts to tell us what is true, when the topic is human or social behavior, we’re all “experts,” so we trust our own opinions at least as much as we trust those of social scientists.

Nor is this tendency necessarily a bad reaction—any theory should be consistent with empirical reality, and in the case of social science, that reality includes everyday experience. But not everything about the social world is transparent from common sense alone—in part because not everything that seems like common sense turns out to be true, and in part because common sense is extremely good at making the world seem more orderly than it really is.

Common sense isn’t anything like a scientific theory of the world.

As sociologists are fond of pointing out, common sense isn’t anything like a scientific theory of the world. Rather it is a hodge-podge of accumulated advice, experiences, aphorisms, norms, received wisdom, inherited beliefs, and introspection that is neither coherent nor even internally self-consistent. Birds of a feather flock together, but opposites also attract. Two minds are better than one, except when too many cooks spoil the broth. Does absence make the heart grow fonder, or is out of sight out of mind?  At what point does try, try again turn into flogging a dead horse? And if experience is the best teacher, when should one also maintain a beginner’s mind?

The problem with common sense is not that it isn’t sensible, but that what is sensible turns out to depend on lots of other features of the situation. And in general, it’s impossible to know which of these many potential features are relevant until after the fact (a fundamental problem that philosophers and cognitive scientists call the “frame problem”).

Nevertheless, once we do know the answer, it is almost always possible to pick and choose from our wide selection of common-sense statements about the world to produce something that sounds likely to be true. And because we only ever have to account for one outcome at a time (because we can ignore the “counterfactuals,” things that could’ve happened, but didn’t), it is always possible to construct an account of what did happen that not only makes sense, but also sounds like a causal story.

Photo:choupigloupi

To take a common example, think about how we explain success. Why is the Mona Lisa the most famous painting in the world?  Why did J.K. Rowling‘s Harry Potter books sell over 300 million copies?  And why is Madonna the most successful female musical artist of all time? Now that we know who these superstars are, their success seems easy to explain—common sense even. They simply outperformed the competition. Whether they did that through pure genius, clever marketing, or sheer tenacity is a matter of debate (you be the judge), but in the end, it doesn’t really matter.  In the competitive marketplace of ideas, a product succeeds because it represents what people want—otherwise, they wouldn’t have devoted their scarce time, money, and attention to it. Right?

Photo: Joaquín Martínez Rosado

Well, sort of. But if that’s true, why are superstars so hard to pick out in advance?  Why did several children’s publishers reject the initial Harry Pottermanuscript? Why did no one pay much attention to the Mona Lisa for nearly 400 years? And why did music critics dismiss the early Madonna as an attention seeker with limited talent? Whatever their personal preferences, how could they have failed to understand the demands of the marketplace, which after all is precisely what they are rewarded for doing?

Nor is it just the critics who get their predictions wrong—marketers can’t seem to figure out what will sell either. If they could, they wouldn’t waste their efforts on the vast majority of books, movies, and albums that lose money. So what explains why some cultural products are stunningly successful, while most aren’t; and why at the same time, no one, including the experts, seems to be able to predict which is which?

A few years ago, my students and I studied exactly this question by setting up an experiment in which roughly 15,000 participants were asked to listen to, rate, and download songs by unknown bands off a website we created. Some of the participants had to make their decisions independently, while others had information about which of the songs other people liked.  We found two results. First, in the “social influence” condition, popular songs were more popular (and unpopular songs less popular) than in the independent case. But second, it became harder to predict which particular songs would be the most popular.

What these results suggest is that in the real world, where social influence is much stronger than in our artificial experiment, enormous differences in success may indeed be due to small, random fluctuations early on in an artist’s career, which then get amplified by a process of cumulative advantage—a “rich-get-richer” phenomenon that is thought to arise in many social systems.

The random fluctuations arising from social influence were larger than those arising from differences in quality.

A critical feature of this experiment was our ability to create multiple “worlds” in which randomly assigned groups of people could create different versions of history in parallel with each other. By observing how popular the same song became in different worlds, we could measure directly how much of its success can be attributed to some intrinsic “quality,” and how much results from random chance. We found that although, on average, good songs do better than bad songs, the random fluctuations arising from social influence were larger than those arising from differences in quality.

The problem with this explanation, however, is that in real life we never get to experience these multiple histories—only the one history that we have lived through. So although one can argue that Madonna or Harry Potter or even Shakespeare may owe their success more to random chance and cumulative advantage than to intrinsic superiority, it is impossible to refute the common sense view that history took the path that it did because the winners embodied precisely the greatness to which we attribute them. And for a Madonna or a Harry Potter or a Shakespeare fan, that is typically the end of the argument.

Common sense, in other words, is extremely good at making the world seem sensible, quickly classifying believable information as old news, rejecting explanations that don’t coincide with experience, and ignoring counterfactuals. Viewed this way, common sense starts to seem less like a way to understand the world, than a way to survive without having to understand it.

That may have been a perfectly fine design for most of evolutionary history, where humans lived in small groups and could safely ignore most of what was going on in the world. But increasingly the problems of the modern world—distributions of wealth, sustainable development, public health—require us to understand cause and effect in complex systems, with consequences unfolding over years or decades. And for these kinds of problems, there’s no reason to believe that common sense is much of a guide at all.

Fortunately, in recent years the explosive growth of the Internet has brought with it the ability to measure the actions and interactions of millions of people simultaneously. Not just social networking services like Facebook, Twitter and YouTube, but email interactions, instant messaging, and Internet phone calls all provide measurable traces of the person-to-person interactions that have always been at the core of social life, but have until recently been invisible to science.

Already this flood of data has generated enormous interest in the research community, with thousands of physicists and computer scientists beginning to pay attention to problems traditionally in the domain of the social sciences. It’s tempting to look back at past technological breakthroughs, such as the telescope or the microscope, when new instrumentation made the previously invisible visible, and wonder if perhaps social science is on the edge of its own scientific revolution.

Social problems, that is, must be viewed not as the subject of rhetorical debates, but as scientific problems

But if we are to make use of these impressive new capabilities to address the kinds of problems that governments have historically failed to solve, we also need to think differently about the problems themselves. Social problems, that is, must be viewed not as the subject of rhetorical debates, but as scientific problems, in the sense that some combination of theory, data, and experiment can provide useful insights beyond that which can be derived through intuition and experience alone.

Clearly we’re a long way from a world in which cause and effect in social and economic systems can be established with the level of certainty we’ve come to expect from the physical sciences. In fact, the world of human behavior is sufficiently complicated and unpredictable that no matter how long or hard we try, we will always be stuck with some level of uncertainty, in which case leaders will have to do what they’ve always done and make the best decisions they can under the circumstances.

It sounds like a lot of effort for an uncertain payoff, but curing cancer has also proven to be an enormously complex undertaking, far more resistant to medical science than was once thought, yet no one is throwing up their hands on that one. It is time to apply the same admirable resolve to understand the world—no matter how long it takes—that we display in our struggles to address the important problems of physical and medical science to social problems as well.

Sale of the century: the privatisation scam

Privatisation promised to turn the UK into an island of small shareholders. It failed: the faceless state bureaucrats have been replaced by faceless (better-paid) private bureaucrats – and big foreign corporations. How did we get to this point?

• 
• • James Meek
  • The Guardian, Friday 22 August 2014 16.36 BST

London Bridge train station. Photograph: Alex Segre/FlickrVision

Train fares are going up. We learned that last week, although "learn" is putting it strongly. We knew they would. It's not as if they would go down: train fares go up, like electricity bills, gas bills, water bills, rent and chief executives' salaries. To the loyalists of the Thatcher-Blair-Cameron succession, higher train fares are a positive, because they mean lower subsidies: another incremental step in a 35-year programme to shift the burden of paying for infrastructure from the well-off to the strugglers. To most of us, it's another sign of the folly of selling off the railways. But amid the dismal annual round of fare rises, it's easy to miss another, stranger, more gradual sign of the failure of the vast social and economic experiment conducted on the British people since 1979: privatisation.

A trio of awkward synthetic words has begun to appear among the owners of private train companies that looks as if a computer has been asked to name the new musketeers: Abellio; Govia; Keolis. What these bland corporate signifiers mask is state-owned but commercialised European rail firms. Collectively, European state railways now own more than a quarter of Britain's passenger train system.

I imagine they will do a decent job. And that's the trouble. If competition shows that the best companies to run Britain's privatised railways are state-owned railways from other countries, what does that say about the justification of privatisation? And what does it say about what privatisation has done to Britain? How did we get to the point where this country's railways, power stations and postal service were ready to be taken over by foreign versions of British organisations that our own government, claiming patriotism, systematically took to pieces?

One winter's morning in 1991 I loaded a guitar, a condensed edition of the Oxford English Dictionary and a Teach Yourself Russian course into an old Volkswagen, left the house near Edinburgh where I had been staying and drove to Kiev. Five days passed on the road. I left the familiarity, order and prosperity of Britain, the island where I had grown up, and travelled east to wait for the Soviet Union to dissolve. I didn't have to wait long. A few weeks after I arrived, it ceased to be. Russia and Ukraine went their separate ways. The Kiev traffic policeman waving down my foreign-plated car had time to utter the words, "What are you doing in the Soviet Union?" before the colour left his face, his mouth went dry, and he turned away, lost, a bully orphaned of his corporate father.

A 70-year experiment to test whether the ethos of the commune could be imposed on a transcontinental empire of hundreds of millions of people was over, long after the answer was in (it couldn't). I wasn't sorry to see Soviet communism go. Despite all that's happened since, I still don't mourn it. There was hope in the beginning that something fine would grow in the gap that was left. It was a while before I realised the cynical, grasping figures who moved in to take possession of the ruins were not, as I had hoped, transitional symptoms of change, but the essence of that change.

 Ukrainian women exchange Soviet rubles against Ukrainian karbovanets, on January 17, 1992 in Kiev. Photograph: SERGEI SUPINSKY/AFP

Watching the vultures come to feast on the carcass of the world's largest state-owned, planned economy, I began to find the terms to question what had been done by politicians, economic theorists, lobbyists and business people in my own country. I had thought, when I left Scotland, in the unconscious way certainties are stowed in one's mind, that I knew Britain; that some essential way of being would be resilient to Margaret Thatcher's rearrangements, which must, as transient policies, be superficial. I had to go home by way of Kiev and Moscow to see that I was wrong, to begin to see how, and how deeply, she and her followers altered Britain.

With hindsight, 1991 was a pivotal year. When it began, the free market economic belief system, with its lead proselytisers Thatcher and Ronald Reagan, had been pushing back for more than a decade against various attempts to impose levelling communitarianism around the world. The Berlin Wall had fallen, as had communist regimes in Poland, Czechoslovakia, Hungary, Romania and Bulgaria. The market belief system, which holds that government is incompetent by default, that state taxation is oppressive, that the desire for wealth is the right and principal motivator of achievement and that virtually all human wants can best be met by competing private firms, was becoming entrenched in the non-communist world, from Chile to New Zealand. Made bold by a popular public perception that government overspending and selfish organised labour were to blame for economic stagnation and high inflation in the 1970s, Thatcher and Reagan had taken on powerful trade unions, and won. Barriers to the international movement of goods and money had fallen; the European Union was, on paper, a single marketplace. In Britain, restrictions on how much ordinary people could borrow to finance their everyday needs had been scrapped, and millions had acquired credit cards. Volumes of regulations controlling how banks were allowed to use people's deposits had been torn up, and unimaginably vast sums were being moved privately from country to country. Government spending had been cut, as had income tax and corporation tax. Sales tax and fees for everyday services had been raised. Council houses and big state enterprises had been privatised, with more on the way, leading to hundreds of thousands of redundancies. Thatcher's programme in Britain was an inspiration for the IMF and theWorld Bank as they experimented with the conditions they attached to bail-out loans to developing countries.

 Margaret Thatcher in Bournemouth, Dorset in 1986. Photograph: David Levenson/Getty Images

But at the end of 1990, the triumph of marketism seemed to hang in the balance. Reagan and Thatcher had relinquished the stage to less fervent, less charismatic successors. The man who'd introduced the market economy to China, Deng Xiaoping, had been blamed by traditional communists for fostering the Tiananmen Square protests, and was in disgrace. In the Soviet Union Mikhail Gorbachev, the great hope of free marketeers, was facing a similar backlash from hardliners, and the Baltic countries' hopes of escape from the USSR looked bleak. Saddam Hussein, dictator of semi-socialist Iraq, had invaded semi-capitalist Kuwait.

Yet the following year conviction began to grow among the marketeers that the final defeat of centrally planned, communitarian government was at hand, the sense that seemed to confirm such ideas as America having "won" the Cold War, and the "end of history". Early in 1991 it became clear that the Soviet leadership had lost the necessary unanimity and ruthlessness to keep Lithuania within the USSR. The humiliating collapse of the coup against Gorbachev that summer presaged recognition of Baltic independence, Ukraine's vote to go the same way, and the end of the Soviet Union. In Kuwait at the beginning of the year I saw experienced British war correspondents squabble for reporting billets among the frontline troops with the ferocity of those who believe something is being offered for the last time; we thought British and American armies might never fight another war. Few doubted Saddam would be beaten, and he was. That November, as I drove off the ferry at Ostend, heading east, it seemed a racing, expanding tide of victorious free marketism glimmered at my wheels, a tide that has gone by many names – consumer capitalism, Reaganism, Thatcherism, neoliberalism, the Washington Consensus. Though the watchtowers still stood at the old border between two Germanys, the border was gone. In eastern Germany, the narrow cobbled streets of medieval towns had jammed solid with second-hand cars. I passed a field where an impatient western German DIY chain, unwilling to wait for steel and breeze blocks, had erected a vast, circular retail marquee, blazing with lights. The canvas superstore seemed to have landed, like a spacecraft from a flashier civilisation, come down to offer shrink-wrapped packs of rawl plugs and a choice of bathroom fittings. In Poland, I got lost in fog near Wrocław, and saw how small shops had sprung up everywhere, even in the tiniest villages. In the middle of the night, in the middle of nowhere, in damp, coal-scented murk so thick I wasn't sure which way my car was pointing, I came across an entrepreneur hawking coffee from a roadside kiosk; the best coffee I ever tasted. He was like a champion of Thatcherite values, the small businessman standing ready to serve at all hours, in all weathers, making up for lost time under communism, silently mocking the market-questioning scepticisms I had brought with me from Scotland.

The effect on me of witnessing the unplanned collapse of a planned economy, where there'd been virtually no private property or private enterprise, was a series of viscerally direct lessons in economics. I saw how badly the Soviet communist system had failed on economic grounds alone, quite apart from its denial of personal freedoms. Long before the end, there was a hopeless housing shortage. Multiple households were sharing two-roomed flats; families were living in dormitories. Apartments seized from their bourgeois owners after the 1917 revolution were still unrepaired more than 70 years later. The infrastructure was rotten; there were cities and suburbs built around factories in the 1960s and 1970s where homes only had mains water for a few hours a day. Surpluses of goods nobody wanted (copies of the complete works of Soviet politicians, busts of Lenin) prevailed beside shortages of goods everybody wanted (cheese, coffee, sausage) because the element sticking together demand for a thing and the amount of trouble it took to produce and deliver it – the price – had been scraped out of transactions and replaced with a made-up figure concocted by planners in Moscow. Inequality was rampant, reflected not just in monetary wealth or property but in the degree to which you actually had access to the cheap goods everyone was supposed to have access to. One consequence of food and drink being allocated by civil servants according to central decrees, rather than by price, was that the restaurant business became an incubator for the black market and organised crime. Airports and railway stations looked like refugee camps because tickets cost virtually nothing, yet there weren't enough flights or trains to move the people who wanted to take them. The first response of the Russian and Ukrainian authorities after independence was to massively increase the production of a single essential item that people were chronically short of: money. Hyperinflation resulted, and millions of people had their savings wiped out.

The other side of the collapse of communism, along with the post-Soviet boons of freedom of movement, freedom of expression and freedom of initiative, was the flourishing of enterprise. Armies of tough middle-aged women made epic journeys to the bazaars of Poland, Turkey and China and returned to Ukraine and Russia with clothes to dress a handsome people as they'd yearned to dress, in jeans, leather and gold. Shops, restaurants, bars, cafés and night clubs opened up; book and music stalls were everywhere. Foreign firms brought wonders: a tampon factory, international direct dialling. Kiev went from a place where you couldn't buy anything to a place where you could buy anything, if you had the means.

Contempt for the planned economy, a new appreciation of the danger of printing excess money, gratitude to the entrepreneurs – there were times, in those early months in Kiev, that I asked myself whether I was becoming a Thatcherite. I can't pinpoint the moment when it soured for me. It might have been the sight of a solid rank of impoverished pensioners, some several hundred respectable old ladies, standing shoulder to shoulder in the freezing winter darkness outside Belarus station in Moscow, each holding a single sausage for sale – the free market as desperation. Or a visit to the Arctic mining city of Vorkuta, where miners were being paid in sandwiches while their bosses pocketed the money from the coal for which they were earning free-market prices.

 A surgeon performs a neck and throat operation in the recently opened Birmingham Queen Elizabeth Hospital Photograph: Christopher Furlong/Getty Images

In the first stages of disillusionment, it didn't seem obvious to me to make connections between the extremes of marketisation and privatisation in the former Soviet Union and the partial privatisation of a British economy that had always been mainly private anyway. After all, where Britain had a series of regulators to set rules for the privatised industries – Ofcom, Ofwat and so on – the principal regulator of privatisation in Ukraine and Russia, at least in the early days, was murder. In Russia in particular, a small number of individuals quickly became fantastically rich when they took private control of state producers of petrochemicals and metals. They were grotesquely rewarded, or grotesquely undertaxed, and money that should have gone to rebuild roads or hospitals or schools went instead towards yachts, property in London and foreign football teams. But that had nothing in common with privatisation in Britain – did it?I began to notice something odd about the British and American business people and financial advisers I met in Ukraine and Russia in the 1990s. It was no surprise, I suppose, that they cared more about businesses being overtaxed than undertaxed, more about protection of private property than about protection of pensioners; that they didn't care how weak and bullied the local trades unions were. Besides, their Russian interlocutors kept being assassinated. What was revealing was how many of these emissaries of the capitalist way seemed to believe the myth that all that was good in the British and American economies had been constructed by the free market. They seemed to believe, or talked, made speeches, wrote papers as if they believed, that the entire structure of their own wealthy modern societies – the roads, the electricity grids, the railways, the water and sewage systems, the universal postal services, the telecoms networks, housing, education and health care – had been brought into being by individual entrepreneurs driven by desire for gain, with the occasional lump of charity thrown in, and that a bloated, parasitical state had come shambling onto the scene, seizing assets and demanding free stuff for its shirker buddies. I don't want to absolve the Russians or Ukrainians of responsibility for their handling of the aftermath of communism, but the template they were handed by the fraternity of the Washington Consensus was based on fake history. If this is what the triumphalists of Wall Street and the City of London told the Russians about the way of the capitalist world, I thought when I moved back to Britain in 1999, what have they been telling us? And what came of it?

When Thatcher's Conservatives came to power in Britain in 1979, much of the economy, and almost all its infrastructure, was in state hands. Exactly what gloss you put on "in state hands" depends on your political point of view. For traditional socialists, it meant "the people's hands". For traditional Tories, it meant "in British hands". For Thatcher and her allies, it meant "in the hands of meddling bureaucrats and selfish, greedy trade unionists". How much of the economy? A third of all homes were rented from the state. The health service, most schools, the armed forces, prisons, roads, bridges and streets, water, sewers, the National Grid, power stations, the phone and postal system, gas supply, coal mines, the railways, refuse collection, the airports, many of the ports, local and long-distance buses, freight lorries, nuclear-fuel reprocessing, air traffic control, much of the car-, ship- and aircraft-building industries, most of the steel factories, British Airways, oil companies, Cable & Wireless, the aircraft engine makers Rolls-Royce, the arms makers Royal Ordnance, the ferry company Sealink, the Trustee Savings Bank, Girobank, technology companies Ferranti and Inmos, medical technology firm Amersham International and many others.

In the past 35 years, this commonly owned economy, this people's portion of the island, has to a greater or lesser degree become private. Millions of council houses have been sold to their owners or to housing associations. Most roads and streets are still under public control, but privatisation has reached deep into the NHS, state schools, the prison service and the military. The remainder was privatised by Thatcher and her successors. By the time she left office, she boasted, 60% of the old state industries had private owners – and that was before the railways and electricity system went under the hammer.

The original background to Thatcher's privatisation revolution was stagflation, a sense of national failure, and a widespread feeling, spreading even to some regular Labour voters, that the unions had become too powerful, and were holding the country back. Labour, and Thatcher's centrist predecessors among the Conservatives, had tried to control inflation administratively, through various deals with unions and employers to hold down wages and prices; Labour had, under pressure from the IMF, cut spending. But Thatcher and her inner circle planned to go further, horrifying moderates in their party with the radicalism of their intentions.

The late Alan Walters, her chief economic adviser, believed a key source of inflation and the weak economy was the amount of taxpayers' money being poured into overmanned, old-fashioned, government-owned industry. Just as in the Soviet Union, he thought, Britain's state industries concealed their subsidy-sucking inefficiency through opaque, idiosyncratic accounting techniques that took little account of how much time and effort were required to do and make things, or what people actually wanted to buy, or how much they were prepared to pay for it. As long as the subsidies kept coming, neither managers nor workers had much incentive to come up with smarter working methods or accept new technology, because that would mean fewer jobs, which would mean less power for the bosses and a smaller union. Yes, Walters knew, his protégée would slash spending on steel and coal and power and all the rest, yes, hundreds of thousands of workers would be sacked, but that wasn't enough. As many state-owned companies as possible must be privatised – be divided up into shares and sold to the public. They'd no longer be subsidised; they'd have to borrow money like any private company, account meticulously to shareholders for every penny they spent or earned, and strive to make a profit. The bigger the profit, the more efficiently the firm would be doing its job, and the more management would be rewarded. Most importantly, they'd have to compete with other firms. If they fell behind their competitors, they'd risk bankruptcy. Managers would face incentives for success and penalties for failure. British industry would become more competitive internationally. It would serve citizens better. Government would save the taxpayer money. The sacked workers would get redundancy payments; they'd go off and start businesses, or find other, more useful jobs once the economy was working properly. Everyone would win, except the lazy, and Arthur Scargill.

 Last-minute subscribers deliver their applications for British Gas shares at one of the receiving banks, National Westminster, in the City of London. Photograph: PA/PA Archive/Press Association Images

Millions did buy shares. Most Britons, bemused by the process, assumed the main reason for privatisation was to raise cash for a desperate government. Harold Macmillan, who before his death provided a snarky Wodehousian commentary from the wings on the work of the grocer's daughter, observed in an often paraphrased line: "The sale of assets is common with individuals and states when they run into financial difficulties. First, all the Georgian silver goes, and then all that nice furniture that used to be in the saloon. Then the Canalettos go."

Another leal privatiser, Nigel Lawson, a minister in the Thatcher government from the beginning almost to the end, dismissed the idea that the government cared about the price it was getting for selling off the family silver. Having many ordinary people owning shares, he writes in his memoirs, was the point. "The prime motives for privatisation were not Exchequer gain," he declares, "but an ideological belief in free markets and a wider distribution of private ownership of property."

Neither Walters nor Lawson, nor other allies like Keith Joseph, the ex-communist Alfred Sherman or Nicholas Ridley, would have been able to implement their ideas without Thatcher herself, her extraordinary sense of the way the political wind was blowing, her conviction of her own rectitude, and the stamina and persistence with which she was able to go on insisting on something until her opponents in government gave in. Hers was a different emphasis to Walters, who saw the curbing of "bloody-minded trades unions" as a useful side effect of privatisation. For Thatcher, privatisation, in the beginning at least, was simply one of many weapons to use in her battle against the unions, which was, in turn, a single episode in her war to exterminate socialism, to be fought in one unbroken front from Orgreave Colliery to Andrei Sakharov's place of exile in Gorky. Her great political inspiration, apart from her father, was the Austrian economist Friedrich Hayek's 1944 book, The Road to Serfdom, written in Cambridge during the war. Hayek was regarded as an able economist; he eventually won a Nobel prize for it. But The Road to Serfdom isn't an economics book. It's a book about society, the recent past and human nature that bears the same relation to sociology, history and psychology as Ayn Rand's Atlas Shrugged bears to literature. It is devoted to the idea that Winston Churchill later nodded to, catastrophically for him, in the 1945 election campaign, when he said Labour would have to fall back on "some form of Gestapo" to implement its welfare and nationalisation programme. Churchill was thrown out of office, and Labour won a huge majority.

The Road to Serfdom claims that socialism inevitably leads to communism, and that communism and Nazi-style fascism are one and the same. The tie that links Stalin's USSR and Hitler's Germany, in Hayek's view, is the centrally planned economy – as he portrays it, the attempt by a single central bureaucracy to direct all human life, to determine all human needs in advance and organise provision, limiting each to their rationed dole and their allotted task. Such a bureaucracy will no more tolerate dissent and deviation than the engineers tending a vast production line will accept a pebble jamming the gears. Confusingly, Hayek denies he is a pure libertarian, and declares the free market must have rules; he also says it is acceptable for government to "provide an extensive system of social services". Yet this is in contradiction to his main message, which is that there can be no mixture of state planning and free market competition. To him they are mutually exclusive. "By the time Hitler came to power, liberalism was dead in Germany," he writes. "And it was socialism that had killed it." Even to try to make socialism work, according to Hayek, is dangerous: "in the democracies the majority of people still believe that socialism and freedom can be combined. They do not realise that democratic socialism, the great utopia of the last few generations, is not only unachievable, but that to strive for it produces something utterly different – the very destruction of freedom itself."

 Author of "The Road to Serfdom" Friedrich Hayek, at the University of Chicago in 1960. Photograph: Bettmann/Corbis

Hayek was proven wrong. As in other western European countries, socialists came and went from power in Britain, introduced a welfare state and took control of large swathes of the economy without democracy and individual freedoms being threatened. The NHS was set up, council houses were built, social security was established, state education was expanded, coal, rail and steel nationalised, yet despite all the planning this required, millions of private businesses, small, medium and large, carried on merrily competing (or co-operating) with each other, flourishing or going to the wall as the market determined. Private doctors kept their clinics on Harley Street, young aristos still ruggered their way across the playing fields of Eton, the private shop windows of Harrods still blazed forth at Christmas time. Bankers and stockbrokers thronged the City, and the farmers owned their land. No one was forced by the government to live in a particular place or do a particular job. There was an argument to be made about how much tax people and businesses paid, and how much of that money government would have been better letting them choose for themselves how to spend. The argument was made, and will always be made; in the end, neither the Gestapo, nor the English Hitler, nor the English Politburo appeared, or looked like appearing.

Hayek's work, that of a frightened refugee in wartime, in the blackouts and shortages of a besieged island, had been superseded by the 1970s. A better framework for understanding the Britain of the time would have been the American Daniel Bell's masterful introduction to his 1976 book The Cultural Contradictions of Capitalism, where, though he spoke in general terms, he seemed to capture the actual contemporary problems of the UK: "A system of state capitalism could easily be transformed into a corporate state … a cumbersome, bureaucratic monstrosity, wrenched in all directions by the clamour for subsidies and entitlements by various corporate and communal groups, yet gorging itself on increased governmental appropriations to become a Leviathan in its own right." Thatcher, however, never stopped seeing the world through a Hayekian prism. After she defeated the attempt by Britain's coal miners to stave off mass redundancies and pit closures by downing tools, she wrote: "What the strike's defeat established was that Britain could not be made ungovernable by the Fascist Left."

About 10 years ago, I began to investigate what happened after the early Thatcherite zeal took effect. I was sceptical when I began my inquiries, but I was prepared to be convinced that privatisation in these half-dozen cases had been a success. I learned that it has not. Privatisation failed to turn Britain into a nation of small shareholders. Before Thatcher came to power, almost 40% of the shares in British companies were held by individuals. By 1981, it was less than 30%. By the time she died in 2013, it had slumped to under 12%. What is significant about this is not only that Thatcher and her chancellor Nigel Lawson's vision of a shareholding democracy failed to come to pass through privatisation, but that it undermines the justification for the way the companies were taken out of public ownership.

There's no doubt that since privatisation the old nationalised industries have sacked colossal numbers of workers and brought in new technology. If efficiency is doing the same job or better with fewer workers, many of the privatised firms are more efficient. But this simply suggests some or all of the nationalised industries should have been commercialised – that is, had their subsidies shrunk and been removed from direct government control, obliging them to borrow money at commercial rates and operate in a world of market prices without making a loss. Apart from the failed attempt to encourage wider share ownership, there was no obvious reason to privatise them by floating them on the stock market and selling them to shareholders. There are many forms of private ownership. The department store chain John Lewis, an unsubsidised commercial firm in a fiercely competitive market, is owned by its employees. The Nationwide Building Society, an unsubsidised commercial firm in a fiercely competitive market, is owned by its members. The Guardian Media Group, an unsubsidised commercial firm in a fiercely competitive market, is owned by a trust set up to support its journalistic values and protect it from hostile takeover. And so on. None of the many alternatives to stock market flotation were put up for discussion by either side: it was either shareholder capitalism or the nationalised status quo.

Privatisation failed to demonstrate the case made by the privatisers that private companies are always more competent than state-owned ones – that private bosses, chasing the carrot of bonuses and dodging the stick of bankruptcy, will always do better than their state-employed counterparts. Through euphemisms such as "wealth creation" and "enjoying the rewards of success" Thatcher and her allies have promoted the notion that greed on the part of a private executive elite is the chief and sufficient engine of prosperity for all. The result has been 35 years of denigration of the concept of duty and public service, as well as a squalid ideal of all work as something that shouldn't be cared about for its own sake, but only for the money it brings. The magic dust of the market was of little use to the bosses of the newly privatised Railtrack in the mid-1990s. They thought they could sack people with impunity – not just signalling and maintenance staff but expert engineers and researchers – and carry out a massive line-upgrade cheaply with the most advanced new technology. Unfortunately the people who could have told them that the new technology didn't exist were the people they had sacked. As a result, the company went bust in 2002, and had to be renationalised.

 A Royal Mail passes the Houses of Parliament behind it, in central London, September 12, 2013. Photograph: EDDIE KEOGH/Reuters

Privatisation failed to make firms compete or give customers more choice – said to be the canonical virtues of privatisation. Pretty hard, you would think, to privatise water companies, when they are all monopolies, with nobody to compete with, and can't offer customers a choice – neither the choice of which supplier to use nor the choice of whether to take a service or not. And yet the English water companies were privatised, and in such a way that customers have been overcharged ever since. The privatisers loved competition, but the actual privatised competitors hate it. The competitive vision of those who designed Britain's electricity privatisation – a rumbustious, referee-supervised free-for-all between sellers and makers of electricity old and new, large and small – has degenerated into an opaque oligopoly of a handful of giant players.

The impression grows, on reading Thatcher's autobiography, that she believed the transformational effect of privatisation was such as to turn executives into self-consciously moral, patriotic, civically minded entrepreneurs like her father; as if a monopoly on water supply for several million people were a local grocery shop in a small English town in the 1940s. Privatisation, she claimed, was "the greatest shift of ownership and power away from the state to individuals and their families in any country outside the former communist bloc". The reality is that the faceless state bureaucrats of the old electricity boards have been replaced by the faceless (and better paid) private bureaucrats of the electricity companies. Not only are the privatised utilities big, remote corporations; most of them are no longer British, and no longer owned by small shareholders. Indeed electricity and water privatisation could not have failed more absolutely to foster the emergence of world-beating, innovative British companies. Most of the electricity made and sold in England is now owned by dynamic, tech-savvy companies from western Europe, a region doomed, Thatcher thought, by creeping socialism. As a direct result of the way electricity was privatised, much of it has now been renationalised – but by France, not Britain. Of the nine big English water and sewerage firms, six have achieved the seemingly impossible feat of being privatised a second time, delisted from the stock market by east Asian conglomerates or by private equity consortia. Today much of England's water industry is, it is true, in the hands of individuals and their families, but they don't use English water; they are millions of former civil servants in Canada, Australia and the Netherlands, investing, unwittingly, through their pension funds. The National Health Service is a special case. It hasn't been privatised, and the political parties vie with each other to show that it's safest in their hands. Yet it has been commercialised and repeatedly reorganised, with competition introduced, in such a way as to create a kind of shadowing of an as-yet-unrealised private health insurance system. The story of the transformation of the NHS is part of the wider story of the inheritance of the Thatcher legacy by a Blairite Labour administration over-filled with politicians who struggled to separate their ambitions for Britain from their ambitions for their own and their families' ascent into the six-figure-income class. After their Sisyphean struggles with the Tories and the conservative socialists in their own party, New Labour in power yielded with all too apparent relief to the charms of the business world. It wasn't the creation of foundation trusts for hospitals – or academy schools, or support for housing associations – that was the mistake, rather a lack of awareness that without elaborate safeguards these structures might prove mere waypoints to the next set of privatisations.

 Young children at a Bristol health centre unsure of the benefits of sun-lamps and the special glasses that must be worn, The health centre was part of the newly introduced free National Health Service in 1948 Photograph: Popperfoto/Popperfoto/Getty Images

What the story of the latter years of the NHS shows is that the most powerful market force eating away at the core of the welfare state is not so much capitalism as consumer capitalism – the convergence of desires between the users of a public service and the private companies providing it when the companies use the skills of marketing to give users a sense of dissatisfaction and peer disadvantage. "If consumption represents the psychological competition for status," writes Daniel Bell, "then one can say that bourgeois society is the institutionalisation of envy." Hip replacement, a procedure invented within the NHS by John Charnley, began as a blessed relief from pain for which patients were, as Charnley said, pathetically grateful. It rapidly progressed to a rationed entitlement. It has now become a competitive market.

This points to a difficulty for anti-marketeers. Since 1945, even if privatisation had never happened, socialism would have struggled with the move from a world of unsatisfied needs to a more complex world of unsatisfied wants.

The selling off of Britain's municipal housing without replacing it was supposed to be a triumphant coming together of the individual and free market principles. It actually ended up as one of the most glaring examples of market failure in postwar history. It wasn't like the other privatisations; its justification as anything other than an electoral bribe to its relatively well-off beneficiaries always rang false. It certainly did to Thatcher in the beginning. She was, she wrote, "wary of alienating the already hard-pressed families who had scrimped to buy a house on one of the new private estates at the market price … They would, I feared, strongly object to council house tenants who had made none of their sacrifices suddenly receiving what was in effect a large capital sum from the Government".

 British Prime Minister Margaret Thatcher hands over the deeds to the council house belonging to the King family of Milton Keynes, Buckinghamshire, 25th September 1979. Photograph: Keystone/Getty Images

In the end, she came round, and made the policy her own. But the gap where the economic rationale for privatising council houses should be becomes a window through which it becomes possible to see beyond the individual privatisations to the meta-privatisation, and its one indisputable success: that it put more money into the hands of a small number of the very wealthiest people, at the expense of the elderly, the sick, the jobless and the working poor.

What do we think we know about taxes since the Thatcher revolution? Government spending has been cut, we know that. Income tax is lower than it used to be, we know that. And we might remember that the one time Thatcher tried to change the principle of progressive taxation, where the amount of tax you pay depends on your income, to a flat fee, where everyone pays the same – when the Conservatives tried to introduce the infamous "poll tax" on council services – it was the catalyst for her downfall. Low tax was her mantra. Her core political message was this, in her own words: "I believe the person who is prepared to work hardest should get the greatest rewards and keep them after tax. That we should back the workers and not the shirkers: that it is not only permissible but praiseworthy to want to benefit your own family by your own efforts."

What we think we know is wrong. Yes, government spending was cut, and it is being cut again, by Thatcher's coalition successors. When the Conservatives came to power in 1979 the top rate of tax was 83%, the basic rate 33. The top rate is now 45% and the basic rate 20%. The message seems clear enough. The Conservatives cut public spending and cut taxes, they kept their promises to working people, and Labour went along with it. But that is not all that happened. At the same time as they cut income tax and public spending, the first Thatcher administration hiked the sales tax, VAT – a flat-rate tax far more remorselessly regressive than the poll tax. When they came to power, the main VAT rate was 8%. It is now 20%. And the poorer you are, the harder VAT hits you. A study by the Office of National Statistics in 2010 showed that, for the richest fifth of the population, VAT added an extra 4 per cent to their tax bill. But the poorest fifth, often thought by the better off to pay no tax at all, actually pay 8.7 per cent of their income to the Treasury in VAT. When the Coalition came to power that year, its first chancellor George Osborne raised VAT by 14 per cent.

Where privatisation comes into this is that VAT isn't the only flat-rate tax on the poor. There are others, and they are onerous; they just aren't called taxes, though they should be – private taxes. One of the other ways the Thatcherites tried to balance the books in their first budgets was by hiking the price of gas, electricity and council rents, then all still under state control. After privatisation, above-inflation price rises have continued, in the private sector. A tax is generally thought of as something that only a government can levy, but this is a semantic distortion that favours the free market belief system. If a payment to an authority, public or private, is compulsory, it's a tax. We can't do without electricity; the electricity bill is an electricity tax. We can't do without water; the water bill is a water tax. Some people can get by without railways, and some can't; they pay the rail tax. Students pay the university tax. The meta-privatisation is the privatisation of the tax system itself; even, it could be said, the privatisation of us, the former citizens of Britain. By packaging British citizens up and selling them, sector by sector, to investors, the government makes it possible to keep traditional taxes low or even cut them. By moving from a system where public services are supported by progressive general taxation to a system where they are supported exclusively by the flat fees people pay to use them, they move from a system where the rich are obliged to help the poor to a system where the less well-off enable services that the rich get for what is, to them, a trifling sum. The commodity that makes water and power cables and airports valuable to an investor, foreign or otherwise, is the people who have no choice but to use them. We have no choice but to pay the price the toll-keepers charge. We are a human revenue stream; we are being made tenants in our own land, defined by the string of private fees we pay to exist here.

It is not racism that makes the foreign identity of some of the owners of our privatised infrastructure objectionable. It's the selling of taxation powers to foreign governments over whom we have even less democratic control than our own. It is the hypocrisy, in particular, of a party that claims to loathe nothing more than communism and totalitarianism obliging Londoners to pay a tithe to the Chinese government just for turning on the tap.