Solving Equation of a Hit Film Script, With Data

By BROOKS BARNES

LOS ANGELES — Forget zombies. The data crunchers are invading Hollywood.

The same kind of numbers analysis that has reshaped areas like politics and online marketing is increasingly being used by the entertainment industry.

Netflix tells customers what to rent based on algorithms that analyze previous selections, Pandora does the same with music, and studios have started using Facebook “likes” and online trailer views to mold advertising and even films.

Now, the slicing and dicing is seeping into one of the last corners of Hollywood where creativity and old-fashioned instinct still hold sway: the screenplay.

A chain-smoking former statistics professor named Vinny Bruzzese — “the reigning mad scientist of Hollywood,” in the words of one studio customer — has started to aggressively pitch a service he calls script evaluation. For as much as $20,000 per script, Mr. Bruzzese and a team of analysts compare the story structure and genre of a draft script with those of released movies, looking for clues to box-office success. His company, Worldwide Motion Picture Group, also digs into an extensive database of focus group results for similar films and surveys 1,500 potential moviegoers. What do you like? What should be changed?

“Demons in horror movies can target people or be summoned,” Mr. Bruzzese said in a gravelly voice, by way of example. “If it’s a targeting demon, you are likely to have much higher opening-weekend sales than if it’s summoned. So get rid of that Ouija Board scene.”

Bowling scenes tend to pop up in films that fizzle, Mr. Bruzzese, 39, continued. Therefore it is statistically unwise to include one in your script. “A cursed superhero never sells as well as a guardian superhero,” one like Superman who acts as a protector, he added.

His recommendations, delivered in a 20- to 30-page report, might range from minor tightening to substantial rewrites: more people would relate to this character if she had a sympathetic sidekick, for instance.

Script “doctors,” as Hollywood refers to writing consultants, have long worked quietly on movie assembly lines. But many top screenwriters — the kind who attain exalted status in the industry, even if they remain largely unknown to the multiplex masses — reject Mr. Bruzzese’s statistical intrusion into their craft.

“This is my worst nightmare” said Ol Parker, a writer whose film credits include “The Best Exotic Marigold Hotel.” “It’s the enemy of creativity, nothing more than an attempt to mimic that which has worked before. It can only result in an increasingly bland homogenization, a pell-mell rush for the middle of the road.”

Mr. Parker drew a breath. “Look, I’d take a suggestion from my grandmother if I thought it would improve a film I was writing,” he said. “But this feels like the studio would listen to my grandmother before me, and that is terrifying.”

But a lot of producers, studio executives and major film financiers disagree. Already they have quietly hired Mr. Bruzzese’s company to analyze about 100 scripts, including an early treatment for “Oz the Great and Powerful,” which has taken in $484.8 million worldwide.

Mr. Bruzzese (pronounced brew-ZEZ-ee), who is one of a very few if not the only entrepreneur to use this form of script analysis, is plotting to take it to Broadway and television now that he has traction in movies.

“It takes a lot of the risk out of what I do,” said Scott Steindorff, a producer who used Mr. Bruzzese to evaluate the script for “The Lincoln Lawyer,” a hit 2011 crime drama. “Everyone is going to be doing this soon.” Mr. Steindorff added, “The only people who are resistant are the writers: ‘I’m making art, I can’t possibly do this.’ ”

Audience research has been known to save a movie, but it has also famously missed the mark. Opinion surveys — “idiot cards,” as some unimpressed directors call them — indicated that “Fight Club” would be the flop of the century. It took in more than $100 million worldwide.

But, as the stakes of making movies become ever higher, Hollywood leans ever harder on research to minimize guesswork. Moreover, studios have trimmed spending on internal script development. Mr. Bruzzese is also pitching script analysis to studios as a duck-and-cover technique — for “when the inevitable argument of ‘I am not going to take the blame if this movie doesn’t work’ comes up,” his Web site says.

Mr. Bruzzese taught statistics at the State University of New York at Stony Brook on Long Island before moving into movie research about a decade ago, motivated by a desire for more money and a childhood love of movies.

He acknowledged that many writers are “skittish” about his service. But he countered that it is not as threatening as it may sound.

“This is just advice, and you can use all of it, some of it or none of it,” he said.

But ignore it at your peril, according to one production executive. Motion Picture Group, of Culver City, Calif., analyzed the script for “Abraham Lincoln: Vampire Hunter,” said the executive, who worked on the film, but the production companies that supplied it to 20th Century Fox did not heed all of the advice. The movie flopped. Mr. Bruzzese declined to comment.

Mr. Bruzzese emphasized that his script analysis is not done by machines. His reports rely on statistics and survey results, but before evaluating a script he meets with the writer or writers to “hear and understand the creative vision, so our analysis can be contextualized,” he said.

But he is also unapologetic about his focus on financial outcomes. “I understand that writing is an art, and I deeply respect that,” he said. “But the earlier you get in with testing and research, the more successful movies you will make.”

The service actually gives writers more control over their work, said Mark Gill, president of Millennium Films and a client. In traditional testing, the kind done when a film is almost complete, the writer is typically no longer involved. With script testing, the writer can still control changes.

One Oscar-winning writer who, at the insistence of a producer, had a script analyzed by Mr. Bruzzese said his initial worries proved unfounded.

“It was a complete shock, the best notes on a draft that I have ever received,” said the writer, who spoke on the condition of anonymity, citing his reputation.

Script analysis is new enough to remain a bit of a Hollywood taboo. Major film financiers and advisers like Houlihan Lokey confirmed that they had used the service, but declined to speak on the record about it. The six major Hollywood movie studios declined to comment.

But doors are opening for Mr. Bruzzese nonetheless, in part because he is such a character. For instance, he bills himself as a distant relative of Einstein’s, a claim that is unverifiable but never fails to impress studio executives.

Mr. Bruzzese, a movie enthusiast with a seemingly encyclopedic memory of screenplays, also speaks bluntly, a rarity in Hollywood.

“All screenwriters think their babies are beautiful,” he said, taking a chug of Diet Dr Pepper followed by a gulp of Diet Coke and a drag on a Camel. “I’m here to tell it like it is: Some babies are ugly.”

The maths formula linked to the financial crash

Black-Scholes: The maths formula linked to the financial crash

By Tim Harford BBC Radio 4, More or Less
 

It's not every day that someone writes down an equation that ends up changing the world. But it does happen sometimes, and the world doesn't always change for the better. It has been argued that one formula known as Black-Scholes, along with its descendants, helped to blow up the financial world.

Black-Scholes was first written down in the early 1970s but its story starts earlier than that, in the Dojima Rice Exchange in 17th Century Japan where futures contracts were written for rice traders. A simple futures contract says that I will agree to buy rice from you in one year's time, at a price that we agree right now.

By the 20th Century the Chicago Board of Trade was providing a marketplace for traders to deal not only in futures but in options contracts. An example of an option is a contract where we agree that I can buy rice from you at any time over the next year, at a price that we agree right now - but I don't have to if I don't want to.

You can imagine why this kind of contract might be useful. If I am running a big chain of hamburger restaurants, but I don't know how much beef I'll need to buy next year, and I am nervous that the price of beef might rise, well - all I need is to buy some options on beef.

But then that leads to a very ticklish problem. How much should I be paying for those beef options? What are they worth? And that's where this world-changing equation, the Black-Scholes formula, can help.

"The problem it's trying to solve is to define the value of the right, but not the obligation, to buy a particular asset at a specified price, within or at the end of a specified time period," says Professor Myron Scholes, professor of finance at the Stanford University Graduate School of Business and - of course - co-inventor of the Black-Scholes formula.

The young Scholes was fascinated by finance. As a teenager, he persuaded his mother to set up an account so that he could trade on the stock market. One of the amazing things about Scholes is that throughout his time as an undergraduate and then a doctoral student, he was half-blind. And so, he says, he got very good at listening and at thinking.

When he was 26, an operation largely restored his sight. The next year, he became an assistant professor at MIT, and it was there that he stumbled upon the option-pricing puzzle.

One part of the puzzle was this question of risk: the value of an option to buy beef at a price of - say - $2 (£1.23) a kilogram presumably depends on what the price of beef is, and how the price of beef is moving around.

But the connection between the price of beef and the value of the beef option doesn't vary in a straightforward way - it depends how likely the option is to actually be used. That in turn depends on the option price and the beef price. All the variables seem to be tangled up in an impenetrable way.
Scholes worked on the problem with his colleague, Fischer Black, and figured out that if I own just the right portfolio of beef, plus options to buy and sell beef, I have a delicious and totally risk-free portfolio. Since I already know the price of beef and the price of risk-free assets, by looking at the difference between them I can work out the price of these beef options. That's the basic idea. The details are hugely complicated.

"It might have taken us a year, a year and a half to be able to solve and get the simple Black-Scholes formula," says Scholes. "But we had the actual underlying dynamics way before."

The Black-Scholes method turned out to be a way not only to calculate value of options but all kinds of other financial assets. "We were like kids in a candy story in the sense that we described options everywhere, options were embedded in everything that we did in life," says Scholes.

But Black and Scholes weren't the only kids in the candy store, says Ian Stewart, whose book argues that Black-Scholes was a dangerous invention.

"What the equation did was give everyone the confidence to trade options and very quickly, much more complicated financial options known as derivatives," he says.

Scholes thought his equation would be useful. He didn't expect it to transform the face of finance. But it quickly became obvious that it would.

"About the time we had published this article, that's 1973, simultaneously or approximately a month thereafter, the Chicago Board Options Exchange started to trade call options on 16 stocks," he recalls.
Scholes had just moved to the University of Chicago. He and his colleagues had already been teaching the Black-Scholes formula and methodology to students for several years.

"There were many young traders who either had taken courses at MIT or Chicago in using the option pricing technology. On the other hand, there was a group of traders who had only intuition and previous experience. And in a very short period of time, the intuitive players were essentially eliminated by the more systematic players who had this pricing technology."

That was just the beginning.

"By 2007 the trade in derivatives worldwide was one quadrillion (thousand million million) US dollars - this is 10 times the total production of goods on the planet over its entire history," says Stewart. "OK, we're talking about the totals in a two-way trade, people are buying and people are selling and you're adding it all up as if it doesn't cancel out, but it was a huge trade."

The Black-Scholes formula had passed the market test. But as banks and hedge funds relied more and more on their equations, they became more and more vulnerable to mistakes or over-simplifications in the mathematics.

"The equation is based on the idea that big movements are actually very, very rare. The problem is that real markets have these big changes much more often that this model predicts," says Stewart. "And the other problem is that everyone's following the same mathematical principles, so they're all going to get the same answer."

Now these were known problems. What was not clear was whether the problems were small enough to ignore, or well enough understood to fix. And then in the late 1990s, two remarkable things happened.

"The inventors got the Nobel Prize for Economics," says Stewart. "I would argue they thoroughly deserved to get it."

Fischer Black died young, in 1995. When in 1997 Scholes won the Nobel memorial prize, he shared it not with Black but with Robert Merton, another option-pricing expert.

Scholes' work had inspired a generation of mathematical wizards on Wall Street, and by this stage both he and Merton were players in the world of finance, as partners of a hedge fund called Long-Term Capital Management.

"The whole idea of this company was that it was going to base its trading on mathematical principles such as the Black-Scholes equation. And it actually was amazingly successful to begin with," says Stewart. "It was outperforming the traditional companies quite noticeably and everything looked great."

But it didn't end well. Long-Term Capital Management ran into, among other things, the Russian financial crisis. The firm lost $4bn (£2.5bn) in the course of six weeks. It was bailed out by a consortium of banks which had been assembled by the Federal Reserve. And - at the time - it was a very big story indeed. This was all happening in August and September of 1998, less than a year after Scholes had been awarded his Nobel prize.

Stewart says the lessons from Long-Term Capital Management were obvious. "It showed the danger of this kind of algorithmically-based trading if you don't keep an eye on some of the indicators that the more conventional people would use," he says. "They [Long-Term Capital Management] were committed, pretty much, to just ploughing ahead with the system they had. And it went wrong."

Scholes says that's not what happened at all. "It had nothing to do with equations and nothing to do with models," he says. "I was not running the firm, let me be very clear about that. There was not an ability to withstand the shock that occurred in the market in the summer and fall of late 1998. So it was just a matter of risk-taking. It wasn't a matter of modelling."

This is something people were still arguing about a decade later. Was the collapse of Long-Term Capital Management an indictment of mathematical approaches to finance or, as Scholes says, was it simply a case of traders taking too much risk against the better judgement of the mathematical experts?

Ten years after the Long-Term Capital Management bail-out, Lehman Brothers collapsed. And the debate over Black-Scholes and LTCM is now a broader debate over the role of mathematical equations in finance.

Ian Stewart claims that the Black-Scholes equation changed the world. Does he really believe that mathematics caused the financial crisis?

"It was abuse of their equation that caused trouble, and I don't think you can blame the inventors of an equation if somebody else comes along and uses it badly," he says.

"And it wasn't just that equation. It was a whole generation of other mathematical models and all sorts of other techniques that followed on its heels. But it was one of the major discoveries that opened the door to all this."

Black-Scholes changed the culture of Wall Street, from a place where people traded based on common sense, experience and intuition, to a place where the computer said yes or no.

But is it really fair to blame Black-Scholes for what followed it? "The Black-Scholes technology has very specific rules and requirements," says Scholes. "That technology attracted or caused investment banks to hire people who had quantitative or mathematical skills. I accept that. They then developed products or technologies of their own."

Not all of those subsequent technologies, says Scholes, were good enough. "[Some] had assumptions that were wrong, or they used data incorrectly to calibrate their models, or people who used [the] models didn't know how to use them."

Scholes argues there is no going back. "The fundamental issue is that quantitative technologies in finance will survive, and will grow, and will continue to evolve over time," he says.

But for Ian Stewart, the story of Black-Scholes - and of Long-Term Capital Management - is a kind of morality tale. "It's very tempting to see the financial crisis and various things which led up to it as sort of the classic Greek tragedy of hubris begets nemesis," he says.

"You try to fly, you fly too close to the sun, the wax holding your wings on melts and you fall down to the ground. My personal view is that it's not just tempting to do that but there is actually a certain amount of truth in that way of thinking. I think the bankers' hubris did indeed beget nemesis. But the big problem is that it wasn't the bankers on whom the nemesis descended - it was the rest of us."

Additional reporting by Richard Knight

The mathematical equation that caused the banks to crash

 Ian Stewart in The Observer 21-02-12

It was the holy grail of investors. The Black-Scholes equation, brainchild of economists Fischer Black and Myron Scholes, provided a rational way to price a financial contract when it still had time to run. It was like buying or selling a bet on a horse, halfway through the race. It opened up a new world of ever more complex investments, blossoming into a gigantic global industry. But when the sub-prime mortgage market turned sour, the darling of the financial markets became the Black Hole equation, sucking money out of the universe in an unending stream.

Anyone who has followed the crisis will understand that the real economy of businesses and commodities is being upstaged by complicated financial instruments known as derivatives. These are not money or goods. They are investments in investments, bets about bets. Derivatives created a booming global economy, but they also led to turbulent markets, the credit crunch, the near collapse of the banking system and the economic slump. And it was the Black-Scholes equation that opened up the world of derivatives.

The equation itself wasn't the real problem. It was useful, it was precise, and its limitations were clearly stated. It provided an industry-standard method to assess the likely value of a financial derivative. So derivatives could be traded before they matured. The formula was fine if you used it sensibly and abandoned it when market conditions weren't appropriate. The trouble was its potential for abuse. It allowed derivatives to become commodities that could be traded in their own right. The financial sector called it the Midas Formula and saw it as a recipe for making everything turn to gold. But the markets forgot how the story of King Midas ended.

Black-Scholes underpinned massive economic growth. By 2007, the international financial system was trading derivatives valued at one quadrillion dollars per year. This is 10 times the total worth, adjusted for inflation, of all products made by the world's manufacturing industries over the last century. The downside was the invention of ever-more complex financial instruments whose value and risk were increasingly opaque. So companies hired mathematically talented analysts to develop similar formulas, telling them how much those new instruments were worth and how risky they were. Then, disastrously, they forgot to ask how reliable the answers would be if market conditions changed.

Black and Scholes invented their equation in 1973; Robert Merton supplied extra justification soon after. It applies to the simplest and oldest derivatives: options. There are two main kinds. A put option gives its buyer the right to sell a commodity at a specified time for an agreed price. A call option is similar, but it confers the right to buy instead of sell. The equation provides a systematic way to calculate the value of an option before it matures. Then the option can be sold at any time. The equation was so effective that it won Merton and Scholes the 1997 Nobel prize in economics. (Black had died by then, so he was ineligible.)

If everyone knows the correct value of a derivative and they all agree, how can anyone make money? The formula requires the user to estimate several numerical quantities. But the main way to make money on derivatives is to win your bet – to buy a derivative that can later be sold at a higher price, or matures with a higher value than predicted. The winners get their profit from the losers. In any given year, between 75% and 90% of all options traders lose money. The world's banks lost hundreds of billions when the sub-prime mortgage bubble burst. In the ensuing panic, taxpayers were forced to pick up the bill, but that was politics, not mathematical economics.

The Black-Scholes equation relates the recommended price of the option to four other quantities. Three can be measured directly: time, the price of the asset upon which the option is secured and the risk-free interest rate. This is the theoretical interest that could be earned by an investment with zero risk, such as government bonds. The fourth quantity is the volatility of the asset. This is a measure of how erratically its market value changes. The equation assumes that the asset's volatility remains the same for the lifetime of the option, which need not be correct. Volatility can be estimated by statistical analysis of price movements but it can't be measured in a precise, foolproof way, and estimates may not match reality.

The idea behind many financial models goes back to Louis Bachelier in 1900, who suggested that fluctuations of the stock market can be modelled by a random process known as Brownian motion. At each instant, the price of a stock either increases or decreases, and the model assumes fixed probabilities for these events. They may be equally likely, or one may be more probable than the other. It's like someone standing on a street and repeatedly tossing a coin to decide whether to move a small step forwards or backwards, so they zigzag back and forth erratically. Their position corresponds to the price of the stock, moving up or down at random. The most important statistical features of Brownian motion are its mean and its standard deviation. The mean is the short-term average price, which typically drifts in a specific direction, up or down depending on where the market thinks the stock is going. The standard deviation can be thought of as the average amount by which the price differs from the mean, calculated using a standard statistical formula. For stock prices this is called volatility, and it measures how erratically the price fluctuates. On a graph of price against time, volatility corresponds to how jagged the zigzag movements look.

Black-Scholes implements Bachelier's vision. It does not give the value of the option (the price at which it should be sold or bought) directly. It is what mathematicians call a partial differential equation, expressing the rate of change of the price in terms of the rates at which various other quantities are changing. Fortunately, the equation can be solved to provide a specific formula for the value of a put option, with a similar formula for call options.

The early success of Black-Scholes encouraged the financial sector to develop a host of related equations aimed at different financial instruments. Conventional banks could use these equations to justify loans and trades and assess the likely profits, always keeping an eye open for potential trouble. But less conventional businesses weren't so cautious. Soon, the banks followed them into increasingly speculative ventures.

Any mathematical model of reality relies on simplifications and assumptions. The Black-Scholes equation was based on arbitrage pricing theory, in which both drift and volatility are constant. This assumption is common in financial theory, but it is often false for real markets. The equation also assumes that there are no transaction costs, no limits on short-selling and that money can always be lent and borrowed at a known, fixed, risk-free interest rate. Again, reality is often very different.
When these assumptions are valid, risk is usually low, because large stock market fluctuations should be extremely rare. But on 19 October 1987, Black Monday, the world's stock markets lost more than 20% of their value within a few hours. An event this extreme is virtually impossible under the model's assumptions. In his bestseller The Black Swan, Nassim Nicholas Taleb, an expert in mathematical finance, calls extreme events of this kind black swans. In ancient times, all known swans were white and "black swan" was widely used in the same way we now refer to a flying pig. But in 1697, the Dutch explorer Willem de Vlamingh found masses of black swans on what became known as the Swan River in Australia. So the phrase now refers to an assumption that appears to be grounded in fact, but might at any moment turn out to be wildly mistaken.

Large fluctuations in the stock market are far more common than Brownian motion predicts. The reason is unrealistic assumptions – ignoring potential black swans. But usually the model performed very well, so as time passed and confidence grew, many bankers and traders forgot the model had limitations. They used the equation as a kind of talisman, a bit of mathematical magic to protect them against criticism if anything went wrong.

Banks, hedge funds, and other speculators were soon trading complicated derivatives such as credit default swaps – likened to insuring your neighbour's house against fire – in eye-watering quantities. They were priced and considered to be assets in their own right. That meant they could be used as security for other purchases. As everything got more complicated, the models used to assess value and risk deviated ever further from reality. Somewhere underneath it all was real property, and the markets assumed that property values would keep rising for ever, making these investments risk-free.
The Black-Scholes equation has its roots in mathematical physics, where quantities are infinitely divisible, time flows continuously and variables change smoothly. Such models may not be appropriate to the world of finance. Traditional mathematical economics doesn't always match reality, either, and when it fails, it fails badly. Physicists, mathematicians and economists are therefore looking for better models.

At the forefront of these efforts is complexity science, a new branch of mathematics that models the market as a collection of individuals interacting according to specified rules. These models reveal the damaging effects of the herd instinct: market traders copy other market traders. Virtually every financial crisis in the last century has been pushed over the edge by the herd instinct. It makes everything go belly-up at the same time. If engineers took that attitude, and one bridge in the world fell down, so would all the others.

By studying ecological systems, it can be shown that instability is common in economic models, mainly because of the poor design of the financial system. The facility to transfer billions at the click of a mouse may allow ever-quicker profits, but it also makes shocks propagate faster.

Was an equation to blame for the financial crash, then? Yes and no. Black-Scholes may have contributed to the crash, but only because it was abused. In any case, the equation was just one ingredient in a rich stew of financial irresponsibility, political ineptitude, perverse incentives and lax regulation.

Despite its supposed expertise, the financial sector performs no better than random guesswork. The stock market has spent 20 years going nowhere. The system is too complex to be run on error-strewn hunches and gut feelings, but current mathematical models don't represent reality adequately. The entire system is poorly understood and dangerously unstable. The world economy desperately needs a radical overhaul and that requires more mathematics, not less. It may be rocket science, but magic it's not.
Ian Stewart is emeritus professor of mathematics at the University of Warwick.

Ian Stewart's top 10 popular mathematics books

Ian Stewart is an Emeritus Professor of Mathematics at Warwick University and a Fellow of the Royal Society. He has written over 80 books, mainly popular mathematics, and has won three gold medals for his work on the public understanding of science. In collaboration with Terry Pratchett and Jack Cohen he wrote the Science of Discworld series. His new book, 17 Equations That Changed the World, is published by Profile.

1. Seventeen Equations that Changed the World
2. by Ian Stewart
3. 
4. Buy it from the Guardian bookshop

Search the Guardian bookshop

1. Tell us what you think: Star-rate and review this book

Buy 17 Equations That Changed the World from the Guardian bookshop
"'Popular mathematics' may sound like a contradiction in terms. That's what makes the genre so important: we have to change that perception. Mathematics is the Cinderella science: undervalued, underestimated, and misunderstood. Yet it has been one of the main driving forces behind human society for at least three millennia, it powers all of today's technology, and it underpins almost every aspect of our daily lives.
"It's not really surprising that few outside the subject appreciate it, though. School mathematics is so focused on getting the right answer and passing the exam that there is seldom an opportunity to find out what it's all for. The hard core of real mathematics is extremely difficult, and it takes six or seven years to train a research mathematician after they leave school. Popular mathematics provides an entry route for non-specialists. It allows them to appreciate where mathematics came from, who created it, what it's good for, and where it's going, without getting tangled up in the technicalities. It's like listening to music instead of composing it.
"There are many ways to make real mathematics accessible. Its history reveals the subject as a human activity and gives a feel for the broad flow of ideas over the centuries. Biographies of great mathematicians tell us what it's like to work at the frontiers of human knowledge. The great problems, the ones that hit the news media when they are finally solved after centuries of effort, are always fascinating. So are the unsolved ones and the latest hot research areas. The myriad applications of mathematics, from medicine to the iPad, are an almost inexhaustible source of inspiration."

1. The Man Who Knew Infinity by Robert Kanigel

The self-taught Indian genius Srinivasa Ramanujan had a flair for strange and beautiful formulas, so unusual that mathematicians are still coming to grips with their true meaning. He was born into a poor Brahmin family in 1887 and was pursuing original research in his teens. In 1912, he was brought to work at Cambridge. He died of malnutrition and other unknown causes in 1920, leaving a rich legacy that is still not fully understood. There has never been another mathematical life story like it: absolutely riveting.

2. Gödel, Escher, Bach by Douglas Hofstadter

One of the great cult books, a very original take on the logical paradoxes associated with self-reference, such as "this statement is false". Hofstadter combines the mathematical logic of Kurt Gödel, who proved that some questions in arithmetic can never be answered, with the etchings of Maurits Escher and the music of Bach. Frequent dramatic dialogues between Lewis Carroll's characters Achilles and the Tortoise motivate key topics in a highly original manner, along with their friend Crab who invents the tortoise-chomping record player. DNA and computers get extensive treatment too.

3. The Colossal Book of Mathematics by Martin Gardner

In his long-running Mathematical Games column in Scientific American, Gardner – a journalist with no mathematical training – created the field of recreational mathematics. On the surface his columns were about puzzles and games, but they all concealed mathematical principles, some simple, some surprisingly deep. He combined a playful and clear approach to his subject with a well-developed taste for what was mathematically significant. The book consists of numerous selections from his columns, classified according to the mathematical area involved. Learn how to make a hexaflexagon and why playing Brussels sprouts is a waste of time.

4. Euclid in the Rainforest by Joseph Mazur

A thoroughly readable account of the meaning of truth in mathematics, presented through a series of quirky adventures in the Greek Islands, the jungles around the Orinoco River, and elsewhere. Examines tricky concepts like infinity, topology, and probability through tall tales and anecdotes. Three different kinds of truth are examined: formal classical logic, the role of the infinite, and inference by plausible reasoning. The story of the student who believed nothing except his calculator is an object lesson for everyone who thinks mathematics is just 'sums'.

5. Four Colours Suffice by Robin Wilson

In 1852 Francis Guthrie, a young South African mathematician, was attempting to colour the counties in a map of England. Guthrie discovered that he needed only four different colours to ensure that any two adjacent counties had different colours. After some experimentation he convinced himself that the same goes for any map whatsoever. This is the remarkable story of how mathematicians eventually proved he was right, but only with the aid of computers, bringing into question the meaning of "proof". It contains enough detail to be satisfying, but remains accessible and informative throughout.

6. What is Mathematics Really? by Reuben Hersh

The classic text What is Mathematics? by Richard Courant and Herbert Robbins focused on the subject's nuts and bolts. It answered its title question by example. Hersh takes a more philosophical view, based on his experience as a professional mathematician. The common working philosophy of most mathematicians is a kind of vague Platonism: mathematical concepts have some sort of independent existence in some ideal world. Although this is what it feels like to insiders, Hersh argues that mathematics is a collective human construct – like money or the Supreme Court. However, it is a construct constrained by its own internal logic; it's not arbitrary. You choose the concepts that interest you, but you don't get to choose how they behave.

7. Magical Mathematics by Persi Diaconis and Ron Graham

Both authors are top-rank mathematicians with years of stage performances behind them, and their speciality is mathematical magic. They show how mathematics relates to juggling and reveal the secrets behind some amazing card tricks. Here's one. The magician mails a pack of cards to anyone, asking them to shuffle it and choose a card. Then he shuffles the cards again, and mails half of them to the magician—not saying whether the chosen card is included. By return mail, the magician names the selected card. No trickery: it all depends on the mathematics of shuffles.

8. Games of Life by Karl Sigmund

Biologists' understanding of many vital features of the living world, such as sex and survival, depends on the theory of evolution. One of the basic theoretical tools here is the mathematics of game theory, in which several players compete by choosing from a list of possible strategies. The children's game of rock-paper-scissors is a good example. The book illuminates such questions as how genes spread through a population and the evolution of cooperation, by finding the best strategies for games such as cat and mouse, the battle of the sexes, and the prisoner's dilemma. On the borderline between popular science and an academic text, but eminently readable without specialist knowledge.

9. Mathenauts: Tales of Mathematical Wonder edited by Rudy Rucker

A collection of 23 science fiction short stories, each of which centres on mathematics. Two are by Martin Gardner, and many of the great writers of SF are represented: Isaac Asimov, Gregory Benford, Larry Niven, Frederik Pohl. The high point is Norman Kagan's utterly hilarious "The Mathenauts", in which only mathematicians can travel through space, because space is mathematical – and, conversely, anything mathematical can be reality. An isomorphomechanism is essential equipment. Between them, these tales cover most of the undergraduate mathematics syllabus, though not in examinable form.

10. The Mathematical Principles of Natural Philosophy by Isaac Newton

There ought to be a great classic in this top 10, and there is none greater. I've put it last because it's not popularisation in the strict sense. However, it slips in because it communicated to the world one of the very greatest ideas of all time: Nature has laws, and they can be expressed in the language of mathematics. Using nothing more complicated than Euclid's geometry, Newton developed his laws of motion and gravity, applying them to the motion of the planets and strange wobbles in the position of the Moon. He famously said that he "stood on the shoulders of giants", and so he did, but this book set the scientific world alight. As John Maynard Keyes wrote, Newton was a transitional figure of immense stature: "the last of the magicians … the last wonderchild to whom the Magi could do sincere and appropriate homage." No mathematical book has had more impact.

• © 2012 Guardian News and Media Limited or its affiliated companies. All rights reserved.