'People will forgive you for being wrong, but they will never forgive you for being right - especially if events prove you right while proving them wrong.' Thomas Sowell
Search This Blog
Sunday, 5 June 2022
Monday, 30 May 2022
On Fact, Opinion and Belief
Annie Duke in 'Thinking in Bets'
What exactly is the difference between fact, opinion and belief?” .
Fact: (noun): A piece of information that can be backed up by evidence.
Belief: (noun): A state or habit of mind, in which trust or confidence is placed, in some person or thing. Something accepted or considered to be true.
Opinion: (noun): a view, judgement or appraisal formed in the mind about a particular matter.
The main difference here is that we can verify facts, but opinions and beliefs are not verifiable. Until relatively recently, most people would call facts things like numbers, dates, photographic accounts that we can all agree upon.
More recently, it has become commonplace to question even the most mundane objective sources of fact, like eyewitness accounts, and credible peer-reviewed science, but that is a topic for another day.
How we think we form our beliefs:
We hear something;
We think about it and vet it, determining whether it is true or false; only after that
We form our belief
Actually, we form our beliefs:
We hear something;
We believe it to be true;
Only sometimes, later, if we have the time or the inclination, we think about it and vet it, determining whether it is, in fact, true or false.
Psychology professor Daniel Gilbert, “People are credulous creatures who find it very easy to believe and very difficult to doubt”.
Our default is to believe that what we hear and read is true. Even when that information is clearly presented as being false, we are still likely to process it as true.
For example, some people believe that we use only 10% of our brains. If you hold that belief, did you ever research it for yourself?
People usually say it is something they heard but they have no idea where or from whom. Yet they are confident that this is true. That should be proof enough that the way we form beliefs is foolish. And, we actually use all parts of our brain.
Our beliefs drive the way we process information. We form beliefs without vetting/testing most of them and we even maintain them even after receiving clear, corrective information.
Once a belief is lodged, it becomes difficult to dislodge it from our thinking. It takes a life of its own, leading us to notice and seek out evidence confirming our belief. We rarely challenge the validity of confirming evidence and ignore or work hard to actively discredit information contradicting the belief. This irrational, circular information-processing pattern is called motivated reasoning.
Truth Seeking, the desire to know the truth regardless of whether the truth aligns with the beliefs we currently hold, is not naturally supported by the way we process information.
Instead of altering our beliefs to fit new information, we do the opposite, altering our interpretation of that information to fit our beliefs.
Fake news works because people who already hold beliefs consistent with the story generally won’t question the evidence. The potency of fake news is that it entrenches beliefs its intended audience already has, and then amplifies them. Social media is a playground for motivated reasoning. It provides the promise of access to a greater diversity of information sources and opinions than we’ve ever had available. Yet, we gravitate towards sources that confirm our beliefs, that agree with us. Every flavour is out there, but we tend to stick with our favourite.
Even when directly confronted with facts that disconfirm our beliefs, we don’t let facts get in the way of our opinions.
Being Smart Makes It Worse
The popular wisdom is that the smarter you are, the less susceptible you are to fake news or disinformation. After all, smart people are more likely to analyze and effectively evaluate where the information is coming from, right? Part of being ‘smart’ is being good at processing information, parsing the quality of an argument and the credibility of the source.
Surprisingly, being smart can actually make bias worse. The smarter you are, the better you are at constructing a narrative that supports your beliefs, rationalising and framing the data to fit your argument or point of view.
Unfortunately, this is just the way evolution built us. We are wired to protect our beliefs even when our goal is to truthseek. This is one of the instances where being smart and aware of our capacity for irrationality alone doesn’t help us refrain from biased reasoning. As with visual illusions, we can’t make our minds work differently than they do no matter how smart we are. Just as we can’t unsee an illusion, intellect or will power alone can’t make us resist motivated reasoning.
Wanna Bet?
Imagine taking part in a conversation with a friend about the movie Citizen Kane. BEst film of all time, introduced a bunch of new techniques by which directors could contribute to storytelling. “Obviously, it won the best picture Oscar,” you gush, as part of a list of superlatives the film unquestionably deserves.
Then your friend says,”Wanna bet?”
Suddenly, you’re not so sure. That challenge puts you on your heels, causing you to back off your declaration and question the belief that you just declared with such assurance.
When someone challenges us to bet on a belief, signalling their confidence that our belief is inaccurate in some way, ideally it triggers us to vet the belief, taking an inventory of the evidence that informed us.
How do I know this?
Where did I get this information?
Who did I get it from?
What is the quality of my sources?
How much do I trust them?
How up to date is my information?
How much information do I have that is relevant to the belief?
What other things like this have I been confident about that turned out not to be true?
What are the other plausible alternatives?
What do I know about the person challenging my belief?
What is their view of how credible my opinion is?
What do they know that I don’t know?
What is their level of expertise?
What am I missing?
Remember the order in which we form our beliefs:
We hear something;
We believe it to be true;
Only sometimes, later, if we have the time or the inclination, we think about it and vet it, determining whether it is, in fact, true or false.
“Wanna bet?” triggers us to engage in that third step that we only sometimes get to. Being asked if we are willing to bet money on it makes it much more likely that we will examine our information in a less biased way, be more honest with ourselves about how sure we are of our beliefs, and be more open to updating and calibrating our beliefs.
A lot of good can result from someone saying, “Wanna bet?” Offering a wager brings the risk out in the open, making explicit what is implicit (and frequently overlooked). The more we recognise that we are betting on our beliefs (with our happiness, attention, health, money time or some other limited resource), the more we are likely to temper our statements, getting closer to the truth as we acknowledge the risk inherent in what we believe.
Once we start doing this (at the risk of losing friends), we are more likely to recognise that there is always a degree of uncertainty, that we are generally less sure than we thought we were, that practically nothing is black and white 0% or 100%. And that's a pretty good philosophy for living.
Thursday, 26 May 2022
Monday, 23 May 2022
India’s once-vaunted statistical infrastructure is crumbling
The reasons are worryingly familiar writes The Economist

The modern Indian state has a proud statistical heritage. Soon after the country gained independence in 1947, the government resolved to achieve its development through comprehensive five-year plans. The strategy, though economically inadvisable, nonetheless required the creation of a robust data-gathering apparatus. In 1950 PC Mahalanobis, the leading light of Indian statistics, designed the National Sample Survey, which sent staff to the far corners of the vast country to jot down data regarding its mostly illiterate citizens. The survey’s complexity and scope seemed “beyond the bounds of possibility”, reckoned one American statistician.
Of late, however, admiration has been replaced by alarm. India’s statistical services are in a bad way. Across some measures, figures are simply not gathered; for others, the data are often dodgy, unrepresentative, untimely, or just wrong. The country’s tracking of covid-19 provides a grim example. As the pandemic raged across India, officials struggled to keep tabs on its toll. Officially, covid has claimed more than half a million lives in India; The Economist’s excess-deaths tracker puts the figure far higher, between 2m and 9.4m. India’s government has also hampered efforts to assess the pandemic’s global impact, refusing at first to share data with the World Health Organisation (who), and criticising its methods.
The preference for flattering but flawed figures is pervasive. In education, state governments regularly ignore data showing that Indian children are performing woefully in school and instead cite their own administrative numbers, which are often wrong. In Madhya Pradesh, a state in central India, an official assessment showed that all pupils had scored more than 60% in a maths test; an independent assessment revealed that none of them had. Similarly, in sanitation, the central government says that India is now free of open defecation, meaning that people both have access to a toilet and consistently use it. Anyone who takes a train out of Delhi at dawn and looks out of the window, however, might question the claim.
When it comes to poverty, arguably India’s biggest problem, timely figures are not available. Official estimates are based on a poverty line derived from consumption data in 2011-12, despite the fact that more recent but as yet unpublished numbers exist for 2017-18. By contrast, Indonesia calculates its poverty rate twice a year. India’s government explains its approach by pointing to discrepancies between recently gathered data and national accounts statistics—but many suspect the true reason is that newer data would probably show an increase in poverty.
In some cases, flawed data seem more a problem of methodology than malign intent. India’s gdp estimates, for instance, have been mired in controversy ever since the statistics ministry introduced a new series in 2015 (a change that was in the works before the current government entered office). Arvind Subramanian, a former government adviser, calculated that the new methodology overestimated average annual growth by as much as three to four percentage points between 2011-12 and 2016-17. Although current advisers insist that the official methodology is in line with global standards, other studies have also found problems with the calculations.
The erosion of India’s statistical infrastructure predates the current government, but seems to have grown worse in recent years. Narendra Modi, the prime minister, has previously bristled at technocratic expertise and number-crunching. (“Hard work is more powerful than Harvard,” he said in 2017.)
India’s data woes are also troubling for what they suggest about the ability of the state to provide the essential public services needed to foster long-run growth. The statistics ministry, short of staff and resources, is emblematic of the civil service. Data-gathering has become excessively centralised and over-politicised. A National Statistical Commission was set up in 2005 and tasked with fixing India’s data infrastructure. But its work has been complicated by turf wars and internal politics; it is widely considered toothless, including by former members.
Who’s counting
The situation is not hopeless, perhaps because of statisticians’ past efforts. According to the World Bank, the quality of Indian data is still in line with that of other developing countries, even after years of neglect. India’s new goods-and-services tax and digital-welfare infrastructure are yielding troves of data. Leading Indian statisticians argue that an empowered regulator could fix existing problems.
State governments and departments are also doing their bit. Telangana, a southern state, is investing in its own household surveys, for example. India’s rural-development ministry recently released a dataset covering 770,000 rural public facilities, such as schools and hospitals, inviting data whizzes to peruse the figures and suggest improvements. Civil society is also responding. During the pandemic, dozens of volunteers co-operated to produce granular, timely estimates of covid cases. New technologies could help gather data quickly and cheaply, over phones and tablets.
Yet in a modern economy there is no substitute for high-quality national data-gathering. The sunlight provided by accurate figures is often unwelcome for an increasingly autocratic government: transparency invites accountability. But neglect of the statistical services also leaves Indian policymakers flailing in the dark, unable to quickly spot and respond to brewing economic and social problems.