Showing posts with label neutral. Show all posts
Showing posts with label neutral. Show all posts
Monday, 10 June 2024
Thursday, 18 March 2021
Wednesday, 31 January 2018
Numbers aren't neutral
A S Paneerselvan in The Hindu
Analysing data without providing sufficient context is dangerous
An inherent challenge in journalism is to meet deadlines without compromising on quality, while sticking to the word limit. However, brevity takes a toll when it comes to reporting on surveys, indexes, and big data. Let me examine three sets of stories which were based on surveys and carried prominently by this newspaper, to understand the limits of presenting data without providing comprehensive context.
Three reports
The Annual Status of Education Report (ASER), Oxfam’s report titled ‘Reward Work, Not Wealth’, and the World Bank’s ease of doing business (EoDB) rankings have been widely reported, commented on, and editorialised. In most cases, the numbers and rankings were presented as neutral evaluations; they were not seen as data originating from institutions that have political underpinnings. Data become meaningful only when the methodology of data collection is spelt out in clear terms.
Every time I read surveys, indexes, and big data, I look for at least three basic parameters to understand the numbers: the sample size, the sample questionnaire, and the methodology. The sample size used indicates the robustness of the study, the questionnaire reveals whether there are leading questions, and the methodology reveals the rigour in the study. As a reporter, there were instances where I failed to mention these details in my resolve to stick to the word limit. Those were my mistakes.
The ASER study covering specific districts in States is about children’s schooling status. It attempts to measure children’s abilities with regard to basic reading and writing. It is a significant study as it gives us an insight into some of the problems with our educational system. However, we must be aware of the fact that these figures are restricted only to the districts in which the survey was conducted. It cannot be extrapolated as a State-wide sample, nor is it fair to rank States based on how specific districts fare in the study. A news item, “Report highlights India’s digital divide” (Jan. 19, 2018), conflated these figures.
For instance, the district surveyed in Kerala was Ernakulam, which is an urban district; in West Bengal it was South 24 Parganas, a complex district that stretches from metropolitan Kolkata to remote villages at the mouth of the Bay of Bengal. How can we compare these two districts with Odisha’s Khordha, Jharkhand’s Purbi Singhbhum and Bihar’s Muzaffarpur? It could be irresistible for a reporter, who accessed the data, to paint a larger picture based on these specific numbers. But we may not learn anything when we compare oranges and apples.
Questionable methodology
Oxfam, in the ‘Reward Work, Not Wealth’ report, used a methodology that has been questioned by many economists. Inequality is calculated on the basis of “net assets”. The economists point out that in this method, the poorest are not those living with very little resources, but young professionals who own no assets and with a high educational loan. Inequality is the elephant in the room which we cannot ignore. But Oxfam’s figures seem to mimic the huge notional loss figures put out by the Comptroller and Auditor General of India. Readers should know that Oxfam’s study has drawn its figures from disparate sources such as the Global Wealth Report by Credit Suisse, the Forbes’ billionaires list, adjusting last year’s figure using the average annual U.S. Consumer Price Index inflation rate from the U.S. Bureau of Labour Statistics, the World Bank’s household survey data, and an online survey in 10 countries.
When the World Bank announced the EoDB index last year, there was euphoria in India. However, this newspaper’s editorial “Moving up” (Nov. 2, 2017), which looked at India’s surge in the latest World Bank ranking from the 130th position to the 100th in a year, cautioned and asked the government, which has great orators in its ranks, to be a better listener. In hindsight, this position was vindicated when the World Bank’s chief economist, Paul Romer, said that he could no longer defend the integrity of changes made to the methodology and that the Bank would recalculate the national rankings of business competitiveness going back to at least four years. Readers would have appreciated the FAQ section (“Recalculating ease of doing business”, Jan. 25) that explained this controversy in some detail, had it looked at India’s ranking using the old methodology.
Analysing data without providing sufficient context is dangerous
An inherent challenge in journalism is to meet deadlines without compromising on quality, while sticking to the word limit. However, brevity takes a toll when it comes to reporting on surveys, indexes, and big data. Let me examine three sets of stories which were based on surveys and carried prominently by this newspaper, to understand the limits of presenting data without providing comprehensive context.
Three reports
The Annual Status of Education Report (ASER), Oxfam’s report titled ‘Reward Work, Not Wealth’, and the World Bank’s ease of doing business (EoDB) rankings have been widely reported, commented on, and editorialised. In most cases, the numbers and rankings were presented as neutral evaluations; they were not seen as data originating from institutions that have political underpinnings. Data become meaningful only when the methodology of data collection is spelt out in clear terms.
Every time I read surveys, indexes, and big data, I look for at least three basic parameters to understand the numbers: the sample size, the sample questionnaire, and the methodology. The sample size used indicates the robustness of the study, the questionnaire reveals whether there are leading questions, and the methodology reveals the rigour in the study. As a reporter, there were instances where I failed to mention these details in my resolve to stick to the word limit. Those were my mistakes.
The ASER study covering specific districts in States is about children’s schooling status. It attempts to measure children’s abilities with regard to basic reading and writing. It is a significant study as it gives us an insight into some of the problems with our educational system. However, we must be aware of the fact that these figures are restricted only to the districts in which the survey was conducted. It cannot be extrapolated as a State-wide sample, nor is it fair to rank States based on how specific districts fare in the study. A news item, “Report highlights India’s digital divide” (Jan. 19, 2018), conflated these figures.
For instance, the district surveyed in Kerala was Ernakulam, which is an urban district; in West Bengal it was South 24 Parganas, a complex district that stretches from metropolitan Kolkata to remote villages at the mouth of the Bay of Bengal. How can we compare these two districts with Odisha’s Khordha, Jharkhand’s Purbi Singhbhum and Bihar’s Muzaffarpur? It could be irresistible for a reporter, who accessed the data, to paint a larger picture based on these specific numbers. But we may not learn anything when we compare oranges and apples.
Questionable methodology
Oxfam, in the ‘Reward Work, Not Wealth’ report, used a methodology that has been questioned by many economists. Inequality is calculated on the basis of “net assets”. The economists point out that in this method, the poorest are not those living with very little resources, but young professionals who own no assets and with a high educational loan. Inequality is the elephant in the room which we cannot ignore. But Oxfam’s figures seem to mimic the huge notional loss figures put out by the Comptroller and Auditor General of India. Readers should know that Oxfam’s study has drawn its figures from disparate sources such as the Global Wealth Report by Credit Suisse, the Forbes’ billionaires list, adjusting last year’s figure using the average annual U.S. Consumer Price Index inflation rate from the U.S. Bureau of Labour Statistics, the World Bank’s household survey data, and an online survey in 10 countries.
When the World Bank announced the EoDB index last year, there was euphoria in India. However, this newspaper’s editorial “Moving up” (Nov. 2, 2017), which looked at India’s surge in the latest World Bank ranking from the 130th position to the 100th in a year, cautioned and asked the government, which has great orators in its ranks, to be a better listener. In hindsight, this position was vindicated when the World Bank’s chief economist, Paul Romer, said that he could no longer defend the integrity of changes made to the methodology and that the Bank would recalculate the national rankings of business competitiveness going back to at least four years. Readers would have appreciated the FAQ section (“Recalculating ease of doing business”, Jan. 25) that explained this controversy in some detail, had it looked at India’s ranking using the old methodology.
Wednesday, 24 July 2013
India's greatest cricket series - Recalling India's collective vow of silence
Dave Richardson, the ICC chief executive, has called for the debate on neutral umpires to be reopened. It is a logical step too, since the nations that produce the best officials are unfairly deprived of the highest standards in officiating.
Umpires, their decisions, the DRS, and general human competence in the face of technology - all have come under the scanner during the ongoing Ashes series.
Ah, the Ashes! The fourth sequel to the greatest series ever - a title that is vehemently contested in India.
The greatest series ever? Neutral umpires? Combine the two and it serves as a natural trigger to take our minds back to 2001, a year before the Elite Panel of ICC umpires was appointed.
It seems a good time then to - if sheepishly rather than fondly, for reasons that will become clear soon - reminisce about the actual greatest series everwhen free men stood against the immortals and, unlike in the Battle of the Hot Gates, miraculously won the combat of the dust bowls.
India isn't so much the land of snake charmers as it is the land of unrivalled cricket fanatics. Fanaticism by definition leads to voluntary blindness and mutism. And from late afternoon onwards on the first day of the famousLaxman-Dravid Test match at Eden Gardens, the symptoms manifested themselves across the nation.
Harbhajan Singh had just become the first Indian to bag a Test match hat-trick, in circumstances so dubious that had Dean Jones been commentating, he'd still be muttering about the injustice in his sleep.
But, fortunately for us, we had the honour of being enthralled by the late Tony Greig, an Englishman whose brand of commentary every Indian could relate to: full of infectious enthusiasm that often came in the way of the facts. Thus, a cricket-mad nation perpetually charged up on adrenaline was further enthused by Tony's awe-inspiring words, and hope of immediate retrospection was lost.
Ricky Ponting was caught plumb in front. Adam Gilchrist smashed a ball that pitched miles (cricket metric) outside leg stump into his pads, but was given out lbw. The swashbuckling wicketkeeper, who had bludgeoned the Indian bowling en route to a match-winning ton at faster than a run a ball in the previous Test, left with a rueful smile.
And finally, Shane Warne was adjudged out caught, though replays were at the very least inconclusive, if not favouring the batsman's claim of a bump ball (though Sadagoppan Ramesh's unbelievable catch alone deserved that wicket, or so we convinced ourselves).
It was probably the most fortuitous hat-trick ever, and we were probably well aware of it at the time. But did we really care? Not a single bit.
An inherent detestation of anything Australian had clouded our senses. The visitors had won a record 16 Tests in a row. They had humiliated India in Mumbai, home of the nation's favourite son. Mark Waugh's paltry spin had made a mockery of batsmen who were born to play spin. Matthew Hayden and Gilchrist's combined onslaught had made a mockery of turning pitches. Ajit Agarkar had made a mockery of himself. Again.
But the tipping point was when Michael Slater - upset at his appeal for a catch being overturned - got in the face of Rahul Dravid, a cricketer for whom Indian mothers would be prepared to go to war, with rolling pins for swords.
And so, there was little remorse about the way India were thrown a lifeline.
The next morning, in offices, in schools, at bus stands, in shared cabs, in autorickshaws, on the footpaths, on news channels, in newspapers, the discussions would revolve around those five minutes of earth-shattering cricket the previous day.
Those who did dare point out India's extremely good fortune were shushed and banished. The implied embargo was added alongside the traditional laws of our cricket culture, which include: No remark can be accepted against the actions of Sachin Tendulkar, even if he unnecessarily paddle-sweeps his way back to the pavilion. And a Pakistani cricketer's communication skills ought to be laughed at irrespective of their educational backgrounds, and independent of how mediocre our own players' English-speaking skills are.
You were to muse over the Test match only in a 2:98 ratio, where 2% of the time is to be spent acknowledging the timing of the hat-trick and 98% of it admiring the lengthy batting partnership that followed two days later. If you were to watch the feat again, it ought to be done in 30 seconds and without replays. You were tacitly prohibited from indulging yourself in the finer details.
Six on-field umpires were used in the three Test matches. The three neutral umpires were experienced ones: David Shepherd, Peter Willey and Rudi Koertzen. All three were selected to be on the Elite Panel a year later, though Willey chose not to take up the option.
The three home umpires are worth looking at. S Venkataraghavan, who was later chosen as the only Indian umpire on the Elite Panel, stood in his 43rd Test in Mumbai. Unsurprisingly, the match went without a glitch.
SK Bansal, who stood in only his sixth Test (and incidentally his last) in Kolkata, and AV Jayaprakash, who stood in his ninth in Chennai, were the other two home umpires. Bansal, in particular, presided over a host of controversial decisions, which included the series-changing hat-trick calls and some key rulings that triggered Australia's second-innings collapse.
The speed at which his decisions were made - as Glenn McGrath found out when batting bravely to save the match in the final hour - suggested they were more impulsive than considered. He was an Indian after all, and only the most hard-hearted of professionals wouldn't have been affected by the screams of 100,000 people.
Such key moments, when the series was completely turned on its head, had more than just divine intervention about them. They also had a very human helping hand - or rather, finger. But a nation awash with patriotism and a renewed sense of pride chose to overlook factors that could possibly dampen their most famous victory.
The greatest series ever? Maybe. One of the greatest endorsements of the need for neutral and qualified umpires? Definitely.
Subscribe to:
Posts (Atom)