Scientific or Pseudo Knowledge? How Lancet's reputation was destroyed

The now retracted paper halted hydroxychloroquine trials. Studies like this determine how people live or die tomorrow writes James Heathers in The Guardian

 

‘At its best, peer review is a slow and careful evaluation of new research by appropriate experts. ... At its worst, it is merely window dressing that gives the unwarranted appearance of authority’. Photograph: George Frey/AFP/Getty Images


The Lancet is one of the oldest and most respected medical journals in the world. Recently, they published an article on Covid patients receiving hydroxychloroquine with a dire conclusion: the drug increases heartbeat irregularities and decreases hospital survival rates. This result was treated as authoritative, and major drug trials were immediately halted – because why treat anyone with an unsafe drug?

Now, that Lancet study has been retracted, withdrawn from the literature entirely, at the request of three of its authors who “can no longer vouch for the veracity of the primary data sources”. Given the seriousness of the topic and the consequences of the paper, this is one of the most consequential retractions in modern history.

---Also watch

---

It is natural to ask how this is possible. How did a paper of such consequence get discarded like a used tissue by some of its authors only days after publication? If the authors don’t trust it now, how did it get published in the first place?

The answer is quite simple. It happened because peer review, the formal process of reviewing scientific work before it is accepted for publication, is not designed to detect anomalous data. It makes no difference if the anomalies are due to inaccuracies, miscalculations, or outright fraud. This is not what peer review is for. While it is the internationally recognised badge of “settled science”, its value is far more complicated.

At its best, peer review is a slow and careful evaluation of new research by appropriate experts. It involves multiple rounds of revision that removes errors, strengthens analyses, and noticeably improves manuscripts.

At its worst, it is merely window dressing that gives the unwarranted appearance of authority, a cursory process which confers no real value, enforces orthodoxy, and overlooks both obvious analytical problems and outright fraud entirely.

Regardless of how any individual paper is reviewed – and the experience is usually somewhere between the above extremes – the sad truth is peer review in its entirety is struggling, and retractions like this drag its flaws into an incredibly bright spotlight.

The ballistics of this problem are well known. To start with, peer review is entirely unrewarded. The internal currency of science consists entirely of producing new papers, which form the cornerstone of your scientific reputation. There is no emphasis on reviewing the work of others. If you spend several days in a continuous back-and-forth technical exchange with authors, trying to improve their manuscript, adding new analyses, shoring up conclusions, no one will ever know your name. Neither are you paid. Peer review originally fitted under an amorphous idea of academic “service” – the tasks that scientists were supposed to perform as members of their community. This is a nice idea, but is almost invariably maintained by researchers with excellent job security. Some senior scientists are notorious for peer reviewing manuscripts rarely or even never – because it interferes with the task of producing more of their own research.

However, even if reliable volunteers for peer review can be found, it is increasingly clear that it is insufficient. The vast majority of peer-reviewed articles are never checked for any form of analytical consistency, nor can they be – journals do not require manuscripts to have accompanying data or analytical code and often will not help you obtain them from authors if you wish to see them. Authors usually have zero formal, moral, or legal requirements to share the data and analytical methods behind their experiments. Finally, if you locate a problem in a published paper and bring it to either of these parties, often the median response is no response at all – silence.

This is usually not because authors or editors are negligent or uncaring. Usually, it is because they are trying to keep up with the component difficulties of keeping their scientific careers and journals respectively afloat. Unfortunately, those goals are directly in opposition – authors publishing as much as possible means back-breaking amounts of submissions for journals. Increasingly time-poor researchers, busy with their own publications, often decline invitations to review. Subsequently, peer review is then cursory or non-analytical.

And even still, we often muddle through. Until we encounter extraordinary circumstances.





Peer review during a pandemic faces a brutal dilemma – the moral importance of releasing important information with planetary consequences quickly, versus the scientific importance of evaluating the presented work fully – while trying to recruit scientists, already busier than usual due to their disrupted lives, to review work for free. And, after this process is complete, publications face immediate scrutiny by a much larger group of engaged scientific readers than usual, who treat publications which affect the health of every living human being with the scrutiny they deserve.

The consequences are extreme. The consequences for any of us, on discovering a persistent cough and respiratory difficulties, are directly determined by this research. Papers like today’s retraction determine how people live or die tomorrow. They affect what drugs are recommended, what treatments are available, and how we get them sooner.

The immediate solution to this problem of extreme opacity, which allows flawed papers to hide in plain sight, has been advocated for years: require more transparency, mandate more scrutiny. Prioritise publishing papers which present data and analytical code alongside a manuscript. Re-analyse papers for their accuracy before publication, instead of just assessing their potential importance. Engage expert statistical reviewers where necessary, pay them if you must. Be immediately responsive to criticism, and enforce this same standard on authors. The alternative is more retractions, more missteps, more wasted time, more loss of public trust … and more death.

Cancer medication as low as Rs 1,000/month on way

Malathy Iyer, TNN

MUMBAI: It's widely known that a month's dose of cancer drugs can cost lakhs, but what isn't common knowledge is that Tata Memorial Hospital's doctors are working on alternatives that could cost less than Rs 1,000 a month.

Dubbed the metronomic treatment protocol, it comprises daily consumption of a combination of low-dose medicines that are cheap because they have been around for decades. "There is no need to worry about patents or recovery of billions spent on research,'' said Dr Shripad Banavali, head of the medical oncology department of Tata Memorial Hospital, Parel, who has been working on the low-dose-low-cost therapies.

His colleague, Dr Surendra Shastri sums up the mood well: "The metronomics experiment is path-breaking in terms of providing good quality and affordable cancer care for a majority of the over 10 lakh cancer cases diagnosed in India each year.'' These findings could revolutionize cancer care in most developing countries, he said.

The catch is, however, that this branch is still in research stage. The conventional cancer treatment comprising chemotherapy is given at "maximum tolerated doses" which are tested and have reams of research to back it. As against this, in metronomic therapies, the drugs are given at very low doses. "But side-effects are fewer and patients have a good quality of life,'' said Dr Banavali.

The word metronomics is borrowed from music; musicians use the metronome to mark time and hence rhythm. Patients are asked to take these medicines for a period of 21 days or more before taking a break of a week; such a cycle continues for months.

Dr Banavali's work has been published in the May issue of the medical journal Lancet Oncology. Thousands of patients have been on metronomic treatment in Tata Memorial Hospital as well as its rural centre, with a sizeable number managing to control the growth of cancer. "In India, the main challenge in cancer is not just finding cures, but to develop affordable treatments'' he said.

In fact, the metronomic work arose out of such cost concerns over a decade back. The Tata doctors found that many patients were lost to treatment because they were overwhelmed by the cost of medicines. "Moreover, many patients came so late for treatment that we had to turn them away without any medicines,'' said Dr Banavali. Instead of turning away such patients, the Tata team decided to adopt the metronomic treatment as palliative treatment that was being tested out in various parts of the world. "We gave them drugs that would help in pain or at least ensure that their case doesn't worsen," he adds.

But the results surprised them. In a large group of children with blood cancers called acute myeloid leukemia (AML), they found that metronomic maintenance treatment had helped increased survival rate to 67%. The group then started its own innovation, going ahead of the rest of the world. "The West used the same set of medicines for all forms of cancer, but we introduced personalised medicines for various types of cancers,'' said Dr Banavali.

The Lancet paper, titled 'Has the time come for metronomics in low-income and middle-income countries', mentions combination of drugs used for four cancers (see box). The Tata doctors feel that these drugs may work as well for newly developed tumours. "While the developed world is going after 'drug discoveries', that is discovering new drugs which are very costly for our patients, we are going after 'drug repositioning', that is using time-tested drugs for the treatment of cancer," said Dr Banavali. For example they are using drugs likemetformin (an anti-diabetic drug), sodium valproate (an anti-seizure drug) and propranolol (an anti-hypertensive drug) in the treatment of cancer.

The Lancet paper said, "The combination of metronomic chemotherapy and drug repositioning might provide a way to overcome some of the major constraints associated with cancer treatment in developing countries and might represent a promising alternative strategy for patients with cancer living in low- and middle-income countries.''

Metronomics chemotherapy works at three levels. It attacks tumours while working on the "micro-environment", like the blood vessels, around the tumour. Thirdly, it works on the immuno-modulating system of the body. "Unless the metronomic dose works on all these three fields, it may not work,'' said the doctor.

Survivor of twin blows, counselor for others

On the occasion of Cancer Survivors Day on June 2, members of Ugam, an NGO, will put up a skit at Tata Memorial Hospital to underline the problems of parents whose children suffer from cancer. Comprising childhood cancer survivors, Ugam members counsel cancer patients undergoing treatment at Tata Memorial Hospital. One of its founder members, Shalaka Mane (29), who lost her right eye to cancer, will be there too. She feels people discriminate against cancer patients. "I feel there is a need to increase awareness about the treatment of cancer in society and the government, and enable these families to get financial help," she says. Diagnosed with blood cancer when she was eight years old, she barely got back to being at the top of her class when she was found to have brain cancer. "I was in Class XII and planning to take up medicine, but it wasn't to be,'' she says. The second cancer was so virulent that she lost her eye. But she emerged victorious a second time and completed her masters. She now teaches at Kalvidhai Mission High School, Andheri, which she attended as a student. "My principal and teachers never discriminated against me because of my illness. I enjoy my job," she adds.

'Cancer can't affect my future'

Artist Sachin Chandorkar is a poster boy of sorts at the Tata Memorial Hospital, Parel. At 28, he has won several awards (one from chief minister Prithviraj Chavan last month) for his murals and sculptures. But the Tata connection comes from his victory over cancer when he was five years of age. He was suffering from Hodgkin's Lymphoma. "I remember feeling irritated due to the itch on my head. When I would stratch my head, tufts of hair would come into my hands. I would then start crying," says Chandorkar, who studied at the JJ School of Arts. He usually doesn't talk about his battle with the Big C. "When my mother and sisters sometimes talk about it, I ask them to stop. I have decided that cancer is a part of my past and cannot affect my future,'' he adds.