Shoddy Science - Study showing that chocolate can help with weight loss was a trick

Kashmira Gander in The Independent

A journalist seeking to lay bare how the research behind fad diets can be “meaningless” and based on “terrible science”, has revealed how he tricked international media into believing that chocolate can aid weightloss.

Posing as Johannes Bohannon, Ph.D, the research director of the fabricated Institute of Diet and Health, biologist and science journalist John Bohannon ran what he called a “fairly typical study” used in the field of diet research.

German broadcast journalists Peter Onneken and Diana Löbl asked Bohannon to conduct a clinical trial into the effects of dark chocolate, as part of a documentary exposing how simple it is for bad science to make headlines.

“It was terrible science. The results are meaningless, and the health claims that the media blasted out to millions of people around the world are utterly unfounded,” Bohannon wrote of his in an article for io9.com.

To collate their data, the team asked 5 men and 11 women aged between 19 to 67 to follow either a low-carbohydrate diet, or the same diet but with an added 1.5oz of dark chocolate. Meanwhile, a control group were asked to eat as normal.

While the data showed that the group which ate a low-carb diet while indulging in dark chocolate lost weight 10 per cent faster and had better cholesterol readings, Bohannon stressed that the study was set up for failure.

“Here’s a dirty little science secret: If you measure a large number of things about a small number of people, you are almost guaranteed to get a “statistically significant” result.”Bohannon explained that thanks to a method known as "p-hacking", the study could have just as likely shown that chocolate helped sleep or lowered blood pressure as the threshold for data being classed as "significant" is just 0.05 per cent.

THE STUDY COULD HAVE JUST AS LIKELY SHOWN A POSITIVE EFFECT ON BLOOD PRESSURE THAN WEIGHT LOSS, SAYS BOHANNON (PHOTO: GETTY IMAGES)
 

After a journal was duped into accepting the shoddy study for money, Bohannon and his colleagues then put together a press release which would be irresistible to journalists and editors looking for a story.

The research was splashed across newspapers and websites across 20 countries in half a dozen languages, seemingly proving how easily dodgy science can circulate across the world’s media.

However, some have questioned the ethic behind the study, with one commenter writing below the piece:

"Going out of their way to court media coverage (that was going to repeat the story but never, ever follow up on the hoax once it was revealed — and this is a hoax) was ethically problematic,” one commenter said.

"If you want to uncover the shady workings, that didn’t really help. It seems like only the people who were already aware of the problems are going to be reached by this reveal."

The stunt came after Bohannon exposed how “open access”science journals were willing to publish deeply flawed research claiming that a drug had anti-cancer properties.

In the investigation for the prestigious journal Science, he revealed the dangers of abandoning the peer-review process expected in science publishing.