Call for more blind data recording

Biologists from The Australian National University (ANU) have stated that journals should insist on blind trials more strongly — meaning that experimenters are unaware of the identity or treatment group of their subjects while conducting research — in order to minimise bias.

The statement follows a study which saw the researchers analyse nearly 900,000 papers from the PubMed life sciences database using automated data mining. In a blind trial, they compared 83 pairs of evolutionary biology papers on similar topics, in which the data was collected blind in one and not in the other. The results were published in the journal PLOS Biology.

The team found that non-blind experiments averaged a 27% stronger result than blind trials. Lead researcher Dr Luke Holman said non-blind papers “tended to exaggerate differences between the experimental group and the control group… For example, a non-blind trial of a new drug might conclude that it is way more effective than a placebo, when in fact the drug’s true effect is rather modest, simply because the researchers’ expectations biased the results.”

The team also found that non-blind studies rejected the null hypothesis more strongly. Dr Holman said, “Non-blind studies more confidently concluded that differences between treatment and control groups were real, and not just due to chance variation.”

Co-researcher Dr Megan Head said scientists use techniques such as blind trials to minimise their biases, but the pressure to get things done faster leads to some people skimping on experimental design. In fact, the review suggested that less than one in four experiments used blind data recording.

“Many researchers are unaware that their expectations can introduce such strong bias, and so they don’t feel the need to work blind,” Dr Holman said.

He and his colleagues believe that journals should insist on blind trials more strongly, perhaps by making prominent statements to authors and peer reviewers about the necessity of using blind trials. Dr Holman suggested better training is part of the solution, with Dr Head noting, “It is not necessarily slower to take data blind, you just need to be a little creative.”

Source