Double-anonymous peer review reduces reviewer bias, finds three-year trial

About the peer review process

In scholarly publishing, scientific papers go through a peer review process before they are accepted for publication in a journal; other researchers will be invited to comment on the quality of the methods and conclusions presented in the paper. These comments play an important role in determining which papers the journal will accept and publish. However, there are concerns that the peer review process is not entirely fair for all authors.

Most life science disciplines currently utilise the practice of single-anonymous peer review to decide which papers are suitable for publishing. This involves hiding the reviewers’ identity from the authors, but not the authors’ identity from the reviewers. This had led to concerns of conscious or unconscious bias regarding gender, nationality or affiliation corrupting the review process.

The BES set out to explore if hiding authors’ identities in addition to reviewers’ identities would reduce the potential for bias in the publishing process. This is a method known as double-anonymous peer review.

A randomised trial, using real manuscripts submitted to Functional Ecology between 2019 and 2022, required authors to submit their paper for review with their identities anonymised. This included papers from authors of various backgrounds. Half of the submissions were then randomly chosen to have author details added to their title pages.

The double-anonymous peer review trial indicates reviewer bias is reduced when author identities are anonymised: when reviewers did not know whose paper they were reviewing, peer review outcomes were similar across author demographics; in contrast, when reviewers did know whose paper they were reviewing (e.g. single-anonymous peer review), papers with first authors residing in higher-income countries and in countries with higher English proficiency were favored.

Interestingly, anonymising author details had no effect on gender differences in reviewer ratings or editor decisions.

What can we take away from the results?

Professor Charles Fox, the lead author of the paper and the previous Executive Editor of Functional Ecology, explains “Our trial provides strong evidence that authors from higher-income and/or English-speaking countries receive significant benefits to being identified to reviewers during the peer review process, and that anonymizing author-identities (e.g., double-blind review) reduces this bias, making the peer review process more equitable.

“It’s critical for science, and for the scientists involved, that peer review be a fair and unbiased process. The results of this trial will help inform publishers on the best ways to minimise some sources of bias in the publishing process.”

The British Ecological Society will begin transitioning its journals to mandatory double-anonymous peer review. This decision is based off of the strong evidence uncovered in the study. This will begin with Functional Ecology, with other journals published by the BES to follow.

Andrea Baier, Director of Publishing at the British Ecological Society, said “The British Ecological Society is committed to promoting equitable practices in international ecological science. Authors from all over the world submit to our seven journals and it is vitally important that the research we publish is reviewed and selected in the most impartial way, regardless of the authors’ backgrounds.”

Professor Rob Freckleton, University of Sheffield and Chair of the British Ecological Society’s publications committee, said: “On behalf of the British Ecological Society, the publications committee supported this important experiment, and from the outset we committed to being led by the results it would produce. We now have the evidence that double-anonymous peer review is an important building block towards greater equity in publishing and we are acting on it.”

Read more here:

Fox, C. W., Meyer, J., & Aimé, E. (2023). Double-blind peer review affects reviewer ratings and editor decisions at an ecology journal. Functional Ecology, 00, 1– 14. https://doi.org/10.1111/1365-2435.14259