What are the pros and cons of open peer review?

Open Peer Review

When the editor of the British Medical Journal (BMJ), Richard Smith, announced in 1999 that the journal would be abolishing anonymous peer review in favour of identifying referees, most other journal editors stood back to watch how this would unfold before jumping on board.

Smith argued that disclosing the names of reviewers and authors was an ethical imperative because, he said, “a court with an unidentified judge makes us think immediately of totalitarian states and the world of Franz Kafka”.

Twenty years later, there is still on-going debate about the pros and cons of open peer review versus anonymous peer review, along with a range of options in between.

The Purpose of Peer Review?

Peer review is the process whereby editors of journals require authors to submit their work to be evaluated by a “jury” of their colleagues working in the field. The peers are asked to read, comment and recommend whether or not the work is valid and worthy of being published.

Typically, peer review has been anonymous, where the names of the people considering the work are not revealed to the author, also known as single-blind. There are double-blind variations of closed review as well, where names of reviewers and names of authors are withheld from each other.

Prestigious journals have traditionally based much of their reputation on the peer review process, which is claimed to bring integrity, impartiality, speed and quality to the publishing process.

The Problems with Closed Peer Review?

But, questions have been raised about how impartial an anonymous reviewer can be. Issues of gender, racial and geographic bias have been noted over time, and there is a sense that some academics might judge those more junior or more senior to themselves more harshly or leniently.

Peer review is supposed to filter submissions and help move a piece of work more swiftly through the process towards publication; however, in many cases, peer review can actually introduce tedious delays.

Peer review is also relied upon to detect instances of error or fraud. Yet, there have been many stories of bad science making it through.

Lastly, there is a notion that the feedback offered by reviewers will improve the article itself. However, behind a cloak of anonymity, reviewers can be tactless at best, nasty at worst, and sometimes downright corrupt if they defer an innovative or competitive idea, or steal it.

When Did Peer Review Start?

The first-ever peer-reviewed journal was in 1732 for the Royal Society of Edinburgh. Interestingly BMJ was the second, in 1893. Current Anthropology has been offering open peer review commentary since 1959. It took another 40 years before BMJ – the next journal to adopt open review – did so after the announcement by Smith in 1999.

Richard Walker and Pascal Rocha da Silva’s timeline is probably one of the best overviews of the evolution of peer review and the various paths that have been taken.

Their study focused on the emerging trend of open review and attributed the rise to rapid growth in pre-print service and Open Access journals. Differences remain, however, especially between the physical sciences and social sciences, which continue to use classical peer review more. Even within these disciplines, differences abound, with some journals offering double-blind review and others offering a non-selective review.

Why Move From Anonymous Peer Review To Open Peer Review?

What Is Open Peer Review?

Open peer review, in its most fundamental form, means removing the veil of anonymity from both the author and the reviewers. The idea is that open review encourages a more careful and considerate analysis on the part of reviewers who know their names will be attached, it encourages reviewers to disclose conflicts of interest, and it brings much more transparency to the process overall.

In an attempt to systematically categorise the myriad definitions of open peer review, Tony Ross-Hellauer counted more than 120 definitions. By analysing these definitions, however, he was able to build a coherent typology that includes the following kinds of open peer review:

Open identities
Open reports
Open participation
Open interaction
Open pre-review manuscripts
Open final-version commenting
Open platforms

The two most common trends are towards open identities – where names of authors and names of reviewers are disclosed to each other, and open reports – where reviewer’s reports are published alongside the final article.

(For starters, if you are unfamiliar with open peer review, simply reading Ross-Hellauer’s article online and seeing the ways in which the reviews are published alongside his report will give you insight into how rich the process can be.)

The Shift to Open Peer Review

The move towards some form of open peer review in terms of reviewers and reports has been driven in large part by the desire to bring accountability to curb hostile comments or unfair criticisms, prevent or expose systematic biases, and avoid conflicts of interest.

Some, like the editors of science journal, Nature Neuroscience, acknowledge that there are such abuses, but claim that most can be prevented by a careful editor. In addition, reviewers may be more willing to do the unpaid labour of reviews, if they can remain anonymous.

One informal option has been to allow reviewers to choose to sign their reviews or reveal themselves in order to have follow-up discussions to revise and resubmit.

Drummond Rennie, deputy editor of the Journal Of the American Medical Association, however, reminds us that “we have a long history to tell us that secrecy is the enemy of justice.”

Benefits of Open Review

There are a number of good reasons to lobby for an innovative approach to open peer review. At an OpenAire conference a couple of years ago, a number of models were discussed, as well as the need for a standardised vocabulary and a common, clear code of conduct.

Among the most compelling reasons for open review is that if, indeed, the purpose of any review is to improve the submission, then publishing a review alongside or concurrently with the article can be considered part of the process. Moreover, it can offer recognition to those who do this labour. And, it can also serve as a demonstration for newer or younger reviewers on best practices. Lastly, a repository of open reviews could be seen as adding to the body of knowledge about a particular topic or a particular methodology.

A special 2015 issue of the journal, The Political Methodologist, was devoted completely to the topic of peer review. In it, Daniel Freiro argues that open peer review creates “incentives for referees to write insightful reports” and that by assigning a DOI number to reviews, giving them status as a piece of scholarly work, ensures that reviewers are given credit too.

What’s It All For?

Despite these challenges, most academics still believe there is some value to peer review. The main goal should always be to assist author in getting a quality piece of work published and different disciplines might demand different forms of review. In the end, it’s up to the editor to mediate between author and reviewer, to determine what process will produce the best possible outcome.