Blog
About

In:  Peer Review  

Peer review: open sesame?

Open peer review has many different aspects, and is not simply about removing anonymity from the process. Open peer review forms part of the ongoing evolution of an open research system, and the transformation of peer review into a more constructive and collaborative process. The ultimate goal of traditional peer review remains the same – to make sure that the work of authors gets published to an acceptable standard of scientific rigour.

There are different levels of bi-directional anonymity throughout the peer review process, including whether or not the referees know who the authors are but not vice versa (single blind review), or whether both parties remain anonymous to each other (double blind review). Open peer review is a relatively new phenomenon (initiated in 1999 by the BMJ) one aspect of which is that the authors and referees names are disclosed to each other. The foundation of open peer review is based on transparency to avoid competition or conflicts born out through the fact that those who are performing peer review will often be the closest competitors to the authors, as they will tend to be the most competent to assess the research.

The publishing of referee names is a distinct, but related, issue to that of whether or not to make reviewer reports open, which we have a post on tomorrow. In fact, the two are often conflated during discussions of open peer review, and for clarity should be distinguished. Therefore, it is more useful to call the actual process of publishing referee reports ‘open peer review’, while the disclosure of referee names could be termed ‘signed peer review’ to avoid conflation.

To name or not to name?

The debate between signed or not-signed reviews is not to be taken lightly. Early career researchers are perhaps the most conservative in this arena as they may feel afraid that by signing overly critical reviews (i.e., those which investigate the research more thoroughly), they will become targets for retaliatory backlashes from senior figures. The traditional double blind process therefore offers, in theory, a sort of protection for those in junior positions. As well as referee anonymity, author anonymity might be viewed as saving junior authors taking their first steps into the world of academia from public humiliation from more established members of the research community. These potential issues are part of the cause towards a general attitude of conservatism from the research community towards opening up the peer review process.

In a perfect world, we would expect that strong, honest and constructive feedback would be well received by senior researchers, but there is an apprehension that this would not be the case. However, retaliations to referees in such a negative manner are serious cases of academic misconduct, and likely to be dealt with as such. In fact, increasing transparency can only make mitigating or tackling this potential issue easier. Therefore, this perceived danger is highly unlikely to be acceptable or ever actually occur in the current academic system, and if it does can be dealt with through increased transparency.

Openness and accountability

It is not clear how this widely-exclaimed but poorly documented potential abuse of signed-reviews is any different from what would occur anyway in a closed system, as anonymity provides a potential mechanism for referee abuse. On the other hand, by reviewing in a public manner, one could argue that such backlashes are prevented as researchers will not want to do anything to tarnish their reputations in a public forum. In these circumstances, openness becomes a method to hold both referees and authors accountable for their public discourse, as well as makes public the decisions which Editors make on referee and publishing choice.

Source
Source

It has been argued that by disclosing both the author and referee names, detection of scientific misconduct will becomes easier, and that over time the quality of submitted articles will improve. Perhaps most importantly, by the names of those involved in the process being open a measure of transparency is gained into the peer review process, and can provide important context into the subject matters being discussed. Furthermore, by signing reviews, referees are able to obtain credit for their work and for contribution to the development of the scientific record.

How can conflicts of interest be detected if referees and authors are anonymous? Image: Hilda Bastian, CC NC
How can conflicts of interest be detected if referees and authors are anonymous? Image: Hilda Bastian, CC NC

Such a system provides a middle-ground between protecting authors from any potential misconduct while making sure that the review content is available for evaluation and reuse. Whether or not referees are identified to the author has been shown in one study to have little effect on the quality of peer reviews, the recommendation regarding publication, or the time taken to review. However, in this instance, reviewers were much more likely to decline to review if they had to disclose their names. The concern that signed peers review leads to less critical reviews is not supported by current analyses, and in fact some have found that the quality of peer reviews actually increases through anonymity. We look forward to seeing continued research into the impact of signed peer review in the future, and in the mean time, see this great review of studies by Hilda Bastian on the advantages versus disadvantages of signed peer review.

At ScienceOpen, we expect peer review to be conducted in a professional and courteous manner, and therefore we publish both signed reviews and author names. This facilitates a continuing dialogue subsequent to the publishing of research, and for us represents an important aspect of providing transparency into the peer review process. We expect the community to be self-regulating in this regard, and to be civil throughout the review process.

This is the third part in a series on the evolution of peer review, with Part 1 being a general overview and Part 2 discussing how to appropriately credit the peer review process.