In:  Peer Review  

Advances in peer review

It’s not too hard to see that the practices of and attitudes towards ‘open science’ are evolving amidst an ongoing examination about what the modern scholarly system should look like. While we might be more familiar with the ongoing debate about how to best implement open access to research articles and to the data behind publications, discussions regarding the structure, management, and process of peer review are perhaps more nuanced, but arguably of equal or greater significance.

Peer review is of enormous importance for managing the content of the published scientific record and the careers of the scientists who produce it. It is perceived as the golden standard of scholarly publishing, and for many determines whether or not research can be viewed as scientifically valid. Accordingly, peer review is a vital component at the core of the process of research communication, with repercussions for the very structure of academia which largely operates through a publication-based reward and incentive system.

In spite of this clear importance of peer review, there is increasing evidence that mistakes are becoming ever more frequent in the process. These range from simple gate-keeping errors based on differences in opinion of the perceived impact of research or the rejection of what then become seminal pieces of research, to fraudulent or incorrect work failing to be detected and entering the permanent scientific record. Attempts to reproduce how peer review filters what becomes published demonstrate that peer review is generally adequate for detecting quality work, but often fails to recognise the most impactful research. Many additionally regard the traditional peer review model as unnecessary as it causes delays to the publication and communication of novel research. In addition, some view traditional peer review as being deeply flawed in that it operates within a closed and opaque system in which it is impossible to trace the discussion and any changes made to original research during the process, as well as the decision process leading to publication. This means that at its worst peer review is often seen as being detrimental to research by providing a closed mechanism to protect the status quo and suppress research which is viewed as radical or innovative.

The current system of research communication and evaluation (source)
The current system of research communication and evaluation (source)

In light of this, there has been increasing momentum towards ‘open peer review’, whereby the process becomes more transparent through various means. An article in the BMJ stated that peer review is “slow, expensive, profligate of academic time, highly subjective, prone to bias, easily abused, poor at detecting gross defects, and almost useless at detecting fraud.”, with evidence to support each allegation. With so many potential issues, it is interesting why peer review in its current form has persisted for so long, and perhaps also then not unexpected that we are witnessing the beginning of a transformation from the traditional model.

The traditional model of peer review is based on a pre-publication evaluation system. It has been described as a “model that simply may have run its course given societal and technological change”, and there has been a considerable amount of debate regarding new models of peer review, and new platforms to bring it into line with the digital age.

A potential future system (source)
A potential future system (source)

These discussions around peer review can be generally sorted into four main categories:

  • Should referees receive credit or recognition for their work and, if so, in what form;
  • Should referee reports be made public or closed;
  • Should referees remain anonymous or sign their reports;
  • Should peer review happen prior or subsequent to the publication process.

In this series (posts to come throughout this week), we will summarise the ebb and flow of the debate around these various aspects of peer review, and demonstrate how ScienceOpen is acting to attempt to resolve these issues.