Blog
About

In:  Peer Review  

Advances in peer review

It’s not too hard to see that the practices of and attitudes towards ‘open science’ are evolving amidst an ongoing examination about what the modern scholarly system should look like. While we might be more familiar with the ongoing debate about how to best implement open access to research articles and to the data behind publications, discussions regarding the structure, management, and process of peer review are perhaps more nuanced, but arguably of equal or greater significance.

Peer review is of enormous importance for managing the content of the published scientific record and the careers of the scientists who produce it. It is perceived as the golden standard of scholarly publishing, and for many determines whether or not research can be viewed as scientifically valid. Accordingly, peer review is a vital component at the core of the process of research communication, with repercussions for the very structure of academia which largely operates through a publication-based reward and incentive system.

In spite of this clear importance of peer review, there is increasing evidence that mistakes are becoming ever more frequent in the process. These range from simple gate-keeping errors based on differences in opinion of the perceived impact of research or the rejection of what then become seminal pieces of research, to fraudulent or incorrect work failing to be detected and entering the permanent scientific record. Attempts to reproduce how peer review filters what becomes published demonstrate that peer review is generally adequate for detecting quality work, but often fails to recognise the most impactful research. Many additionally regard the traditional peer review model as unnecessary as it causes delays to the publication and communication of novel research. In addition, some view traditional peer review as being deeply flawed in that it operates within a closed and opaque system in which it is impossible to trace the discussion and any changes made to original research during the process, as well as the decision process leading to publication. This means that at its worst peer review is often seen as being detrimental to research by providing a closed mechanism to protect the status quo and suppress research which is viewed as radical or innovative.

The current system of research communication and evaluation (source)
The current system of research communication and evaluation (source)

In light of this, there has been increasing momentum towards ‘open peer review’, whereby the process becomes more transparent through various means. An article in the BMJ stated that peer review is “slow, expensive, profligate of academic time, highly subjective, prone to bias, easily abused, poor at detecting gross defects, and almost useless at detecting fraud.”, with evidence to support each allegation. With so many potential issues, it is interesting why peer review in its current form has persisted for so long, and perhaps also then not unexpected that we are witnessing the beginning of a transformation from the traditional model.

The traditional model of peer review is based on a pre-publication evaluation system. It has been described as a “model that simply may have run its course given societal and technological change”, and there has been a considerable amount of debate regarding new models of peer review, and new platforms to bring it into line with the digital age.

A potential future system (source)
A potential future system (source)

These discussions around peer review can be generally sorted into four main categories:

  • Should referees receive credit or recognition for their work and, if so, in what form;
  • Should referee reports be made public or closed;
  • Should referees remain anonymous or sign their reports;
  • Should peer review happen prior or subsequent to the publication process.

In this series (posts to come throughout this week), we will summarise the ebb and flow of the debate around these various aspects of peer review, and demonstrate how ScienceOpen is acting to attempt to resolve these issues.

14 thoughts on “Advances in peer review”

  1. Yes, referees should get credit. But how – as ‘scientific service’ by their employer? what if self-employed, retired, unemployed, or their employer does not care?
    Some journals give free online access but that is a weak incentive because many scientists already get free access through their institution and can always email the author for a pdf.
    I suggest OA journals give vouchers to referees that reduces their cost of publication (some journals already do this). Primarily subscription journals could also give OA vouchers because they all have some OA papers.

    1. Hey Mark,

      Thanks for your comment! We actually have a whole post on the ‘credit’ aspect of peer review out later today!

      Jon

  2. I think whether reviews are made public or not will change the review process. No longer will it be a semi-private corresspondence but referees will be ‘publishing’ their views. I am not sure if this is good or bad but such experiments are underway and I look forward to seeing how they work out.

    Whether referees names should be public or not, should remain discretionary. Some referees are afraid of vindictive authors.

  3. My preference is peer review before publication so what we do read will be generally higher quality. I believe the opn-review and post-review (e.g. online comments on papers) get limited use and so are (at present) less reliable that editors organising prior peer-review.

  4. I think the system of uploading preprints onto the ArXiv preprint server when sending a paper off to a journal works quite well in maths and physics. Essentially, the community gets to see the paper at the same time as the referees, so it combines the traditional test-by-referee with open access.
    The best part is that one can request a weekly summary of all papers uploaded to the ArXiv in a certain sub-field that week. …

    1. Sure, those communities know what they’re doing. In a forthcoming post in this series, I’ll touch briefly on ‘piggyback’ journals that leverage the existing infrastructure of the arXiv to essentially make legacy publishers entirely redundant (eg Discrete Analysis).

  5. The danger with preprints in biology is that some journal’s editors may consider them prior publication and refuse to accept them as a submission.

  6. Peer review can never be perfect but usually improves papers and passing it is an immediate indicator of quality. Thus it benefits readers, authors, employers, funding agenicies, and publishers. That is why it dominates science despite its imperfections. Scientists can still disseminate their work without it as they always have through conference presentations, books, reports, Web pages, and other non peer reviewed ways.

Leave a Reply

Your email address will not be published. Required fields are marked *