‘Open research’ isn’t just about sharing resources like data, code, and papers, although this is a big part of it. One big, and often under-appreciated aspect of it is about making research accessible, inclusive, and participatory. A major principle driving this is leveraging transparency to bring processes and factors that are currently hidden into public view.

One area of research and scholarly communication where the debate is still very much ongoing for this is for peer review – our system of validation and gatekeeping to the vast archives of public knowledge.

OpenAIRE have released an important new survey and analysis on attitudes and experiences towards ‘Open Peer Review’ (OPR), based on more than 3000 respondents (full data available here to play with). This is important, as OPR is all about the principles above – making the process transparent, collaborative, inclusive, and in the end, better!

Below, we discuss some of the major findings of the survey, and how we at ScienceOpen fit into the bigger picture of Open Peer Review.

The future is Open

The main result of the survey is that the majority (60.3%) of respondents are in favour of OPR becoming a mainstream scholarly practice, particularly regarding open interaction, open reports and final-version commenting. Part of this is due to the relatively lower satisfaction scores reported, with just 56.4% of respondents being satisfied with traditional closed peer review, and 20.6% being dissatisfied – a much lower gap than all previous reports. From the survey, more than three quarters of respondents had previously engaged with OPR either as an author, reviewer, or editor. This suggests that OPR, in one form or another, is already probably more common practice than we might think.

Interestingly, this development is similar to what we saw with other aspects of ‘open science’ such as open access and open data – there is debate, experimentation, variable implementation, and finally they start to become accepted as the norm as policies, practices, and cultures adapt. The survey also showed that 88.2% of respondents were in favour of Open Access to publications, a much higher value than several years ago. It also found that support for OPR is correlated with support for Open Data and Open Access, which is perhaps not surprising, although conversations regarding OPR are still in their relative infancy.

This suggests that as debates around OPR mature, we are likely to see an increase in the uptake and support of it, as with other areas of ‘Open’. Indeed, the survey also found a difference in generational support for OPR, with younger generations favouring it more over more-established researchers. As it is these generations who will inherit and govern the system in the future, it is more likely to have the characteristics that they favour.

Make peer review a dialogue

68% of respondents favoured some sort of interaction between authors and reviewers, with peer review becoming a process of reciprocal discussion rather than a monologue or a checklist. At ScienceOpen, this is something we strongly encourage and facilitate, with authors being able to directly respond to reviewers in public, and vice versa.  A fantastic working example of this in action at ScienceOpen can be found here.

Interestingly, 85% of respondents thought that reviewers would only participate if directly invited to review, as opposed to voluntary participation. This is why we’ve always been explicit that editorial control is a crucial part of any open peer review system, and allow anyone, including collection editors, to mediate their own peer review across ScienceOpen.

To name or not to name?

However, some aspects of OPR, such as revealing reviewer identities to authors, are still contentious. More than half of the survey respondents believed that open identification would reduce the quality of peer review. This seems to reflect concerns that making reviewers eponymous would lead to a lack of control and protection for them, particularly if at a junior career stage. This in turn would inhibit those reviewers who fear the most retaliation from being able to be thorough or candid with their reviews to due possible reprisals from senior authors.

Now, this is not an issue to be taken lightly, and is clearly one of the greatest barriers to OPR. We addressed this issue of retaliation and power dynamic abuse in a recent post, noting that the current system of peer review empowers bad actors with position and status to marginalize those with relatively less power. This means that the inverse is also possible too, in that some people might use OPR to be deliberately confrontational in public, and to talk down to or intimidate their junior peers. Therefore, any alternative or complimentary system has to reduce or minimalise this negative dynamic, make sure that an accountability process is inbuilt and managed, and that marginalised communities feel invited to participate. Too many social platforms already have comments that are negatively geared against women or minorities, and therefore any OPR platform has to have a mechanism in place to reduce or negate this entirely.

At the moment, there doesn’t seem to be any consensus on how to resolve this issue. There are arguments for increasing anonymity (blinding peer review) for protection, and also increasing openness to expose bad behaviour and make researchers more accountable. These arguments apply to both pre- and post-publication OPR, and at the moment there is generally insufficient data on the impact of anonymity vs identification on OPR.

73.9% of survey respondents thought that reviewers should at least have the choice of making their identities open or not. Furthermore, 67.2% thought that potential reviewers would opt of journals who enforced open identities, and 65.2% believed that open identification would lead to less strong criticisms during peer review.

However, the survey also found that 44% of respondents believed that open identification would increase the overall quality of review, versus 35% who thought the opposite. As such much of the issues against open identification are more to do with the moderation of the process, and expose a fairly dark perception of academic culture, as mentioned above. This is probably where much of the skepticism derives from, and in fact once you see past or are able to overcome these issues, then the inherent value of open identification increases. However, perceptions are extremely important in this case, and much more research is required into the impact of open identification for not just reviews, but also reviewers and the culture of peer review.

At ScienceOpen, we have found that all OPRs performed to date on our platform have been civil and constructive. All reviewers have been identified, as this is a pre-requirement due to the integration with ORCID. This doesn’t mean that we have managed to eliminate all potential biases and issues, but it at least seems to be working better than the traditional system, in which the many biases still largely remain (see, for example, a recent study on gender bias here).

Open reports = more research context

65.4% of survey respondents think that peer review reports provide important additional information to readers, and that publishing reports will lead to higher review quality. However, 46% also felt that publishing review reports would also make reviewers less likely to make strong criticisms.

At ScienceOpen, review reports, including images and tables, are always published alongside the relevant articles. They are published under a CC-BY license, and each is assigned a CrossRef DOI to maximise their potential utility. This makes the process more reproducible and therefore valid as a scientific process, and also allows reports to be treated equivalent to any Open Access publication.

Overall, we couldn’t be happier to see the results of this survey, and the trend of OPR becoming more mainstream. We’ve been strong advocates for open peer review since our inception, and have many functions on our platform to help researchers perform this.

One important point from the survey though was the differences between disciplines. We should always make sure that we are listening to, and engaging and empathising with the experiences of others, and not trying to create a ‘one size fits all’ solution. This is important, as communities have different properties depending on their composition, diversity, culture, and history, among other things. These are values that all need to be considered when developing structural changes in peer review.

Taken together, these findings are very encouraging for OPR’s prospects for moving mainstream but indicate that due care must be taken to avoid a “one-size fits all” solution and to tailor such systems to differing (especially disciplinary) contexts. More research is also needed in some areas, particularly the relative impact of blinded versus identified reviewers.

OPR is an evolving phenomenon and hence future studies should be strongly encouraged, especially to further explore differences between disciplines and monitor the evolution of attitudes.