This guest post was created by the founding partners of the first ever Peer Review Week (#peerrevwk15), namely ORCID, ScienceOpen, and Wiley, with support from Sense About Science.
We talk a lot about peer review in the scholarly communications world. Many of us – and our organizations – are working to improve both the process and the experience for researchers, which has led to a significant increase in the range of options available, especially – but not exclusively – for reviewing journal articles. From double blind to completely open review, pre- and/or post-publication, and even transferrable peer review, not to mention the work being done on peer review recognition and validation by organizations like Publons and PRE, there’s a plethora of new approaches and services to choose from.
Image credit: Listen Up by woodleywonderworks, Flickr, CC BY
But what do researchers make of all this? What are their experiences of peer review? How and why do they review themselves, and what do they get from reviews of their own work? In this reflection from researchers around the world, we asked some of them to tell us about their views of peer review.
By and large, their feedback was very positive, with good experiences outweighing bad and universal agreement that peer review is, as Elizabeth Briody of Cultural Keys, USA, says: “a critically important process for evaluating the merit, content, relevance, and usefulness of scholarly publications” – or as Hugh Jarvis, Cybrarian, University at Buffalo, USA, describes it: “Peer review is the glue of academic publishing.” Saurabh Sinha, Executive Dean, Faculty of Engineering & the Built Environment, University of Johannesburg, South Africa agrees that: “it positions our work with respect to the body of already published knowledge. The approach also helps to ensure, as far as possible, the correctness of the work, elimination of potential blind spots, and validity of assumptions for a practical world.”
Image credit: AJ Cann/Flickr, CC BY-SA
Pretty much everyone noted the importance of peer review – both as reviewer and author – to them personally as well as professionally. For example, Professor Yongcheng Hu, a medical researcher in China commented that: “Peer review is an essential arbiter of scientific quality, no doubt, it has a great impact on scientific communication and is of great value in determining academic papers’ suitability for publication, while for me, via personal experience, it is also an process of exploration and sublimation.” Erik Ingelson, Professor of Molecular Epidemiology at Uppsala University in Sweden, currently Visiting Professor at Stanford University, USA adds: “Mostly, my experiences of being a reviewer have been positive; I get to think critically about study design and methods and learn new things on the way. Similarly, most of the time the review process is positive also as the author, since you get valuable input and the paper that comes out is often better than the original submission.” Anna Cupani, a Belgian researcher, agrees: “Having someone reading and commenting on your research is beneficial for several reasons: it validates your work, it confirms what you are doing is meaningful not only for you but for a wider scientific audience and it helps you focus and improve your research. You never grasp the meaning of something as deeply as when you have to explain it to someone else!” And Lee Pooi See, Associate Chair (Research), School of Materials Science and Engineering, Nanyang Technological University, Singapore adds: “My personal experience of being reviewed has been interesting; especially in receiving scientific viewpoints from different reviewers on emerging topics. Peer review also steers us to identify those unaddressed aspects of the related research topics.
Several people also commented that there are upsides and downsides to peer review. Janine Milbradt, who is currently working on her PhD at the Institute for Human Genetics, University of Cologne, Germany, says: “You never know what is going to happen! All you can be sure about is that you will have to put another 3-6 months of work into your paper. Having a paper reviewed is a nerve-stretching process, filled with hopes and dreams about the reviewers actually liking your research. On a more serious note, the review process is a very important tool to find incomprehensible or knowledge lacking parts of your research to improve your paper.” Professor Wong Limsoon, KITHCT Professor of Computer Science, National University of Singapore comments: “I appreciate very much constructive reviews that gave me really useful suggestions on my work. I am sometimes annoyed by uninformed comments, but fortunately these are few.”
Logo: download wiley.com/peerreviewweeklogos
So what improvements to peer review would our group of researchers like to see? To quote Professor Sinha again: “Scholarly peer-review has…the opportunity to improve beyond the past, where today, coupled with data, crowd-sourced reviews/discussion, newer open-access technologies could play a dynamic role of developing credibility of research-work and at the same time increasing competition!” Hugh Jarvis likewise has “great hopes that peer review will develop a much more expanded role in the future, and provide input before and after publication, similar to the role the comments serve in Current Anthropology and the product ratings in sites like Amazon.com.” And Joao Bosco Pesquero, Professor, Federal University of Sao Paulo, Brazil would also like to see a more open approach: “The more openly we produce science and expose our work to criticism, the more it helps to improve what we do.”
Perhaps the best summary of why researchers continue to value peer review – both as authors and as reviewers – comes from PhD student, Grace Pold of UMass – Amherst, USA, who told us: “Although I have had the opportunity to formally review only four or five papers, reviewing papers is one of my favorite things to do. First off, it is a good reminder that not all papers are born perfect, and when I am struggling to try and finish my own work and the prospect of a well-polished manuscript seems too far in the distance, it gives me hope. Second, is there a better opportunity to see what your colleagues are working on and thinking about than by reviewing their work? Third, the idea of being able to help shape the information released into the public sphere is a very enticing. Fourth, it is a great excuse to really think about the assumptions you and others make in your research…when you review, it is your responsibility to stop and think about why this is the way things are done. Fifth, thinking up alternative interpretations and then filtering through the data presented in the paper to determine the robustness of the conclusions is a rewarding challenge. Finally, reviewing papers provides an opportunity to slow-down and formulate a full, well-rounded opinion on something, something which happens unfortunately rarely in the life of the frantic modern scientist stuck in with the nitty gritty details of doing experiments. And I think that from a personal perspective, that final point of generating a sense of accomplishment in doing a good job in thinking things through to the end is probably the greatest motivation for me to review papers.”
Image credit: Into by Gisela Giardino, Flickr, CC BY-SA
Imagine if you will a perfect world where all knowledge is openly available to use and share without restriction. This might seem like a bit of a stretch most days but bear with me here!
Believe that the content narrative continues to move beyond the confines of today’s mainly static article. That an ongoing stream of results, data, figures and ideas flows for transparent review and discussion. In short, that a reductionist approach to scientific communication prevails which renders journals with their slow publication cycles and impact factors obsolete.
It’s not that hard to see the evidence of these trends already. Think about the rise of blogs and social media as suitable places for scientific discussion, the growing importance of continuous publication, data sharing and interactive figures. All this in the pursuit of making research and researchers themselves more visible, as they deserve to be.
Logo: download wiley.com/peerreviewweeklogos
This Peer Review Week, ScienceOpen wants to pose a simple question. As the number of research outputs grow and diversify (data sets, negative results, case reports, preprints, posters…) is the research community going to be able to peer-review all these objects prior to publication?
We think not. There isn’t enough time in the day, money to pay for it or even appetite for doing this now. Will these outputs be useful none-the-less? Absolutely, if we have a powerful way to find and filter them based on parameters readers find helpful and authors find rewarding. For example:
What do my peers think of this information?
Are there any updates to it?
What impact did it make in the world and who noticed?
Which work is worth highlighting in a specific field?
How many times was it cited and where?
If I took the time to review it, can my contribution be found and cited?
Will these efforts enhance my career prospects?
How many times was it cited and where?
None of these valid questions are impacted by an evolution away from blind or double-blind anonymous peer review, apart from the speed with which we can answer them. Transparent processes and simple web tools can filter faster, better and cheaper than journals and pre-publication peer review ever could.
This is why at ScienceOpen we’ve developed systems for Post-Publication Peer Review; Versioning; DOI allocation; Article Metrics; Collections; Open Citation Information and more – to demonstrate a different (and we would argue better) way forwards.
This inaugrual Peer Review Week, we invite you to consider this argument and disagree with us by all means. We look forward to a lively and spirited debate!
Image credit: Life is good by John Hain, Flickr, CC BY
Life in California is good. Truthfully, that’s an understatement. As an ex-pat Brit, it’s great. Public holidays are rarely marred by rain; tomatoes grow outdoors (as do Oranges and Avocados); every work day is “casual Friday”.
There’s really only one downside, and that’s our time zone which means that in terms of the global conversation, we are constantly last to the party!
And so it goes with the first ever Peer Review Week. As the “lady at the helm” for social media, it’s lunchtime here in San Francisco and I am frantically trying to catch up with all the stories that everyone else has already posted.
Logo: download wiley.com/peerreviewweeklogos
Rather than give you an exhaustive list of the conversations and coverage, which you can see for yourself from #peerrevwk15, I am going to highlight a few that particularly stood out from me.
Posts from our fellow Peer Review Week founding partners – Wiley, ORCID (Alice Meadows of ORCID on Scholarly Kitchen) and Sense About Science.
Author instructions for an additional peer review process here at ScienceOpen, called Peer-Review by Endorsement or PRE with a reduced publication fee of 50% off. More on this from Board Member Jan Valterop this week, in the meantime here’s a previous post.
A Webinar on October 1st, which includes among its speakers our co-founder Alexander Grossmann. Sign up here.
Listen to this podcast by Chris O’Neil from Bioscientifica which begins with a truism “none of us like the peer review process”! He goes onto explain that despite this visceral reaction, that most researchers accept that their article is improved by it.
Image credit: Thank you by NoirKitsuné, Flickr, CC BY
These articles (and those that we hope to publish) are curated by Professor Friedrich C. Luft, Director of the Experimental and Clinical Research Center (ECRC) at Charite and Max Delbrueck Centre in the Helmholtz Association in Berlin, Germany and Dr. Nana Bit-Avragim, Physician-Scientist and Open Access Advocate.
Clinical case reports remain an essential part of lifelong learning in medicine. Reading at least one a day allows clinicians to hone their differential diagnosis skills beyond their own immediate bedside. Indeed, this knowledge is so vital for shaping the best patient outcomes that it deserves to be openly published so that everyone, regardless of their resources, can read and re-use it as they wish enabling them to:
Share interesting and unique disease manifestations and diagnostic methods
Provide invaluable first-hand source of evidence about general and novel therapeutic approaches across the globe
Help identify life threatening adverse reactions to medications
Exchange practice information and generate a wider search for evidence
To all clinicians out there we say “unlock your education doc!” by openly reviewing the articles in this collection or any that you find interesting from nearly 10 million (open articles and toll stubs) items of content on the platform. If that number sounds a bit intimidating, then remember that we have sophisticated search tools (<3 minute video), including an open citation index, to help you find exactly what you are looking for.
It seems like just yesterday that we were preparing for our first COASP meeting in Paris. Now Stephanie is traveling to Amsterdam for the 2015 event which begins tomorrow.
A great deal has happened at ScienceOpen since then, here’s a quick rundown:
What do we hope to achieve at this year’s COASP meeting? Stephanie (our CEO) would like to chat with as many of our fellow publishers as possible about how you, like Thieme, can use our platform to raise the visibility of the OA research that you publish even further, which is of benefit to your authors and their careers.
CEO of ScienceOpen, Stephanie Dawson
As some, but not all, of you know, ScienceOpen has developed a Collections Tool which allows Community Editors to curate articles from any OA publisher in any way they choose.
We offer a safe and legal networking option for encouraging conversation around content, that complies with publisher policies. ScienceOpen invites those attending COASP to find Stephanie (@SDawsonBerlin) and get involved!
Peer review – the evaluation of scientific, academic, or professional work by others working in the same field – lies at the heart of scholarly communication.
At its best, Peer Review is a rigorous analysis with the aim of improving either the article itself or the science behind it and frequently both. At its worst, peer review is an obstacle course, that appears engineered to prevent publication or at best delay it by months, or even years!
This dichotomy of author experience plus the ever increasing publicity attached to retractions are both real issues in terms of faith in the process itself and public trust in science. It seems prudent then that we should act in a cohesive manner to see what improvements can be made whilst still acknowledging the important role that Peer Review plays.
Image credit: Peer Review Week organizers
Peer Review Week grew out of informal conversations between ORCID, ScienceOpen, and Wiley. Each organization has a different perspective on peer review, and has been working independently to better support its role in scholarly communications. Joining forces enables all three organizations to share their central message – that good peer review, whatever shape or form it might take, is critical to scholarly communications – more widely and powerfully. Sense About Science has joined the week to ensure the wider benefits of peer review – as a quality mark and tool for making sense of science claims – are shared with the public.
Our informal partnership will promote the first ever Peer Review Week, from Monday 9.28 thru Friday 10.2.
During this time we’ll be sharing stories, videos, participating in a Webinar on Trust and Transparency in Peer Review (Kent Anderson, Alexander Grossmann, Laure Haak, Andrew Preston and Verity Brown) and a Twitter campaign with the hashtag #peerrevwk15. We also invite other orgnanizations working in this space, such as The Winnower; PeerJ; F1000Research; BMC; Publons; PubPeer; and more to participate in this virtual campaign.
Here’s our position on peer review at ScienceOpen and we know that everyone doesn’t agree with us!
Our goal is to augment trust in the peer review process by making it entirely transparent. We facilitate Post-Publication Peer Review from named individual experts with 5 or more peer-reviewed publications listed on their ORCID to nearly 10 million open access articles and toll stubs currently available on the platform. We’re delighted to support this inaugural Peer Review Week.
EIO is an exciting young Open Access journal covering the gastrointestinal field and published by the award-winning international medical and science publisher Thieme. EIO joins two other Thieme journals (the American Journal of Perinatology Reports and The Thoracic and Cardiovascular Surgeons Reports) that already have collections on our platform.
Open Access means sharing essential scientific and medical knowledge as widely as possible and ScienceOpen is taking up this challenge to help authors and publishers get added visibility for their work. These articles are available for commenting, sharing, and Post-Publication Peer Review (PPPR), by experts with 5 publications on their ORCID on ScienceOpen. Every review receives a CrossRef DOI so each contribution can be found and cited which gives credit to the important work of reviewers, too.
CEO Stephanie Dawson has worked extensively on this pilot program with Thieme, specifically Frauke Gisela Ralf, Vice President of Open Access and Fiona Henderson, Director International Marketing. Here are Stephanie’s thoughts about the benefits of highlighting OA journals using the Collections tool on ScienceOpen:
“As a research article aggregator (peer-review reformer and publisher) we have brought together open and toll access content to demonstrate how nearly 10 million articles can be pulled together in different ways using our article collection tool with the goal of amplifying the best research results. One method is for a publisher to re-create a journal or highlight their best content on the platform, another is for a Society or individual community member to draw together papers in their area of research specialization or around a theme such as scholarly communication. The digital age offers unlimited permutations of content mash-ups and gives a voice to those with a story to tell, be they a publisher, a society or an individual community member.”
We are naturally delighted that Thieme Publishers have expanded their cooperation with us and look forward to speaking with other publishers at the Frankfurt Book Fair which this year runs from 14-18th October to see what we can do for them.
