Combating the reproducibility crisis in research – Joe Akin

In the last Open Science Stars post, we spoke with Obinna Ojemeni who gave us an eye-opening account about the state of Open Access in Nigeria. We’re shifting gears again this week and delving into the murky world of reproducible research, a bit of a hot topic at the moment. Joe Akin of Scimpact was kind enough to tell us about how he is helping to make science more open and reproducible.

Hi Joe! Can you tell us a little bit about your background to get things rolling?

I have always been interested in science and technology. It was this interest that, in part, drew me to attend the US Air Force Academy and afterwards serve as a scientist in the Air Force Research Laboratory, supporting the demanding technological needs of the US Air Force. After completing my service and wanting to direct my scientific effort towards biomedical science, I pursued a PhD in immunology at Harvard University. Because of the university’s great breadth of scientific pursuits, I was able to find a lab where my previous expertise in engineered materials and biomaterials was useful within the context of immunology research—novel materials for cancer vaccine delivery.

When did you first hear about Open Access and Open Science? What were your original thoughts?

I first heard these concepts at the Council of Science Editors conference in the spring of 2015. I thought they sounded like laudable ideas, and I was particularly interested on two fronts:

• How to get scientists to change from convention and support a new paradigm?
• How to get publishers to do likewise, especially when it threatens the financial outlook for their institutions?

What was the rationale behind building Scimpact? How does this fit into your future vision for Open Science?

The impetus for Scimpact grew out of Girija and my frustrations, towards the end of our PhDs, in knowing that a lot of the hard work we had done would never be communicated to the larger scientific community and the prospect of many others duplicating the work we had done, needlessly. I believe it was an altruistic driver, from the outset.

What are the advantages of using Scimpact over traditional publishing models?

Scimpact aims to integrate with a lot of the current activity around making science more open. We are just one piece of a potential solution. We focus on the under-resourced element of making reproducibility of results transparent. We believe reproducibility can be the foundation for communication, rather than novelty.

We believe reproducibility can be the foundation for communication, rather than novelty.

Why is rapid publishing of results so important for research?

When you think about it, closing the cycle time of experiments directly impacts reproducibility. How hard is it to confirm an experimental result, especially in biomedical science, when the published experiment was conducted months, if not years previously? The original researchers might not even be able to do that, what’s more someone in a different lab.

How has the research community embraced Scimpact? Why do you think some are so reluctant to move beyond non-traditional communication methods?

Mainly due to Girija’s efforts, we have over a hundred followers on Twitter and have presented our idea at multiple conferences. Most recently, Girija was awarded a fellowship by Force11, an organization created to foster a new era of open science.

How do tools like Scimpact help to combat replication issues, or similar issues with a lack of transparency or reproducibility?

Our vision for Scimpact will allow all users to curate experimental replicates they are interested in, in effect crowdsourcing the reproducibility of published literature. A user of the platform would be able to view all the reproductions of an experiment and all the associated materials and methods with as little clutter as possible. Under the conventional model, materials and methods are often curtailed due to print word count restrictions. We believe this is some of the most interesting aspects of a story. Suppose a fictional experiment– why did two researchers obtain the exact opposite result from each other, ostensibly doing the same experiment? Was it the antibody, or something else?

What can you tell us about the Mini Reproducibility Project?

Mini Reproducibility Project aims to provide the first tool in our vision of crowdsourced reproducibility. It is a web extension that helps users curate single figure publications, adding important context to a growing resource of published data.

Why is it important to publish “negative results”?

Most bench scientists intuitively understand the answer. Negative results are still revealing the underlying natural phenomena. As a result of human nature, we artificially tend not to report these results, and scientists are generally on-guard for things that can lead to bias in interpretation.

How can we move academics away from the view that research that doesn’t give the results they want is “negative”?

My personal opinion is we are verging on a full-blown crisis in the scientific community. I read once in a book by Stephen Jay Gould, in one of his works on science history, that when scientists become too aloof from the society they live in, it hurts the individual researcher’s own science. I think this is true. At the end of the day, as much as it is satisfying to receive accolades and reveal abstruse cosmic phenomena, modern science is meant to be a public good. If there is no longer trust between scientists and lay people, the system grinds to a halt. When the public hear about scientific fraud and irreproducible results, this chips away at this trust. Scientists should embrace all results, not just because it could be part of the professional ethos, but because the knowledge they create impacts the society they live in. Scientists don’t pay attention to the results they publish at their own peril.

We are verging on a full-blown crisis in the scientific community

Does Scimpact help to challenge what some call the ‘publish or perish’ culture in academia?

Yes, our vision intends to allow recognition for all sorts of activities that are normally not recognized in conventional publishing. We are really interested in Project CRediT that grew out of joint industry and government involvement.

As much as it is satisfying to receive accolades and reveal abstruse cosmic phenomena, modern science is meant to be a public good.

How important do you think it is for researchers to be able to publish early, easily, and rapidly?

Very important. In the academic apprenticeship system, an individual scientist develops her habits directly from the habits of her advisor. Allowing for positive feedback from generating, disseminating and discussing her own work can complement the formative apprenticeship.

Scientists should embrace all results, not just because it could be part of the professional ethos, but because the knowledge they create impacts the society they live in.

How can tools like Scimpact work with platforms like ScienceOpen to foster a more open, sharing, research culture?

There is a lot of exciting activity in the area of open science. As I referred to the CRediT taxonomy, we have also spoken with people from Hypothes.is and eLife. It is an exciting time for all of these ideas and more. We are excited to contribute to this evolution of science communication.

How do you see the future of scholarly communication? What steps do we need to take to get there?

At the fundamental level, your earlier question hit the target: how do we change the engrained human behaviour surrounding scientific output, bound by conventional publication? Or, as I alluded to above, as the gap grows larger between the scientific elite and society at large, science becomes less useful and distrust sets in. Changing such behaviours is exceptionally hard, especially when left too long. I can’t think, off the top of my head, of a super successful case. I strenuously support science education at all levels.

What one piece of advice would you give to students looking to pursue a research career?

Follow the science you find most interesting and enthralling—I use the last word, because you almost have to be compelled by your inner interest, to overcome a lot of the obstacles that can emerge. But, really, what area of life doesn’t present significant obstacles? If it’s too easy, you’re missing out.

Joe Akin is currently working as a scientist in the bio-pharmaceutical industry. He has worked for many years as a bench scientist first in the US Air Force and most recently completing a PhD in immunology at Harvard University. His work focuses on cancer vaccines using approaches learned from his cross-disciplinary background in materials engineering and immunology.