A week ago, a scientific journal asked me if I wanted to evaluate an article (peer review). Since this is the first time that I was asked, I was flattered. A second later, I felt a responsibility to contribute to science. In order to estimate the value of the project, I quickly scanned the website of the journal, checked the review request and let the editor know that I would not do the review. My decision occupied my thoughts for the following days, until I realized that I first had to decide for myself what I consider peer review promoting open science. What is open science peer review ?
I am an advocate of open science and transparency, so as a reviewer I would need at least a plan of the study (protocol) and the data so I can check that the planned research is carried out and that results arise from the data.
I think it is also important that:
• the publication is freely accessible (open access), especially if the research was funded with public money. A community that pays for research has the right to view its results.
• the authors state they share the dataset with the scientific community. By reusing existing datasets science can progress quicker with no unnecessary duplication of efforts.
• the protocol is available as an addendum. Today most articles are published using the internet so a protocol, extensive analyzes, etc … can easily be provided as an additional download.
• a ‘film credit’ clearly identifies who did what. Who had the idea for the study, who sought funding, who did the study, who analyzed and who wrote the paper ?
• the paper mentions the funding: who paid what amount and to whom ?
• identification of conflicts of interest 1) by the authors: any stocks or consultancy work ? 2) by the journal: What sources of revenue has the journal, what reprint orders (reprint of articles) were done for the industry and for what amount ? Editors should mention anonymously any commercial pressure exerted them, their stocks, etc ….
From this list, is becomes clear that a critical evaluation of a research paper is an undertaking. Probably not a single paper meets these criteria. So if I ever want to do a peer review, I have to make a priority list. For me, the absolute minimum to start the review process is a protocol, a dataset and if public money is used, a guarantee of a free, accessible publication.
I want to know what you think. Write a comment below or a brief comment on Twitter or Facebook, and together we will broaden the debate.