Towards five stars of transparent pre-publication peer review

Inspired by the new OASPA, DOAJ, COPE and WAME “Principles of transparency and best practice in scholarly publishing“. I like to zoom in on their first point. The peer review process. The following is stated on peer review:

“All of a journal’s content, apart from any editorial material that is clearly marked as such, shall be subjected to peer review. Peer review is defined as obtaining advice on individual manuscripts from reviewers expert in the field who are not part of the journal’s editorial staff. This process, as well as any policies related to the journal’s peer review procedures, shall be clearly described on the journal’s Web site.”

I don’t think this is statement alone is sufficient to improve transparency of the peer review process. I have seen many journal that patiently state that their journal is either blind or double blind peer reviewed. But there is not a shred of evidence for this being the case. In my role as subject chair for Scopus I review a lot of journals. I have therefore thought quite a bit about this subject. I would propose a five star system for the transparency of the peer review process. Inspired by the five stars of linked (open) data as drafted by Tim Berners Lee. I wish to present the 5 stars for peer reviewtransparency.

1 *: Providing clear dates of submission, revision, acceptance and publication
2 **: Listing the reviewers involved once a year
3 ***: Providing a yearly overview of submissions and acceptance
4 ****: Naming the handling editors and reviewers per article
5 *****: Publishing the review reports online alongside the final article

1 *: Providing clear dates of submission, revision, acceptance and publication
It seems obvious, and many journals do mention at least the date of submission and the date of (electronic) publication. But there are still many journals that don’t provide these basic dates. Research by Björk and Solomon (2013) showed that only 64% of the journals indicated submission and acceptance dates. In addition to those two mentioned dates, transparent journals should list the dates of revision, electronic publication ahead of “print” and the official date of publication in an issue. Björk and Solomon are mistaken that the last date could be derived from the issue information. The journal of food chemistry’s May 2014 issue is completely available online at the end of December 2013. The later practice is a typical publishers’ practice to game the Impact Factor system.

2 **: Listing the reviewers involved once a year
Journals normally list the editorial board members on the journal’s website. But external reviewers are essential in the peer review process. If there are no external peer reviewers involved, the journal is review through editorial peer review, which is something completely different. Even if journals decide not to indicate the reviewers on an article basis, they should present a list of external reviewers (names and affiliations) in a yearly issue to thank them for their effort, but also to show to the public that external reviewers are involved. We can than study to what degree external and capable researchers are involved in the journal. If the pool of reviewers is really small, the net cast very narrow, we certainly don’t deal with an international journal. It goes without saying that these articles with the yearly listing of reviewers involved are linked on the editorial pages of the journal and are open accessible.

3 ***: Providing a yearly overview of submissions and acceptance
One star more in the transparency of peer review process is the an actual overview of yearly number of submissions and accepted papers. A good journal should also self report the a yearly summary of statistics provided for the first star, thus the average time period from submission to publication and preferably the in between periods as well. For gold OA journals an overview of the number of actually paid submissions and waived APCs should be provided as well. This number should be taken so serious that they could (and should) be accompanied by a declaration of an independent accountant. Similar to the annual financial report of the publisher, only the difference is that it should be reported on a journal level, open and available to the readers and potential authors. It goes without saying that the editorials such as these are linked to the general journal information pages. Examples with practices suggested here are less prominent. CMI Clinical Microbiology and Infection is a good example. Christian Gutknecht (pers. comm.) provided the example of APA which comes a long way, but should be reported on the individual journal and the Elsevier’s journal insight pages give some useful insights.

4 ****: Naming the handling editors and reviewers per article
Mentioning the handeling editor such as eg. in Hydrobiologia is the before last step towards a fully transparent peer review process. A important step further in this process is Biology Direct where editors and reviewers are revealed, and even the editorial board members who suggested the reviewers. It should be noted that other articles in Biology Direct have open peer review (5 star transparency). The idea behind the revelation of the peer reviewers is that it improves the quality of the peer reviews. The reviewers can’t hide behind anonymity.

5 *****: Publishing the review reports online alongside the final article
Complete transparent peer review are peer review reports that are available together with the final published article. Open peer review is gaining traction, but the number of journals embracing this model is still very small. Copernicus Publications and Frontiers are two of the publishers deserving special mention in this respect. Other examples include BMJ Open which includes the original submission and the reviewers comments (which makes excellent study material for peer review courses). It is still needed though that the journals include the 3 start report on their editorial pages as well.

Are these five stars for transparency of the peer review system really needed. I think the Bohannon (2013) scam is pointing to the lack of good peer review practices by many (young) publishing houses. These stars for transparency of peer review might be a help for journals to improve and upgrade their peer review system and for authors an opportunity to make better informed decisions where to submit their articles. This listing should be regarded as a proposal, comments and suggestions are welcome.

References
Björk, B.-C., and D. Solomon. 2013. The publishing delay in scholarly peer-reviewed journals. Journal of Informetrics, 7(4):914-923. http://dx.doi.org/10.1016/j.joi.2013.09.001 OA version at http://openaccesspublishing.org/oa11/article.pdf

Bohannon, J. 2013. Who’s Afraid of Peer Review? Science, 342(6154):60-65. http://dx.doi.org/10.1126/science.342.6154.60

4 thoughts on “Towards five stars of transparent pre-publication peer review

Comments are closed.