Project topics and materials
Get Complete Project Material File(s) Now! »
Publishing and the peer-review process
When scientists finish a study, they summarize and present their findings in a report. This report needs to be communicated to the appropriate audience so that it updates the scientific knowledge of all interested parties. Traditionally, the role of dissemination of scientific reports (papers) is assumed by the scientific journals, which are focused on publishing research on either a specialized or general topic.
Authors usually submit to journals that may maximize their audience and journals need to select papers that may maximize the interest and size of their audience.
Thus, journals implement screening techniques to make sure the content they publish is valid, relevant and interesting to their audience.
Submitted manuscripts first pass through an in-house screening process, during which an editor is assigned to each of them with the responsibility to decide whether a paper is relevant for the journal. In many journals, this decision is taken at a periodical meeting of the editorial board aided by the opinion of the handling editor. This procedure is usually fast, ranging from some hours to a few days. If a paper is rejected, then it may be resubmitted to another journal. If not, then the editor contacts through email other scientists, who are experts or have previously published in the same topic as the submitted manuscript, and asks them to review it. The invitations, which the editors send, usually contain only the abstract of the paper, and based on that, those receiving them should decide whether they want to review the paper or not and notify the editor of their choice.
Candidate reviewers may reject an invitation to review for various reasons such as not having enough time, the paper is not in their field of expertise etc. (Mulligan et al. 2013). Those who accept to review usually do so as volunteers, even though some journals may choose to reward them with some form of discount coupons.
Moreover, some journals publish an annual list of their reviewers’ names to thank them. Reviewers may also gain credit for their reviews through certain online recognition platforms e.g. Publons (Review rewards 2014; Warne 2016). Peer review usually is single or double blind, meaning that authors do not know the identity of the reviewers while the identity of the authors may either be hidden (double blind) or not from the reviewers (single blind). The rationale for this, is to minimize the possibility of receiving biased reviews, due to the author’s seniority, or even retaliation from authors who received critical evaluations of their work towards the reviewers.
The editors, after gathering enough review reports (typically between 1 and 3, sometimes more), take a decision on whether they will reject the paper or ask the authors to make modifications based on the reviewers’ comments and resubmit the manuscript for further evaluation. Papers which are not rejected, are re-evaluated and re-revised as many times as necessary until a final decision for rejection or acceptance is made, though more than 2 or 3 review rounds are not frequent.
Accepted papers are included in a next issue of the journal, but nowadays they are very often uploaded on its website much earlier than publication in print.
Depending on the domain of the paper, the whole procedure may span from a few months to more than one year, whereas a single review report usually requires only a few hours to be completed.
Figure 1: Diagram of the most typical way peer review is currently being held. One can see how authors, journal editors and reviewers interact with each other in the peer-review system (Sense About Science 2012).
Criticism of the peer-review system
Peer review has been recently debated and criticised (Gura 2002; Smith 2006; B.Alberts et al. 2008; Stahel and Moore 2014; Rennie 2016; Csiszar 2016). The huge increase in scientific manuscripts has increased the demand for peer review and potentially introduced a significant burden to the scientific community with a risk of downgraded quality standards on the review reports. It has been stated that, overall scientists need to devote tens of millions of hours per year to perform peer review in biomedicine alone, from which most of it is potentially redundant (i.e. multiple reviews for papers that have already been reviewed once) (Kovanis et al. 2016). In addition, the ability of the system to detect mistakes has also been challenged. For instance, a randomized controlled trial conducted by the British Medical Journal showed that most reviewers could spot 2 to 3 out of 9 major methodological errors in a paper, even after training (Schroter et al. 2004).
Another study showed that psychological journals may reject articles that were already published by them, when they were submitted back to them with slightly altered content (Peters & Ceci 1982). Finally, peer review is estimated to cost billions of dollars to scientific institutions annually due to the amount of time that scientists devote to it instead of their normal research activities. (Look and Sparks 2010).
The peer-review process may also be gamed by scientists with unethical motives.
A resent scandal of fabricated review reports led to multiple retractions in many journals (Cat et al. 2014; Callaway 2015; Cohen et al. 2016). The scam was revealed when the editors realized that some authors who proposed well-known scientists as reviewers of their papers, were in fact providing them with fake email addresses leading back to themselves. Then, the authors provided very favourable reviews for their own papers, trying to maximize the probability of acceptance.
Moreover, another scandal, involving papers published in scientific conferences, revealed that more than one hundred papers, which were automatically generated by a computer had passed through their peer-review process (van Noorden 2014).
After the adoption of open access by the scientific community, various journals that have been considered to be predatory have appeared (Sorokowski et al. 2017). In 2013, Science’s editorial staff conducted an experiment and sent to many open access journals around the world a fake paper (Bohannon 2013). While journals should have rejected the paper on sight, many of them accepted the paper sometimes almost instantly after submission. All journals claimed that the paper passed through peer review but without providing any evidence of it. The journals, then, asked the author to pay the regular open access fee, thus aiming to earn easy money by publishing anything.
Interventions and studies on the peer-review system
Even though the implementation of the peer-review process is standard in science, there have been many other alternatives proposed and implemented to some extent.
Some journals have experimented with and applied double-blind or open peer review (Blank 1991; van Rooyen et al. 1999; Pöschl 2012; Hopewell et al. 2014; Pontille & Torny 2014; Bruce et al. 2016). Journals have also experimented with providing review training to new authors, because critically appraising a scientific manuscript is considered to be a separate skill than writing one. (Schroter et al. 2004; Houry et al. 2012). Finally, it has also been proposed that manuscripts may be split into parts, such as reporting of outcomes or statistics, which may be reviewed independently by specialized reviewers. Some journals already require an independent statistical review of submitted manuscripts.
Apart from these micro-scale interventions, macroscopic changes have also been proposed and implemented (Walker and Rocha da Silva 2015). First, a system widely adopted in the domain of physics and mathematics, which is the immediate publication of manuscripts before submission to a journal. As described before, in these fields, when authors write a paper they usually upload it on ArXiv and then they submit it to a journal where they follow the standard peer-review process. Thus, all fellow scientists can read the work, discuss and comment on it before publication by the journal. Second, post-publication peer review has gained a lot of support recently (Herron 2012; Hunter 2012). This alternative system allows other scientists to comment and review papers after they have either been posted online or accepted by a journal. Journals currently implementing this system are f1000Research, eLife, the journal of Atmospheric Chemistry and Physics (ACP) etc. Third, another alternative system is one in which an article, after rejection, is resubmitted to another journal together with the previous journal’s reviews (Cals et al. 2013; van Noorden 2013). This system aims to eliminate redundant reviews and speed up the decision process.
In the next chapter, I describe my 1st paper, which is about estimating the burden that peer review has been posing to the scientific community. The third chapter contains the description and results of my agent-based model of the current and alternative peer-review systems (2nd & 3rd paper). Finally, the fourth and fifth chapters contain the general discussion and conclusions, respectively.
Table of contents :
Introduction
Objectives
The system of scientific publications
Publishing and the peer-review process
Criticism of the peer-review system
Interventions and studies on the peer-review system
Paper 1 – The global burden of journal peer review in the biomedical literature: Strong imbalance in the collective enterprise
Summary
Article
Modeling the peer-review system
Introduction
Agent-based modeling
Models of peer review
Paper 2 – Complex systems approach to scientific publication and peer-review system: development of an agent-based model calibrated with empirical data
Summary
Article
Paper 3 – Evaluating alternative systems of peer review: a large-scale agent-based modeling approach to scientific publication
Summary
Article
General discussion
Summary of the results
Limitations
Impact and implications
Perspectives
The future of peer review and scientific publication
Conclusions
References
