Skip to main content

Information Literacy Archaeology: The peer review process

What is peer review?

Researchers record the results of their work in academic articles. The aim of these articles is to inform other researchers about research findings and to lay claim to the findings as new and original insights. Such claims are not usually acknowledged by other researchers until the article has been published in a peer-reviewed journal.

What is peer review? Peer review is a form of quality assessment. In this context, the word ‘peer’ (meaning an equal) refers to a fellow researcher. A peer review is therefore an assessment by a fellow specialist in the relevant field. Peer reviewers assess whether an article meets certain requirements, such as originality, accessibility and solid substantiation of the results. All academic journals work with a system like this in order to assure the quality of the articles they publish.

If an article is peer-reviewed, you can assume that the information is reliable and of the required standard. In many bibliographic databases you can search specifically for peer-reviewed articles.

Peer review in 3 minutes

The peer review system criticized

The peer review system has come in for quite a bit of criticism lately.

Criticism focuses on:

  • Slowness: the peer- review system slows down the publication process by months, sometimes by more than a year
  • Clash of interests: sometimes the independence of reviewers is called into question. In small disciplines it is often quite easy to discover who the author of the article is. This makes it possible for reviewers to obstruct the publication of others. Sometimes self-interest comes into play, when the reviewer stipulates that the author must refer to a certain 'essential article' (which happens to be written by the reviewer)
  • The system takes up a lot of (unpaid) time of sought-after reviewers
  • Highly innovative, experimental research and associated publications are less likely to be published because it differs from standard research and is therefore considered to be non-publishable by reviewers
  • Strict requirements as to the use of English can be a barrier for non-native speakers and impede international publication of what may be excellent research
  • Also the importance of peer-review to the academic process is called into question by some academics

There are websites where academics share their experiences with peer-review of journals, the Dutch initiative SciRev is one such website.

Some journals, such as PLOS One (Public Library of Science One), experiment with new forms of peer-review. Here reviewers only indicate whether the research methods used for an article have been satisfactory. The assessment of scientific merit occurs afterwards, by the readers. By using this method many more articles are published in a much shorter period of time.

How does peer review work in practice?

Step 1

A researcher (or research group) submits an article to the editor of an academic journal that is most relevant to the subject of the article. The editor gives the article a cursory reading and determines whether the article is of the required quality, contains new insights and is well written.

Step 2

If the editor decides to proceed with the article, two or more peer reviewers must be sought. The reviewers may be the authors of articles previously published in the journal, or other researchers who have an established reputation in a particular research field. The editor will always look for reviewers who are fellow researchers and experts in the subject of the article, but he/she will also aim for diversity. This diversity aspect can be problematic; in a relatively small discipline few fellow researchers will have the required specialist knowledge.

The selected reviewers will receive a letter or e-mail asking them to review the article by a certain deadline. The reviewers only get to see the title, authors and occasionally the abstract of the article. In general researchers are happy to oblige, partly because their own status within the research field is enhanced in this way.

Step 3

The editor will eventually receive two or more reviews, which are then sent anonymously to the author(s) of the article. On the basis of the reviews, the editor decides whether to reject the article, ask the author(s) to make major or minor revisions, or to accept it without further action. In most cases, authors are required to rewrite part of the article.

Step 4

The revised version is resubmitted by the author(s), along with a response to the reviewers’ comments. The editor will then reassess the comments, the response of the author(s) and the revised article. The article may still be accepted, but more often than not it is sent back to the reviewers for further comments. This means that an article may be revised several times before it is ultimately accepted or rejected.

Other peer-rview systems

Other peer-review systems now in use are:

  • post-publication peer review: peer-review by means of blogs and twitter posts
  • open peer review: a completely transparant form of peer review, based on cooperation instead of the traditional double-blind peer review. In the Humanities the Shakespeare Quarterly is an example of a journal using this kind of peer review
  • alternative metrics, of altmetrics. Here an individual article is evaluated instead of the entire journal. This form of peer review occurs mostly in the medical and exact sciences