Writing a good referee report for a journal article

[This article was originally posted on Politicsblog, of the journal Politics.]

Peer-reviewing articles for journals is one of the important professional contributions made by academics. But it can seem an unusual exercise when undertaken for the first time and it is a difficult art form to master. There are a number of extended resources online with useful guidance, from academics, editors, publishers, and the American Philosophical Association, and a set of links to discussions on aspects of peer-review on various blogs. Here I offer seven quick tips that are designed to help make a review useful to both an editor and an author:

1. Try to review the paper in the mindset of an umpire or jury member. It is not your job to consider which tactics you would have employed to play the game or argue the case. Your task is to consider whether the objectives set in the article are defensible – would an article that realised those objectives make a significant, original contribution to literature? – and whether the argument offered for the conclusions is sufficiently developed and persuasive.

2. Be specific about the merits and the problems with the paper. It is not helpful to say merely that “this is a strong article” or that “the theoretical framework needs tightening”. When praising a paper, identify aspects, sections, or arguments that were convincing and state their strengths, and if the theoretical framework needs tightening, try to be specific about why it is not sufficient for meeting the objectives of the paper and which exact components of it need improving. Similarly, if terms need clarifying, which terms and why? If arguments need improving, which step in the argument is problematic and what is wrong with it? If data analysis needs tightening, which particular interpretations or claims need attention and in what way are they unsatisfactory?

3. Be clear about the significance of the problems. It is helpful to specify whether the identified problems should be thought “major points” – points that the paper must address for publication to be appropriate – or “minor/other issues” – matters that ought to be addressed, but are not fundamental, matters such as tidying arguments that are not central to the overall thesis or moderating overly strong claims. This helps editors get a clear sense of where you stand and helps direct the author’s attention in undertaking revisions.

4. Be constructive. Although it is important to identify specific places in which the paper ought to be improved, your task is not simply to criticise it. It is particularly helpful to the author if you can suggest avenues or literature that might help frame or develop the argument. It is not a problem to suggest some of your own work in these recommendations, but peer-review is not an appropriate space for self-aggrandisement and editors tend to notice and be unsympathetic to reviews that engage in it. In this case, and more generally, make sure any suggestions are sensible and relevant to the paper. A good practice here can be to re-read your comments before submitting them and reflect on whether you would find them useful for developing the paper.

5. Be polite and professional. There is more than one way to make even the most critical comment and, particularly given the daunting nature of having one’s work subjected to critical review and possible rejection, there is no place in referee reports for language that is aggressive, angry, disdainful, or resentful. There is some good online guidance on what should be avoided and, again, re-reading and reflecting on how you would feel receiving the comments in your report can be helpful.

6. Make your recommendation appropriate. The selected recommendation for a paper will be taken seriously by an editor, so it is important to align it carefully with the good qualities of the paper and the significance of the identified problems. Many journals have rejection rates around 80-90% (philosophy journals average around 92%), so if you think a paper genuinely does offer a significant, original contribution that ought to be published, say so clearly to avoid your suggested revisions being understood too negatively. It is important to note also that not every objection constitutes a reason to reject a paper. On the other hand, if a significant, original contribution is not present and you cannot see how it would emerge, recommend rejection. Even if you feel that the topic is interesting and a subject on which original contributions can be made, it is valuable to distinguish between papers that warrant ‘major revisions’ and papers that warrant ‘renovation’ – for example, if an original contribution may be established if the paper were written in an entirely different way, it is, again, reasonable to recommend rejection. In offering any recommendation, it is important to remember that you are probably not the only referee and that the editor will not base her decision on only your views, so you should not overestimate the consequences of your selection. It is better to focus on making it appropriate and consistent with your review.

7. Be precise in your recommendation. Although journals typically offer referees a choice between a set of possible recommendations (such as Accept, Revise & Resubmit, and Reject), it is useful to remember that these options stand somewhat like points on a spectrum. Papers for which it is appropriate to recommend a Revise & Resubmit can vary considerably in the amount of revising they require. Accordingly, it can help to make use of confidential comments to the editor to offer a more specific verdict, to clarify, for example, if your Revise & Resubmit should be understood closer to the border with Accept or the border with Reject.