At some point in almost every scientist’s early career, he or she is asked to peer review a research paper for a journal. Peer reviews are crucial to maintaining a high standard in research and should not be taken lightly. Ultimately, what you’re doing is making a recommendation to the editor, who may or may not have the expertise required to evaluate the work being done. If you’re lucky, your advisor will guide you through this process. However, with very few opportunities for formal training in reviewing manuscripts, many are left to figure it out for themselves, and can end up feeling lost and intimidated.
This post will guide you through the main points of writing a good peer review. To gain some professional insight on the process, I spoke with Dr. Patrick Herendeen, the Editor-in-Chief of the International Journal of Plant Sciences.
Accepting the Invitation to Review
The first thing to ask yourself when being invited to review an article is whether you are the right person for the job. The two major considerations here are time and expertise. Do you have the time to spend several hours reading over and developing a position on this paper? While editors typically allow for some wiggle room on deadlines, it’s best to cultivate a good professional reputation by having your review in on time.
How about expertise? Do you have enough familiarity with the field of study and the methods being used to be able to critically evaluate them? This can be a stressful question for early career scientists who aren’t very confident in their own abilities yet. However, Dr. Herendeen says, “if it is a reputable journal, the editor has probably spent some time investigating potential reviewers and thinks you have relevant background.” So to some extent, you can consider the invitation itself a vote of confidence. “If you are not certain whether you have the background, you can always ask the editor for clarification. If you get into the manuscript and find that it really hinges on expertise that you do not have, just tell the editor. You may still have useful feedback to provide, even if it does not get at the core issues.”
How to Critically Read a Manuscript
Once you’ve made the choice to review the paper, it’s time to get down to business. Grab a pen and a notepad and read through carefully while noting any questions or concerns you have. Or print the article out and scrawl notes all over it, as I do. Here are some points to consider and make notes on while you read:
Overall organization
- Is the paper well-organized, with the various sections properly separated?
Introduction
- Do the authors provide a reasonably thorough summary of the literature?
- Has the research question, hypothesis, and/or predictions been clearly and concisely stated?
Methods
- Is the methodology clear and complete enough to be reproducible?
- Are the statistical and analytical methods presented in a complete and clear format?
- Have the authors provided a sufficient explanation that assumptions of the statistical methods or models that they have used have been adequately met?
Results (including figures and tables)
- Are the results clearly and completely described in an organized manner?
- Are tables and figures used effectively and appropriately?
Discussion
- Are the questions posed in the introduction all addressed by the end of the article?
- Have the authors effectively explained their results in the context of the literature?
- Has the study helped to advance knowledge, and if so, have the novel results been appropriately highlighted?
- Have the authors extended their conclusions beyond what their data actually shows, and if so, have the potential limitations of the study been adequately discussed?
- If the study is applied in nature (i.e., the results are relevant to management or policy), have the authors proposed relevant and reasonable recommendations based on their results?
Overall assessment
- Does the paper fit the scope of the journal?
- Does the author’s interpretation of the results make sense? Are you convinced by the conclusions being drawn?
- Is speculation clearly delineated as such?
- Is the article well-written, concise, and easy to understand?
- Is the research both original and relevant? Does it advance science?
Formatting Your Review
Peer reviews typically have three sections. First, a peer review should begin with a brief summary of the research done, its conclusions, and its contributions to the field. This assures both the editor and the author(s) that you have read and understood their work. Second, go through any major criticisms and “big picture” general comments you have, based on the questions above and any other significant scientific or structural problems. Third, you can then move on to minor issues such as labelling, formatting, references and citations, grammar, and so on. You’re not expected to be a copy editor for the article, but if you notice some small mistakes as you go, particularly those related to scientific areas such as misspelled species names, editors will appreciate your mentioning them. If the spelling and grammar are a complete mess, you can simply note that the paper would benefit from further editing. And don’t forget to spellcheck and proofread your own review!
Since peer review is at its essence a critical assessment of others’ work, striking the right tone can be tricky for new reviewers. No one wants to be the dreaded ‘Reviewer #2’: a reviewer who is overly harsh and critical in a non-constructive way. It can also be hard to criticize the work of scientists who are older and more experienced than ourselves. Herendeen suggests that you “try to put yourself in the place of the author. Provide constructive feedback that you would like to receive.” Ultimately, providing a thoughtful, constructive, and detailed review will help the authors to improve their manuscript. Remember- everyone’s goal here is to produce a good publication that will be read and cited by other researchers.
Your Recommendation
Of course, the big question is, what will you recommend? Your options, in most cases, are ‘Accept’, ‘Accept with minor revisions’, ‘Accept with major revisions’, or ‘Reject’. Accepting a paper with no revisions at all is rare. More likely, you’ll find at least some small issues or necessary changes to the text, such as a concept that needs further explanation or a conclusion requiring clarification. But what if more than this is necessary? You have the option of recommending major revisions, which can include further experiments or a substantial re-write and resubmission of the paper. When submitting your recommendation, certain journals will ask you if you are willing to review a revised version of the manuscript; if possible, agreeing to this is extremely helpful to the editor, since they will have continuity among reviews and since you are already experienced with the study design and paper itself—it also typically means less time required by you to review the paper an additional time.
Not sure whether a paper requires major revisions or should just be rejected? “If the paper has something of value in there that could be published after major problems are fixed, then ‘major revision’ might be a better recommendation,” suggests Herendeen, “The editor can still decide to reject even if both reviewers say major revision. If the paper requires substantially more data, the experimental design is fundamentally flawed, or there is some other major problem, then it is best to recommend reject.” Remember, too, that for some journals with high rejection rates, sound science may not be enough; an article will need to present a significant original finding. In all cases, the paper must be a good fit for the journal, in both topic and scope.
A Word on Ethics
Due to both the influence the reviewer wields and the need for discretion surrounding unpublished data, peer review can be a bit of an ethical minefield. To make sure you don’t step on one, there are a few things to keep in mind.
First, confidentiality. You are being trusted with unpublished data that, in the case of rejection or major revisions, could end up not seeing the light of day for a while. You should not be discussing the manuscript or its conclusions with anyone but the journal editor prior to publication, and you should above all not attempt to use any delays to your advantage.
Second, conflicts of interest. If you truly have a conflict of interest with one or more of the authors, you should not review the paper. This includes both positive and negative biases. In smaller fields, it can be difficult to find a reviewer who both has the expertise to review a paper, and has not had personal contact with the author(s). So what should you do if you know the people behind the research? As Herendeen puts it, “The question is one of objectivity. If you believe you can review the manuscript objectively and provide honest feedback […], then there is no problem. If an editor asks you to review a paper for a close colleague, you might want to let him/her know this, but say that you feel comfortable providing objective feedback, and leave it to the editor to decide.” So go with your gut, but let the editor know.
Finally, anonymity. Some journals will allow you to sign your review, others won’t. Though you may feel compelled to sign your work, Herendeen recommends against it. “I would be cautious, as a junior scientist trying to get established, on signing reviews, especially critical ones, and especially if you do not know how the authors will respond.” Play it safe. Stay anonymous.
Feeling like the fate of someone else’s research is in your hands can be stressful when you’re a relative newbie to the field, but know that you got the invitation because someone thought you had the chops for it, and do your best, because you’ve been entrusted with one of the most important safeguards of the scientific process.
