How to respond to an R&R

[Note: Some of the follow suggestions I picked up from my faculty advisors and colleagues while in grad school, others I came up with on my own, and still others came from Wendy Belcher's excellent book: Writing Your Journal Article in 12 Weeks]

So, you submitted a manuscript to a journal, and the editor liked it enough to give you an R&R... now what?

First, congrats! An R&R is about as good as you can expect for a regular journal, since very few manuscripts are accepted 'as is.'

Depending on the editor's letter and reviewers' comments, it might be an easy R&R that won't take much time, or a very hard R&R that will take weeks or months to address. Also, depending on the journal, an R&R might mean the manuscript is basically accepted, pending a little polishing, or alternatively that there's a good chance you'll still get rejected later on in the process. In any case, your goal should be to address as many of the recommended changes as is reasonably possible in the revision to make the paper better, without accidentally 'messing up' your paper (or your main points) in the process. Hopefully this post helps guide you there.

So the first thing you should do is read the editor's letter in its entirely (you probably opened the letter/email and just skimmed the page(s) until you saw 'revise and resubmit'...).  The editor's job is to read the reviews, make a decision on your manuscript (in this case, R&R), and then (in my opinion) give you guidance on what are the most important things to address in the revised manuscript. If the editor didn't do that, or gave you a generic 'please address the reviewers' comments in your revised submission,' then your job will be a little harder than it probably should be.

Anyhow, read the editor's letter and see if s/he gave you specific instructions as to what to address. The editor might note the important of a missing variable in your model, as mentioned by reviewer 2, or that your treatment of the prior literature was lacking in X area, as noted by reviewers 1 and 3. Whatever the editor says in their letter regarding potential changes should be your first priority in the revision. What I like to do is open a new word document and make a bulleted list of the changes the editor specifically mentioned in the letter (these bullets go under an 'Editor' heading). This will become part of your 'potential revision' list/spreadsheet that you will continue building in the future. (More about this in a bit).

Next, read through all of your reviewers' comments, without taking any notes. Make sure you have your submitted manuscript draft in front of you, so you can reference it when a reviewer mentions something on page X or in Table Y. Keep an eye out for comments that come up from more than one reviewer, such as the suggestion to use robust standard errors in your model, to provide a correlation matrix table, or to cite the work of authors A and B.

Chances are, you won't agree with everything the reviewers recommend, and you'll form the following opinions about the reviewers [note: for ease of comparisons, all three reviewers are assumed to be male in this example]:

Reviewer 1: Even my
bowel movements are
high impact.

Reviewer 1: You'll probably think this reviewer is a genius, particularly since he gave you the most useful recommendations, and other than a couple minor suggestions, loved your entire manuscript. His suggestions are relevant, fairly easy to implement, and will really help boost the quality of your manuscript. This reviewer, while rarely seen in the wild, is the best you can hope for.

Reviewer 2: "Derp?"

Reviewer 2: You'll probably think the second reviewer is an idiot, but a fairly harmless one. He probably doesn't really know anything about either your analytical methods or the literature related to your topic, but tried to fake it so the editor didn't get angry. Most of the suggestions he offered aren't very useful and/or make no sense given your model, argument, and findings. Still, you can easily address his main concerns, and easily dismiss his unhelpful and misguided recommendations.

Reviewer 3: My comb-over
is horrible, I can't eat sweets
due to my diabeetus, and I'm
going to take my anger out
on you and your manuscript.

Reviewer 3: Upon reading reviewer 3's comments, your first thought will probably be "what a f*cking d*ckhead." You'll probably think that every suggestion by reviewer 3 is complete garbage, that he misunderstood your entire argument, knows nothing of the methods you used, and decided to use his "anonymous" review as an outlet for his deep-seated inner frustrations concerning sexual impotency and male pattern baldness. You'll likely question his academic and genetic pedigree, and wonder how the editor could have chosen such an arrogant jerk. This reaction is completely normal, and has a long historical precedent.



In fact, Einstein even developed a theory related to reviewer comments that largely supports these three general categories, as previously noted in my post on how to get over manuscript rejections:

More useful than relativity.

However, these categories are not exhaustive, nor are they exclusive. You may, for instance, get a review from a reviewer that combines traits of 2 (idiot) and 3 (jerk).

 Regardless of the types of reviewers you get, or the content of their reviews, the fact that you received an R&R meant the editor saw some promise in your manuscript. Therefore, now that you've surveyed all of the reviews, you need to get to work addressing the specific suggestions and critiques, and revising your manuscript for resubmission.

So the next step is to go back through the reviews and add to an excel spreadsheet (or equivalent) every specific suggestion or critique, from each reviewer, that requires you to change something (or might require you to say why you didn't change something.) Each suggestion/comment should be its own entry on a separate line. I recommend five columns in the spreadsheet: ID # (increments up), Reviewer (who made the comment), Comment (what was the change/critique), Importance (How important does this seem to your overall revised manuscript, or to the editor/reviewer?), and Ease (How long and/or how much work is this going to take to address). I also like to color code the importance and ease cells using the an easy to distinguish color scheme - red for important/hard, yellow for medium/moderate, and green for low/easy. If you set it up like this, your spreadsheet should look something like the following (excluding the earlier comments from the editor):

Once you've finished adding the reviewers' comments to your spreadsheet, add the editor comments you previously listed out if they're not already covered by a reviewer, and mark anything the editor said to do as high importance. If the editor says something a reviewer already said, then add 'editor' to the reviewer cell as a reminder, and mark it as high importance regardless of your original importance rank. If the editor specifically says something is not important to address, then mark it as low importance, and if they say not to do it at all, then change the font style for the associated entry to strikeout.

After everything is down on your spreadsheet, you can get a sense of the overall work that's ahead of you by looking at the importance/ease categories for each entry. A roadmap to these combinations, and what they mean for your revision, is below:

High/Hard: These are the changes that will give you the most headaches. You need to do them (high importance), but they're going to take a long time and generally be a P.I.T.A. (hard to do).  Doing these first will make your life much easier later on, and if they're major changes, some of the less important changes may be obsolete once you do these. If you decide not to do one of these, you need a very good reason why not.

High/Moderate: These are changes that will be a little easier than the High/Hard changes. You still need to do them, and they're going to take you some time, but they won't be absolutely horrible to address.

High/Easy: These are the changes that will make you feel most productive, since you can address these important 'need to do' changes very quickly and without much work.


Medium/Moderate: You should probably do these changes, especially if you have the time.  If you don't do one of these change requests, then you should definitely address why you didn't in your cover letter (more about the cover letter in a minute).


Medium/Easy or Low/Easy: Since these are easy to do, you might as well do them, especially for the ones of medium importance. If you choose not to do these, you should at least acknowledge why you're not doing the medium importance ones, if not the low importance ones as well (since not doing the latter will make you look lazy).

Low/Moderate or Low/High: These might be changes you decide not to do, given their relatively low importance compared to the moderate-to-high amount of work they'd require of you. Again, you may want to address why you didn't do these in your cover letter, but given their low importance, if you miss discussing some of these it probably won't be the end of the world.


Ok, so you have your potential changes all in order, you've ranked their importance and ease, you know which ones you need to address, which you should address, which you can easily address, and which you're not wasting your time with. Now comes the hard part: Make these changes in the manuscript. That's about as specific as I can get for this step, but this is what will take the longest.

Once you're done making (or not making) the various changes, you need to reread your manuscript from start to finish to make sure things still make sense. This is especially true if you had to do some serious changing of text. Edit as necessary.

"Dear Editor: Thank you for
the opportunity to revise
my manuscript. Unfortunately,
as I am not fluent in dumbass, I
have been unable to interpret
Reviewer 3's critiques."

After your revised manuscript is re-polished into a final version, you need to draft your cover letter to the editor. I recommend starting the letter with a brief paragraph thanking the editor for the opportunity to revise and resubmit your manuscript for their journal. You should also thank the editor and reviewers for their helpful suggestions and critiques (if even you think most of the suggestions were garbage). Then I recommend stating something along the lines of "I've provided specific responses to each of the suggestions and critiques below. I look forward to hearing your final decision." This will provide a transition between your formal paragraph and the subsequent list of changes that's about to follow, as well as obviate the need to write a closing paragraph after your list.

Skip a line or two, and then list out each change you want to address (all of the high importance ones, most/all of the ones of medium importance, some/most of the low importance ones). Briefly summarize the suggestion or critique, identify who said it, its location in the manuscript, and what you did to address it (or, alternatively, why you did not address it). Examples of wording for this section are provided below:

Reviewers 1 and 2 recommended adding r^2 values to Table 3 (p. 21). This has been done for all three models.

 Reviewer 2 recommended revising the paper to focus on racial threat theory. While this is a valid alternative theoretical focus, I feel that racial threat is not appropriate for the current study given the current focus on non-racial factors, as well as the comments of Reviewer 1 regarding the appropriateness of the current theoretical model. Further, using racial threat would generate the problem of..."

 Reviewer 2 questioned the need for spatial lag measurements in the final model. I have expanded the justification for this measurement on page 4, but believe that the inclusion of this measurement is essential in order to control for the spatial autocorrelation of the observations that was indicated by the Moran's I tests discussed on page 12, footnote 3. As such, this measurement has been left in the final model.

Reviewer 3 recommended deleting the third paragraph on page 2, as it was considered unnecessary. On the contrary, I believe the key points of that paragraph are essential to understanding why the study includes measurements for different types of vehicles as well as the differences in operator characteristics. Without it, the subsequent three paragraphs appear disjointed from the rest of the literature review. As such, this paragraph was left in the article as was originally written.

 Make sure you come across as respectful in your comments, especially when discussing suggestions that you didn't implement, and certainly don't word your responses as attacks on the reviewers' or editor's suggestions. If it's clear that one of the reviewers is clearly confused on a certain point, model, or statistic, spin the wording of your response in such a way so that it seems like it was your fault for not being clear, and then justify your original point/method:

Reviewer 2 seemed confused by my use of the term "spatial proximity" on pages 3 and 6. In order to address this potential point of confusion for readers, I have added footnote 2 on page 3 so as to clarify what this term precisely means for this study. 

When you're done, reread your cover letter to make sure, one more time, that you come across as respectful, attentive, and appreciative of all of the suggestions. When you're satisfied that everything sounds good, send the revised manuscript and cover letter to the editor (or upload them to the journal website, as appropriate), and cross your fingers. With any luck, you'll hear back in a few weeks (or a few months) with good news! If not, then see if you can revise the manuscript any further, and then submit it to a different journal to start the process over again.

---------------------------------------------------------------
Image Credits: The Most Interesting Man in the World, Mr. Bean, Reviewer 3, Einstein, Letter Writer

How to handle a journal rejection

In one of my prior posts I discussed how to write reviews for journals. Sometimes those reviews result in good news from the editor, like a revise and resubmit (R&R) decision. The majority of reviews, though, will probably result in a rejection letter. This is the result of basic math, given that top-tier journals have acceptance rates well below 50%. Having recently experienced two back-to-back article rejections myself (same journal, I might add), I figured it might be useful to talk about how I, personally, handle having my manuscript rejected, as well as some thoughts on how to effectively deal with it.

One of Einstein's more relevant theories

First off, this is typically the timeframe I go through with my rejections, and my actions/thoughts along the way:

Day 1: The editor and reviewers at journal X are complete idiots!! Efffffffff Yooooooooooo reviewers!!!

Day 2: Ugh, I can't read this review again without getting angry.  Efff Yoooo!

Day 3-14: [Do something else]

Day 15: [After rereading the manuscript and making comments on the draft]  Hmmm, maybe my submission wasn't as strong as I thought, and maybe the editor and reviewers are only mildly stupid (Eff Yoo!). Still, they might have brought up a couple things that I might be able to fix.  ...And maybe the manuscript was a little long and did have a few too many appendices.  I'll work on the revision tomorrow.

Day 16-???: [Revise manuscript]

Day ??? + 1: Ok, the manuscript has been revised, it's much better than the previous submission, so now where should I send it?  Obviously I can't resubmit to journal X (eff u), so lemme look into journal X...

******************************

Does this above sound like a good timeline/plan that might help you get over a manuscript rejection?  Did you just get a rejection and are still stewing with righteous anger?  If so, then read my step-by-step instructions below!

First, when you initially get the rejection email or letter, you'll likely feel like this:

'Generalizability Concerns?'
Well Effff Yoooooo!!!

...at least, I typically do.  I bet there are few people who can receive a rejection letter and feel fine about it.

I recommend the very first thing you do is print and read the rejection comments (from the editor and the reviewers, if included) as soon as possible, preferably that same day.  

Next, do one (or more) of the following to the pages you just read:

• Burn them (watch out for smoke detectors)
• Smash them into a ball and punch it with your fist
• Shred it in the paper shredder.  Throw in a blank CD-R at the same time for a feel-good crunching sound
• Tear it to bits and toss it in the air like confetti (make sure to do this in a common area that you don't have to clean up)
• Stomp on it (works better if you first crunch it into a ball), and then kick the smushed ball across the room.
• Use them for gun/archery/whatever target practice (this one is probably the most satisfying)

Tony recently had his manuscript rejected from
the Journal of Drug Control Policy...
They didn't like his theoretical model I guess.

...feel better?  Only a little?  Ok, good enough for now.

Now that you're hopefully feeling a little better, reward yourself with a treat.  You've earned it, so treat yourself to a box of cookies (or a drink, or a steak, whatever).  You might be asking: "But my manuscript just got rejected?!? What did I do to earn a reward?"  Ahh, it's what you didn't do that earned you the reward.  Specifically you (probably) didn't:

• Fire off a snarky email to the editor telling them off
• Punch a hole in your LCD monitor
• Throw your laptop across the room and/or into the face of a coworker
• Curl into a little ball and cry (though it's ok if you did this one... you can still have a cookie)

So, treat yourself with cookies or something.

C is also for Craptastic Reviewers

Ok, now that you're hopefully a little calmer and feeling a little better, this next part is very, very important.  If you're not under an extreme deadline to get the paper published (e.g., you're racing against a competing academic so you don't get scooped), then take a week or two without looking at either the paper, the associated data, or (ESPECIALLY) the rejection email/reviews.  Just work on something else for a week or two.

After a couple weeks, you should be feeling a little more like this:

A little less pissed off... that's good!

...so, still a little hurt, and a little pissed off, but you're getting there.  Now, read through your original submission (the one that got rejected), and try and read it as if you're a reviewer and that is not your own writing.  As soon as you're done reading what you submitted, then read the rejection letter and reviews again.


Having read what you submitted, do the reviews seem a little more legitimate in their critiques? Oh really, your submission was perfect?  THEN READ IT AGAIN.  Ok, did you find something that might be worthy of revising a little?  Good.  So, how do you feel towards the reviewers/editor now?

Notice the smile?  That's the spirit!

Ok, that's good!  So, you probably noticed that your masterpiece of a manuscript had a few places that you could improve.  If so, great!  Get to work revising the manuscript, trying to incorporate into the revised version as many of the valid critiques from the reviewers and/or editor as you feel comfortable with.  Eventually, you'll have a finished revision that you can submit to a different journal, hopefully with a better outcome this time (if not, start back at the beginning of this post).

Regardless of whether your coping strategies are similar or different (or the advice above works for you), you should at least be willing to revise the manuscript and send it somewhere else.  Don't let a single rejection result in tossing the manuscript in the garbage (maybe after 3 rejections, but not after the first...).  Also, you have to keep in mind that any decent journal is going to reject more manuscripts than they accept, so chances are if you send them something else, they won't hold your prior rejection against you, and so you shouldn't hold the prior rejection against them (eventually... give yourself some time to be pissed with them).

-----------------------------------------------------------
Image Credits: Finger Old_Finger Young_Finger Tony Cookie_Monster Einstein

How to review a journal article

One of the things I've had to do recently is to write anonymous reviews for manuscripts that are submitted to journals. It was something that I never received any real training on, and I had to pick up from looking at the reviews of my own manuscripts I had received previously, chatting with colleagues, and simply thinking about the things I would want a reviewer to tell me.

Manuscript... REJECTED!
Ha ha ha! Huzzah for anonymity!

One of my friends who is still in grad school just received a request to review a manuscript for an article, and asked if I had any advice on how to do that. I figured this would be a great topic for a blog post. So without further ado, here is my (relatively new, fairly untested) advice about how to review a manuscript for a journal. Remember that this is what I find works for me as an anonymous review, and what I like to see in a review. Your mileage may vary.





1. Start by putting the manuscript into whatever format you typically read journal articles.
I find it much easier to critique an article for a manuscript when I approach it as if I was reading a regular article already published in a journal. When I read most of my regular journal articles, I download the PDF, print it out, and put it in my 3-ring binder of things I need to read. I can then take the binder to the couch, on the porch, on the bus, or wherever I decide to get some reading done, rather than being tethered to my computer.

Yes, I realize that my
reviewing method is indirectly
responsible for the death of
many trees.

This also makes it easier for me to read the manuscript and think about its contents, rather than being distracted by its odd format, as I would be if I tried reading it on the computer in the format originally sent to me by the editor (typically a MS Word or Adobe PDF). I can also make notes much easier on the printed out pages than I can using the MS Word or PDF electronic comments and markups, and it's easier for me to review my comments later on when I write the summary (more about this later in the post).


2. Relatedly, read the article as if it was already published.
Read the article as if it was coming out in the latest issue of the journal. Don't cut it any slack because it's in 'draft' form. As you read, make marks in the margins, on the text, or in a separate list about things you could add into your review.

For instance:

• What do you agree with? What did the author do well?
• What do you disagree with? Note that this is not whether you like or don't like the outcome of the results. Instead, do you disagree with things like:
• The way the literature review was setup
• The type of analytical model used
• The statistics reported or not reported in the tables
• The interpretation of results
• The link to policy/practice 
• Is the literature review fairly comprehensive and does it set up the analyses and discussion well? Or, instead, is it missing a key piece of the literature, key citations in the field, or does it read as a very lopsided argument, ignoring any prior literature that contradicts the theory/hypotheses being tested by the authors.
• Related to the literature review, does it misinterpret results from prior studies? Does it say that study X found Y, when it really found Z?
• Does the study have a 'consistent read,' or does it instead read like two (or more) people wrote different pieces independently and just stitched them together at the end?

Dear editor: Me strongly disagree
with reviewer two, who say me writing
read like two people be 'stitched together.'
Me feel much unhappy and me want to
squeeze reviewer two's neck until dead.
P.S. Fire bad.

3. Do you spot any errors?
Do you see anything that obviously needs to be fixed? Any typos, formatting errors, missing table labels, citations that you can't find in the references section, etc. Regarding the citations, you don't obviously have to check every one as that will be spotted in the proofing stage, but if a study sticks out in the text and you can't find the citation in the reference section, note that. If the reference section is obviously missing something important based on the journal's style (DOI's where required, first initial/names, etc.), note that.


4. Are there any citations that seem fishy? 

Why yes I did attend the
1932 presentation I cited
in my manuscript

One of my pet peeves when reviewing manuscripts is when the authors cite a conference presentation that's more than a few years old. First, I think "Were they actually THERE, or are they just citing the abstract they found online?" I then think "If this presentation was so important, why hasn't it been turned into a paper yet?" Either way, if you're citing a presentation that's more than, say, 3 years old, you probably should either find the resulting paper or find a different source, in my opinion.

If you see 12 different citations for
studies by Manatee and colleagues,
you're probably reviewing a paper
by Manatee and colleagues...

Do the authors cite websites that aren't scholarly (e.g., Wikipedia, Urban Dictionary, Blog Posts, etc.), but cite them as if they are scholarly sources of information? Do you notice any 'unpublished' citations (if so, they probably shouldn't be citing them unless those manuscripts are actually 'in press').

If the discussion and

conclusion sections sound

like these guys wrote them,

they probably don't deserve

a revise and resubmit..

5. Do you feel the article ties the lit review/analyses/etc. all together into a coherent final discussion and conclusions? 
When you finish the manuscript, does it make you go "Ah ha! That's the point!" or does it make you shrug your shoulders and go "So what? Who cares?" If the authors don't do a good job setting up the front end of the article, then they're probably going to run into a problem at the back end when trying to tie everything together into a coherent point. If you can see a glimmer of hope in the article, and the authors simply didn't do a great job at getting across the 'so what' about the article, then you should note that in requesting a rewrite of the discussion/conclusions. Relatedly, does the author miss a big piece of the results? Do they downplay an important finding that should be highlighted? Note these in the review, explaining exactly what you think the author missed or should otherwise correct, and why they should do this.


6. Overall, does it seem like the manuscript is a good match for the journal you're reviewing for? 
For instance, if the journal is focused on linking research to practice and policy (e.g., Justice Quarterly, Criminology and Public Policy), does the manuscript talk about implications for policy and practice? If it doesn't, and the main takeaway doesn't seems to have a clear implication for policy and practice (even after a revision), then perhaps the authors should instead submit to a less policy-focused journal (which obviously means they shouldn't get an R&R at the current journal...). The same goes for journals focused on quantitative research methods (Journal of Quantitative Criminology), theoretical aspects (Theoretical Criminology), Policing, or other specializations. If the manuscript is simply not a good fit for the journal, then you should note what journal might be a better fit.

Interesting conclusion,
Dr. Crazy-Sweater...
Revise and Resubmit.

7. Is the paper written in a neutral way, or does it come across as an advocacy piece?
If the paper is meant to be a scholarly research manuscript, it probably shouldn't read like an advocacy piece. For example, if the manuscript examines the relationship between religiosity and crime/deviance, it shouldn't read like something that could have been written by either the Vatican or the Freedom from Religion Foundation. If it does, then it probably needs to be scaled back in terms of tone so as to come across as more neutral. Note that this doesn't apply to all scholarly articles, as some forums (like the response essays in Criminology and Public Policy) are geared more for discussing certain viewpoints or implications related to the findings of the research manuscript.

Unless this guy wrote
the paper, there are
probably at least some
limitations to mention.

8. Does the author note the important limitations of the study?
All studies should mention important limitations. For example, if the study is about a single county, the generalizability of the study should be one of its limitations. If the manuscript doesn't mention this limitation, then it should. Related to this, does the study stretch the limits of generalizability? Does the one-county-one-state study try to pass off results as valid across the entire US or even the entire state? If so, something's wrong that needs more fixing than just adding something in the limitations section.


9. Time to make the master list.
Once you have all of the things noted on the manuscript (or otherwise written down) that you think are either 'wrong' with the article or need some type of revision, and at least a few things that are well done, you are almost ready to write your actual review.

I like to go back through the manuscript and make a final list of everything I didn't like, want changed, or liked, and order it by page in the manuscript. I like this method, because it lets the author more easily address comments and change requests, as they'll be able to quickly find what the review is talking about in each point.

While Smith and Fluffy (2011)
raise interesting points, I believe
they should give less emphasis
to the government mind-ray
theory in their manuscript.

If there are a number of things that all fall under the same category (e.g., a phrase you think should be changed throughout the manuscript, a misinterpretation of results, a typo that keeps popping up, etc.), then simply note the first occurrence of it on your list and indicate that this is a problem in other places as well.


10. Time to summarize everything.
After you have your final list with all of the things you're going to note to the authors, you now should write a few paragraphs (maybe a page or so) about your overall impressions of the article. A lot of the reviews I've seen begin with a short (2-3 sentence) summary of the article: what was the research question, theory, findings, and conclusions? If you choose to do this, great, but the 'meat' of the review comes after this.

While perhaps a correct
review, try for something
a little more scholarly.

When you write up the summary of what you think of the article, you should remember to emphasize what was really well done, and not just focus on what the major problems/fixes. Remember the complement sandwich! If something needs a little more research, feel free to suggest citations to use, but if you keep suggesting the author incorporate citations by "[Your Name] and Colleagues," you're unlikely to remain an anonymous reviewer for long.

The summary, in my opinion, should read as if you read a friend's dissertation and are giving the most important feedback up front to them. Don't mention every minor typo on your list (the list will speak for itself later), but if there are a bunch of typos throughout, mention that the author needs to do some major proofreading in the next draft.

You don't want to come across
as "the special reviewer"

Make sure that your summary comes across fairly neutral in tone, especially if your comments are geared towards the negative. You don't want to blow the author out of the water with nastiness, nor do you want to be kissing their ass. Praise the manuscript where you feel it's appropriate, and take the author to task for things that are egregious errors, but make it a fairly evenhanded summary.

The final paragraph of summary (before you provide the list of changes/notes/etc.) should indicate whether you feel the article is good enough to publish (pending a minor R&R of course), or if you instead feel that it's not an appropriate manuscript for the journal. If you think it's not a good fit for this journal, but a perfect match for another one, then say so.


11. Making anonymous comments to the editor.
You will also probably have a chance to write directly to the editor about the manuscript. Your comments in this special section will remain anonymous to the author of the manuscript. You should end up with perhaps a couple paragraphs of private comments to the editor that state what you really feel about the article, but don't feel comfortable saying to the authors. For example, if the article reads like an undergraduate student wrote the article, gave it to a professor to quickly read through and sprinkle in citations to his own work, and then shipped it off to the journal, egregious spelling mistakes and all, then say so. If the study is one of the best you've ever read on the topic, and that you think the study should be published with very few corrections, then say so. If you can't be honest when writing directly to the editor, when the author won't see your comments, then you probably need to reconsider reviewing the manuscript.

This is what happens when
you don't proofread things...

12. Proofread your damn review!
One last thing you should do is to proofread your entire review. You should be reading it both for grammar/spelling mistakes (especially if you call out the author for making spelling errors in their manuscript), and also to ensure you don't come across as either a complete jerk or as a clueless reviewer. You can say a manuscript shouldn't be accepted and still give the author some good feedback and a gentle letdown. Alternatively, you can say the manuscript should be published and is a great example of scholarly work, while still providing insightful feedback to incorporate before the final acceptance. If you read your own review and think "I wish I could get reviews (negative or positive) that read like this!" then you've done a good job.

Addendum: My friend has this bit of advice to add (Thanks Dr. Liz!): 

No matter how terrible the paper you review, tone down negative criticism. Avoid inflammatory language or personal attacks. Don't use exclamation points. Also, reviewing papers is an amazing learning experience about a) what journals ask reviewers to comment on and what kind of scale - from reject to accept - they use and b) what a good paper "needs" (this can help your own writing).

You mean I can review things
for you, for free, and not receive
any credit other than a line on my
C.V. that won't really help me get
a job or tenure? Hell yes I'd be
interested!

Postscript: How to become a reviewer for a journal
When I first started my PhD program, I wasn't really sure how people became eligible to review for journals. Sometimes, once you have an article accepted by a journal, you might automatically be added to their 'list' of potential reviewers. However, if you want to review for a journal you haven't been accepted at before, you can simply send the journal editor a short email explaining your interest in reviewing for them, and what your area(s) or expertise are. They will likely be happy to have you volunteer, and you might receive something to review within a few weeks (although don't hold your breath, it all depends on what is submitted to the journal.)





Final Note: A good link from a major journal publisher explaining what they are looking for in a journal review can be found here.

Do you have any other tips for reviewing manuscripts?  If so, add them below in the comments or send me a message!
---------------------------------------------------------
Photo Credits: Bag_Head Clearing Frankenstein Old_Man Thoughtful_Manatee Dumb_and_Dumber Protester Mr._Perfect Tinfoil Proofreading You_Suck Owls Phone_Guy (Yes, I'm addicted to this photo...)