How to 'do' research (Part 1)

[Sorry about the long wait time since the last post.  The first-year-on-the-tenure-track mid-semester crunch really slapped the hell out of me these past month or so.]

I was chatting with a new Ph.D. student the other day about their research ideas and (eventual) dissertation topic.  They said that they had plenty of statistical training (they have two master's degrees), but haven't had much training (yet) in the way of research methods.  Midway through the conversation about their research interests and background, they asked me one hell of a question: "How do you actually do research?  Like, how do you find the literature, figure out a research question, write up a literature review, choose a dataset, that kind of stuff?"

I think 'doing research' is closer
to this than anything else...

The answer to that question, I think, is both very complicated and very personal.  I assume that everyone does their research a little differently from anyone else, and it likely depends on whose research methods courses you took in grad school, which articles and books about research you read (and understood/liked), your own personal preferences and habits, the kind of research you're doing, and a whole lot of other things that aren't immediately coming to mind.  So, I don't think it's as simple as "Do X, then Y, then Z, and viola!  Top-tier pub here we come!"

Still, it's a fair (and good) question to ask, especially as a new Ph.D. student whose largely expected to hit the ground running.  While my answer right then wasn't really well thought out, insightful, or likely even coherent, I've had a little more time to think about it.  Thus, this is the way that I typically conduct research (or at least part of it -- I'm sure there will be more to come, hence the 'Part 1' in the title).  It seems to work pretty well for me thus far, so feel free to steal anything that looks like it might help you, since I'm sure I stole most of this from other people along the way.

1. Read the literature for stuff that interests you.

Researching uninteresting stuff
is like watching Golf on TV:
Both make me want to stab my
eyes out with a spoon.

Note the second part of that statement -- 'for stuff that interests me.'  I think one of the biggest mistakes people make as Ph.D. students and new researchers is choosing research topics only because it's what's 'hot' in the field, or because their advisor does it, or because they can be funded through a project to do it, or for any number of other reasons that don't involve being genuinely interested in it. (Yes, there's obviously something to be said for doing research that will eventually get you a good job, but you should be able to balance the marketability of a particular research subject with your interest in it.)

There's a reason I left my stable, fairly well paying office job to earn my Ph.D.: I got to the point where couldn't see myself doing something, as my career, that I didn't absolutely love.  I still feel that way, but now instead of that unloved 'thing' being the 9-5 office job of shuffling papers, it is now conducting research that I really couldn't give two sh*ts about (or even one sh*t about). So I try not to do that on a day-to-day basis, and I think I've been fairly successful at it thus far.

So, I guess the precursor to step 1 would be 'figure out what interests me.'  This likely involves reading a lot of broader literature in your field to find the things that you want to read more about.  Once you've done that, and you know that you're really, really interested in X and Y, go ahead and look up more literature on X and Y and read it.

Keep in mind that this isn't simply reading the latest 3 years of top-tier journal articles that used X and Y as keywords, but also involves figuring out who the big names are (and were) in the field, what the major studies on X and Y are that get cited over and over again are, and what types of methods, data, etc. are generally used in those studies.  You should also read studies that might not be on X or Y, but relate to it somehow (similar methods, similar units of analysis, similar theoretical models, etc.).  All of this reading allows you to talk coherently in your research papers about what's been done before, who has done it, how they did it, and how your stuff is similar.

So, now that you're knee deep in the literature of the stuff that interests you, what's next?  Well, it's something that takes place while you're reading the literature...


2. Figure out what people have been doing poorly, or not at all, in those studies (i.e., find the 'gaps' in the literature).

Similar to the UK warnings,
if you don't mind the gaps in
the literature, you'll be hit by a
metro train called 'peer review'

This really only comes from reading study after study after study.  Invariably you'll be able to spot the limitations of studies that you read, whether it's generalizability, unsophisticated methods, not using Theory A, not studying area B, etc. etc. etc.  Sometimes the authors will do you a solid and list the 'real' limitations of their studies in their respective limitations section.  From what I've seen, however, most of these so-called limitation sections are really just a list of one or two of the most minor problems that barely constitute an actual limitation, with no mention of the more serious limitations of the study.  So read the limitations sections with a grain of salt, but if something keeps popping up over and over again (e.g., 'this study only used cross-tabs and not multiple regression', 'we couldn't study X, so we had to study a proxy of it, Y', 'we could only examine one county of data, so generalizability is a concern', 'historical/longitudinal data were unavailable for X, so this was cross-sectional in design'), then it's likely one of the main gaps in the current literature.  You want to figure out what these gaps are, because they can help you either craft a research question or craft your study's methods section.

For instance, if you find that prior studies have all examined individual states/counties/cities, and no one's compared multiple places, that could be the source of a new study.  Your research question might be as simple as "What are the state/county/city characteristics that influence X", and even though this exact same question might have been asked by multiple prior studies, if your study does something those other studies didn't (e.g., examine multiple places rather than just one), you'll probably be making an important contribution to the literature.  If studies have all ignored Theory A, and you think Theory A could be important in explaining X or Y, then your research question might be "Does Theory A explain the occurrence of X?"

Ok, so you've read (and are continuing to read) the literature on your topic, you've found some things that are missing in the current literature that you might be able to address, and you've incorporated that into your research question.  Now what?

3. Write a good, but concise, literature review. 

You should by now have a handle on who the big names are in your field, what the major studies are, and what the general findings of the literature are on topic X.  Now comes what is, for me, one of the hardest and most boring things: writing the literature review. There are a number of guides on this floating around the interview (see one example here), and I guess now there's one more...

Note that I'm talking not about the 40+ page literature reviews that get published as stand-alone manuscripts in journals. Rather, I mean the shorter, ~5-10 page reviews you see at the front of a 'regular' journal article.

The literature review helps lay the foundation for your study - it talks about what people have done, how they did it, and what they found, but it also connects those findings into a coherent narrative, as well as exposing the gaps that you will be filling with your study. That said, the literature review can vary tremendously depending on how your study is set up. I have most of my experience with quantitative, policy-oriented studies, so that's the type of literature review I'll be discussing. I'm sure this varies for qualitative, historical, theoretical, etc. studies, so buyer beware.

In any case, the point of your literature review should be to give the readers a crash course in what's already been done in the field that relates to your research. Sometimes this will be fairly easy and direct, since a lot of people will have done studies before you on a similar or same issue.

Another research idea:
The effect of access to hoodies
on rates of juvenile delinquency.

For example, suppose you're writing about the relationship between doing poorly in school and juvenile delinquency. If there were a number of studies, or even a specific theory, that can be easily applied to this issue (let's go with Strain Theory, since apparently everybody in my field wants to test this at one time or another), then much of your literature review might be discussing the theory, what it says, and how these studies have supported (or not supported) this theory. So you might want to set up your literature review by first discussing Strain theory generally, and then noting how it easily applies to your specific situation (i.e., doing poorly in school leads to delinquency). Then you can cite prior studies on school achievement and delinquent outcomes, and note whether they support Strain Theory or not. You would note the problems with the prior research, and how more research on this issue is sorely needed (although, really, it isn't in this case... so please stop doing these studies, folks...). This sets up your study by placing it in relation to the prior research and theory, and also helps distinguish it since you've addressed the limitations of prior research.

However, sometimes you're paving new ground, and need to discuss two or three different types of research and blend them together into a coherent story.  For example, maybe you're trying to examine whether access to a cell phone influences juvenile delinquency. It makes sense, since with it you can chat with your friends wherever you are, and you can more easily decide where to meet up to vandalize the school, or burgle the liquor store, etc. Maybe no one has previously looked at this specific issue (if not, I call dibs...). BUT, people HAVE looked at how communicating with friends (in person) can influence delinquency. So, you review this prior research. Then, you realize that socio-economic status (SES) is going to influence who has a cellphone in the first place, and so you need to review the research on how SES can influence delinquency. Now that you've done this, you finish your literature review by pulling together these two prior research fields (communication and delinquency, SES and delinquency), and relate this to juveniles communicating via cellphone, and support why you think this would (or would not) have the same influence on delinquency as face-to-face communication with friends, controlling for SES.

I can't be too much more specific on how to actually organize your literature review, because the organization is going to depend on a number of factors specific to what you're studying, how you're studying it, and what's been done in the past. I recommend reading a lot of articles, mostly from top journals, that have looked at similar things and see how those were set up. This should give you an idea on how to set up your own review section.

Side rant: Tone
Let me take just a minute to talk a little about tone when discussing the prior literature. I'm assuming that the ultimate goal of doing this research will be some type of scholarly journal article that will need to survive the peer review process. If this is the case, then the journal will send your work to other 'experts' in field. While this might include grad students that are just beginning their careers, it might also include some experienced professors whose past research you've cited and discussed in your literature review. How you discussed their research, then, matters a great deal...

Let's consider the following examples:

Example 1: Standing on the shoulders of giants.

Example 1:
"The prior research has been
tremendously important, but
has suffered from some minor
issues which I will address"

Imagine you're one of those distinguished professors reading an article for a journal. The author discusses your article in a neutral fashion, commends certain things that it (you) did well, and even-handedly notes what was missing or otherwise could have been improved upon. The author is acknowledging the contributions of your work, and indirectly suggests that they are 'standing on the shoulders of giants' with their present study. This sets the tone for your thoughts on the article. You think to yourself "Good call, sir! I should have used more than three years of data. I'm glad you're taking the ball and running with it, and glad my research could contribute!" as you sip your brandy and puff on your Meerschaum pipe while sitting by a roaring fire in your study (or where ever/however distinguished professors review journal articles...). The rest of the article is well done, so you send off a positive revise and resubmit recommendation to the editor, with encouraging words for the author.


Example 2: All my predecessors were idiots. 

Example 2:
"The prior work is total
doo-doo done by smelly
caca-heads. My research
is good because it isn't
poop based." 

Now, imagine you're that same distinguished professor reviewing a very similar article. The author cites your work, but does so in a dismissive, critical tone. It's clear that they think your prior research was garbage, and that you were obviously a drooling idiot and/or untrained monkey. While not specifically saying this, they imply that their research can 'save the field' from its past horrific, worthless studies conducted by lesser researchers. "Humph and habberdashery!" you exclaim as you quickly type up your review to the editor with the strong recommendation to reject this horrible, insulting manuscript on the spot. "Little sh*t thinks my research is garbage, does he? Well good luck getting into [top journal of the field], muahahahahaha!" you cackle as you fire off your recommendation to the journal and casually toss the reviewed manuscript into your fireplace.

So kiddos, what did we learn from these examples? Well, first, apparently there are still some people in the world who use the phrase 'habberdashery.' Second, and more importantly, the tone you use to discuss the prior literature can be very important. Yes, you need to point out the gaps in prior studies so you can set up reasons for why your research is an important contribution. However, praising the prior researchers for their contributions, and implying that you're using their research as a basis for yours to help further the field, is a very, very different story compared to one where you're bashing the idiocy of prior researchers to promote your (obviously) much more sophisticated study. You want your manuscript to come across as 'standing on the shoulders of giants,' rather than 'chopping the giants off at the kneecaps.'  So tone matters.

Hopefully what I've written so far will be somewhat helpful to those of you just starting the research process. I'll be posting more on this topic in the future, but this should get you started.

--------------------------------
Image credits: Golf, Hoodies, Old Scholar, Baby Scholar, Mind the Gap, Then a miracle occurs (Harris, 1977)

Writing an Acknowledgement Page in the Dissertation

One thing you'll have to do near the end of your dissertation writing days is to come up with an acknowledgement page.  This can be a tough thing to write, but it's your chance to put some real personality into your writing.

Even though you might feel
like this near the end of the
dissertation process, try to
refrain from thanking your
dissertation directly as if it
were a human being.  Save
the crazy for after tenure.

If you haven't really thought about writing the acknowledgement page yet, but it's getting close, I'd recommend checking out a few recent dissertations and seeing what THEY wrote.  You'll likely read some that you think are horrible/boring/sterile, and some that are really touching/interesting/funny.  It's really up to you whether you want to go with the standard business-like thank yous or make it more personal, though I recommend the latter.

So, once you decide the type of page you want to write, you have to come up with a list of people to thank.
Here's some ideas that might help you remember everyone who helped you along the way.

1. Your dissertation advisor/committee -- Obviously these are the people that worked with you to get your dissertation finished, and in my opinion they should be either the first people you thank in the acknowledgments, or the last people you thank.  I went with first, saving the last thank yous for my family.  You should single out your dissertation advisor (DA) from the other committee members (CMs), as the DA likely put the most effort into your dissertation compared with the other CMs.

2. Your other 'advisors' in the program.  You may have been assigned an advisor early on in your Ph.D. program that ended up not becoming your DA, or even a CM, but provided a lot of early advice and encouragement.  If so, they should be included in your acknowledgements.  If they are one of your CMs, then single them out in addition to your DA, and if they are you DA, then be sure to mention the non-dissertation support they provided for you.

3. The other faculty you frequently bugged for help.  When I was in the Ph.D. program, there were a few faculty that I frequently bothered for advice on a number of topics, despite having no formal mentor/mentee relationship with them.  The advice ranged from detailed methodological/statistical advice to inform career advice.  I'm sure by the time I was ready to graduate, they had developed an inadvertent eye-twitch whenever they heard me knocked on their door and/or start a conversation with the phrase: "So I've got a quick question for you..."  Even though they may not have directly helped on my dissertation, they still helped me navigate life as a Ph.D. student, and so they deserve to be thanked.

4. The staff you bugged for help.  There is likely at least one staff member who not only knew you by name, but did things above and beyond the call of duty to help you out.  This might have involved helping you file all of the misc. paperwork for a grant application at the last minute, helping find you a last-minute assistantship position to keep your health insurance in tact, or even printing on their fancy high-quality laser printer 20 sheets of stick-on beer labels for your newly brewed batch of Maple Ale.  If anything like this applies to a staff member know you, then you should thank them.

There's a reason the show wasn't
just called Garfield. Dude needed
help filling the other 15 minutes.

5. Your friends.  The Ph.D. program can be very isolating and lonely.  If you're lucky, you will find a tight-knit group of friends to commiserate with, whether they are part of the program or not.  Even if they don't know the first thing about your research and never read a draft, they had a small part in helping you get through the program and finish the dissertation.  You should thank them in your acknowledgement page (by name for at least some, with a catch all 'all others who I haven't yet thanked').

6. The people who got you to grad school.  I know that if it weren't for the people who wrote my recommendation letters, I wouldn't have been accepted to the grad program that I graduated from. Therefore, I felt I should thank them for the time they put into me both as an undergrad student of theirs, as well as for the letter writing that they did for my application packet.

8. Other support networks.  If you have other support networks, like a grad student forum that you've used for advice over the years, writing support, etc. don't forget to include them if it makes sense to.

8. Your family.  This is an obvious one, and who you specifically mention will vary based on your own circumstances.  In my own acknowledgement page, I put my family (parents, sibling) next to last, and put my wife last, as I felt the first and last positions held the most importance.


Final thoughts: Finally, just a couple words of wisdom regarding the acknowledgement page.

First, I don't personally believe this is something that your dissertation committee needs to see.  It is personal and doesn't affect the merit of your dissertation work in terms of defensible or not.  So, if you don't feel you want the committee to see this page until after the dissertation is bound and printed, then I see no concerns with slipping it into the dissertation after the final defense and before the final product is sent to the university's graduate office for final approval and publication.  (Your mileage may vary, however).

Second, you should spell check your acknowledgement page very, very, very carefully.  This is because the rest of the dissertation has probably been read dozens of times by various people, but this is something that, probably, very few people will have a chance to read before it's in print.

Dr. Seuss had many friends.
Dr. Seuss liked to use pens.
Those who mattered minded not.
Those who did he f*cking shot.

Third, I think there's nothing wrong with adding some humor to this page.  To crudely paraphrase Bernard Baruch and/or Dr. Seuss, "those who matter won't mind and the rest can go f*ck themselves."  I would much rather have someone read my acknowledgement page and laugh (or cry, if you're my wife -- yes, it was that good/touching/etc.), than have someone read it, shrug, and go "eh, pretty good."

Finally, you should be proud of your acknowledgement page when you're done.  My acknowledgement page, I think, is pretty damn good, as well as interesting, funny, and touching.  Plus, where else in my dissertation could I have used phrases such as "hobo salary," "purchase the 'good' beer that individuals of stature and sophistication drink," "ridiculously strong back muscles," "totally-should-be-subject-to-freedom-of-information-laws-but-it-is-not," "slow-but-steady financial support (one poker game at a time),"  "squeezing the bejesus out of him," "defend myself from an onslaught of up to four enemy samurai," and "zombie apocalypse."
...See, don't you feel at least a little interested in reading my acknowledgement page now?  That's what I was going for.
------------------------------------------
Image credits: Hugging_Words, Garfield, Dr. Seuss,

How to respond to an R&R

[Note: Some of the follow suggestions I picked up from my faculty advisors and colleagues while in grad school, others I came up with on my own, and still others came from Wendy Belcher's excellent book: Writing Your Journal Article in 12 Weeks]

So, you submitted a manuscript to a journal, and the editor liked it enough to give you an R&R... now what?

First, congrats! An R&R is about as good as you can expect for a regular journal, since very few manuscripts are accepted 'as is.'

Depending on the editor's letter and reviewers' comments, it might be an easy R&R that won't take much time, or a very hard R&R that will take weeks or months to address. Also, depending on the journal, an R&R might mean the manuscript is basically accepted, pending a little polishing, or alternatively that there's a good chance you'll still get rejected later on in the process. In any case, your goal should be to address as many of the recommended changes as is reasonably possible in the revision to make the paper better, without accidentally 'messing up' your paper (or your main points) in the process. Hopefully this post helps guide you there.

So the first thing you should do is read the editor's letter in its entirely (you probably opened the letter/email and just skimmed the page(s) until you saw 'revise and resubmit'...).  The editor's job is to read the reviews, make a decision on your manuscript (in this case, R&R), and then (in my opinion) give you guidance on what are the most important things to address in the revised manuscript. If the editor didn't do that, or gave you a generic 'please address the reviewers' comments in your revised submission,' then your job will be a little harder than it probably should be.

Anyhow, read the editor's letter and see if s/he gave you specific instructions as to what to address. The editor might note the important of a missing variable in your model, as mentioned by reviewer 2, or that your treatment of the prior literature was lacking in X area, as noted by reviewers 1 and 3. Whatever the editor says in their letter regarding potential changes should be your first priority in the revision. What I like to do is open a new word document and make a bulleted list of the changes the editor specifically mentioned in the letter (these bullets go under an 'Editor' heading). This will become part of your 'potential revision' list/spreadsheet that you will continue building in the future. (More about this in a bit).

Next, read through all of your reviewers' comments, without taking any notes. Make sure you have your submitted manuscript draft in front of you, so you can reference it when a reviewer mentions something on page X or in Table Y. Keep an eye out for comments that come up from more than one reviewer, such as the suggestion to use robust standard errors in your model, to provide a correlation matrix table, or to cite the work of authors A and B.

Chances are, you won't agree with everything the reviewers recommend, and you'll form the following opinions about the reviewers [note: for ease of comparisons, all three reviewers are assumed to be male in this example]:

Reviewer 1: Even my
bowel movements are
high impact.

Reviewer 1: You'll probably think this reviewer is a genius, particularly since he gave you the most useful recommendations, and other than a couple minor suggestions, loved your entire manuscript. His suggestions are relevant, fairly easy to implement, and will really help boost the quality of your manuscript. This reviewer, while rarely seen in the wild, is the best you can hope for.

Reviewer 2: "Derp?"

Reviewer 2: You'll probably think the second reviewer is an idiot, but a fairly harmless one. He probably doesn't really know anything about either your analytical methods or the literature related to your topic, but tried to fake it so the editor didn't get angry. Most of the suggestions he offered aren't very useful and/or make no sense given your model, argument, and findings. Still, you can easily address his main concerns, and easily dismiss his unhelpful and misguided recommendations.

Reviewer 3: My comb-over
is horrible, I can't eat sweets
due to my diabeetus, and I'm
going to take my anger out
on you and your manuscript.

Reviewer 3: Upon reading reviewer 3's comments, your first thought will probably be "what a f*cking d*ckhead." You'll probably think that every suggestion by reviewer 3 is complete garbage, that he misunderstood your entire argument, knows nothing of the methods you used, and decided to use his "anonymous" review as an outlet for his deep-seated inner frustrations concerning sexual impotency and male pattern baldness. You'll likely question his academic and genetic pedigree, and wonder how the editor could have chosen such an arrogant jerk. This reaction is completely normal, and has a long historical precedent.



In fact, Einstein even developed a theory related to reviewer comments that largely supports these three general categories, as previously noted in my post on how to get over manuscript rejections:

More useful than relativity.

However, these categories are not exhaustive, nor are they exclusive. You may, for instance, get a review from a reviewer that combines traits of 2 (idiot) and 3 (jerk).

 Regardless of the types of reviewers you get, or the content of their reviews, the fact that you received an R&R meant the editor saw some promise in your manuscript. Therefore, now that you've surveyed all of the reviews, you need to get to work addressing the specific suggestions and critiques, and revising your manuscript for resubmission.

So the next step is to go back through the reviews and add to an excel spreadsheet (or equivalent) every specific suggestion or critique, from each reviewer, that requires you to change something (or might require you to say why you didn't change something.) Each suggestion/comment should be its own entry on a separate line. I recommend five columns in the spreadsheet: ID # (increments up), Reviewer (who made the comment), Comment (what was the change/critique), Importance (How important does this seem to your overall revised manuscript, or to the editor/reviewer?), and Ease (How long and/or how much work is this going to take to address). I also like to color code the importance and ease cells using the an easy to distinguish color scheme - red for important/hard, yellow for medium/moderate, and green for low/easy. If you set it up like this, your spreadsheet should look something like the following (excluding the earlier comments from the editor):

Once you've finished adding the reviewers' comments to your spreadsheet, add the editor comments you previously listed out if they're not already covered by a reviewer, and mark anything the editor said to do as high importance. If the editor says something a reviewer already said, then add 'editor' to the reviewer cell as a reminder, and mark it as high importance regardless of your original importance rank. If the editor specifically says something is not important to address, then mark it as low importance, and if they say not to do it at all, then change the font style for the associated entry to strikeout.

After everything is down on your spreadsheet, you can get a sense of the overall work that's ahead of you by looking at the importance/ease categories for each entry. A roadmap to these combinations, and what they mean for your revision, is below:

High/Hard: These are the changes that will give you the most headaches. You need to do them (high importance), but they're going to take a long time and generally be a P.I.T.A. (hard to do).  Doing these first will make your life much easier later on, and if they're major changes, some of the less important changes may be obsolete once you do these. If you decide not to do one of these, you need a very good reason why not.

High/Moderate: These are changes that will be a little easier than the High/Hard changes. You still need to do them, and they're going to take you some time, but they won't be absolutely horrible to address.

High/Easy: These are the changes that will make you feel most productive, since you can address these important 'need to do' changes very quickly and without much work.


Medium/Moderate: You should probably do these changes, especially if you have the time.  If you don't do one of these change requests, then you should definitely address why you didn't in your cover letter (more about the cover letter in a minute).


Medium/Easy or Low/Easy: Since these are easy to do, you might as well do them, especially for the ones of medium importance. If you choose not to do these, you should at least acknowledge why you're not doing the medium importance ones, if not the low importance ones as well (since not doing the latter will make you look lazy).

Low/Moderate or Low/High: These might be changes you decide not to do, given their relatively low importance compared to the moderate-to-high amount of work they'd require of you. Again, you may want to address why you didn't do these in your cover letter, but given their low importance, if you miss discussing some of these it probably won't be the end of the world.


Ok, so you have your potential changes all in order, you've ranked their importance and ease, you know which ones you need to address, which you should address, which you can easily address, and which you're not wasting your time with. Now comes the hard part: Make these changes in the manuscript. That's about as specific as I can get for this step, but this is what will take the longest.

Once you're done making (or not making) the various changes, you need to reread your manuscript from start to finish to make sure things still make sense. This is especially true if you had to do some serious changing of text. Edit as necessary.

"Dear Editor: Thank you for
the opportunity to revise
my manuscript. Unfortunately,
as I am not fluent in dumbass, I
have been unable to interpret
Reviewer 3's critiques."

After your revised manuscript is re-polished into a final version, you need to draft your cover letter to the editor. I recommend starting the letter with a brief paragraph thanking the editor for the opportunity to revise and resubmit your manuscript for their journal. You should also thank the editor and reviewers for their helpful suggestions and critiques (if even you think most of the suggestions were garbage). Then I recommend stating something along the lines of "I've provided specific responses to each of the suggestions and critiques below. I look forward to hearing your final decision." This will provide a transition between your formal paragraph and the subsequent list of changes that's about to follow, as well as obviate the need to write a closing paragraph after your list.

Skip a line or two, and then list out each change you want to address (all of the high importance ones, most/all of the ones of medium importance, some/most of the low importance ones). Briefly summarize the suggestion or critique, identify who said it, its location in the manuscript, and what you did to address it (or, alternatively, why you did not address it). Examples of wording for this section are provided below:

Reviewers 1 and 2 recommended adding r^2 values to Table 3 (p. 21). This has been done for all three models.

 Reviewer 2 recommended revising the paper to focus on racial threat theory. While this is a valid alternative theoretical focus, I feel that racial threat is not appropriate for the current study given the current focus on non-racial factors, as well as the comments of Reviewer 1 regarding the appropriateness of the current theoretical model. Further, using racial threat would generate the problem of..."

 Reviewer 2 questioned the need for spatial lag measurements in the final model. I have expanded the justification for this measurement on page 4, but believe that the inclusion of this measurement is essential in order to control for the spatial autocorrelation of the observations that was indicated by the Moran's I tests discussed on page 12, footnote 3. As such, this measurement has been left in the final model.

Reviewer 3 recommended deleting the third paragraph on page 2, as it was considered unnecessary. On the contrary, I believe the key points of that paragraph are essential to understanding why the study includes measurements for different types of vehicles as well as the differences in operator characteristics. Without it, the subsequent three paragraphs appear disjointed from the rest of the literature review. As such, this paragraph was left in the article as was originally written.

 Make sure you come across as respectful in your comments, especially when discussing suggestions that you didn't implement, and certainly don't word your responses as attacks on the reviewers' or editor's suggestions. If it's clear that one of the reviewers is clearly confused on a certain point, model, or statistic, spin the wording of your response in such a way so that it seems like it was your fault for not being clear, and then justify your original point/method:

Reviewer 2 seemed confused by my use of the term "spatial proximity" on pages 3 and 6. In order to address this potential point of confusion for readers, I have added footnote 2 on page 3 so as to clarify what this term precisely means for this study. 

When you're done, reread your cover letter to make sure, one more time, that you come across as respectful, attentive, and appreciative of all of the suggestions. When you're satisfied that everything sounds good, send the revised manuscript and cover letter to the editor (or upload them to the journal website, as appropriate), and cross your fingers. With any luck, you'll hear back in a few weeks (or a few months) with good news! If not, then see if you can revise the manuscript any further, and then submit it to a different journal to start the process over again.

---------------------------------------------------------------
Image Credits: The Most Interesting Man in the World, Mr. Bean, Reviewer 3, Einstein, Letter Writer

How to handle a journal rejection

In one of my prior posts I discussed how to write reviews for journals. Sometimes those reviews result in good news from the editor, like a revise and resubmit (R&R) decision. The majority of reviews, though, will probably result in a rejection letter. This is the result of basic math, given that top-tier journals have acceptance rates well below 50%. Having recently experienced two back-to-back article rejections myself (same journal, I might add), I figured it might be useful to talk about how I, personally, handle having my manuscript rejected, as well as some thoughts on how to effectively deal with it.

One of Einstein's more relevant theories

First off, this is typically the timeframe I go through with my rejections, and my actions/thoughts along the way:

Day 1: The editor and reviewers at journal X are complete idiots!! Efffffffff Yooooooooooo reviewers!!!

Day 2: Ugh, I can't read this review again without getting angry.  Efff Yoooo!

Day 3-14: [Do something else]

Day 15: [After rereading the manuscript and making comments on the draft]  Hmmm, maybe my submission wasn't as strong as I thought, and maybe the editor and reviewers are only mildly stupid (Eff Yoo!). Still, they might have brought up a couple things that I might be able to fix.  ...And maybe the manuscript was a little long and did have a few too many appendices.  I'll work on the revision tomorrow.

Day 16-???: [Revise manuscript]

Day ??? + 1: Ok, the manuscript has been revised, it's much better than the previous submission, so now where should I send it?  Obviously I can't resubmit to journal X (eff u), so lemme look into journal X...

******************************

Does this above sound like a good timeline/plan that might help you get over a manuscript rejection?  Did you just get a rejection and are still stewing with righteous anger?  If so, then read my step-by-step instructions below!

First, when you initially get the rejection email or letter, you'll likely feel like this:

'Generalizability Concerns?'
Well Effff Yoooooo!!!

...at least, I typically do.  I bet there are few people who can receive a rejection letter and feel fine about it.

I recommend the very first thing you do is print and read the rejection comments (from the editor and the reviewers, if included) as soon as possible, preferably that same day.  

Next, do one (or more) of the following to the pages you just read:

• Burn them (watch out for smoke detectors)
• Smash them into a ball and punch it with your fist
• Shred it in the paper shredder.  Throw in a blank CD-R at the same time for a feel-good crunching sound
• Tear it to bits and toss it in the air like confetti (make sure to do this in a common area that you don't have to clean up)
• Stomp on it (works better if you first crunch it into a ball), and then kick the smushed ball across the room.
• Use them for gun/archery/whatever target practice (this one is probably the most satisfying)

Tony recently had his manuscript rejected from
the Journal of Drug Control Policy...
They didn't like his theoretical model I guess.

...feel better?  Only a little?  Ok, good enough for now.

Now that you're hopefully feeling a little better, reward yourself with a treat.  You've earned it, so treat yourself to a box of cookies (or a drink, or a steak, whatever).  You might be asking: "But my manuscript just got rejected?!? What did I do to earn a reward?"  Ahh, it's what you didn't do that earned you the reward.  Specifically you (probably) didn't:

• Fire off a snarky email to the editor telling them off
• Punch a hole in your LCD monitor
• Throw your laptop across the room and/or into the face of a coworker
• Curl into a little ball and cry (though it's ok if you did this one... you can still have a cookie)

So, treat yourself with cookies or something.

C is also for Craptastic Reviewers

Ok, now that you're hopefully a little calmer and feeling a little better, this next part is very, very important.  If you're not under an extreme deadline to get the paper published (e.g., you're racing against a competing academic so you don't get scooped), then take a week or two without looking at either the paper, the associated data, or (ESPECIALLY) the rejection email/reviews.  Just work on something else for a week or two.

After a couple weeks, you should be feeling a little more like this:

A little less pissed off... that's good!

...so, still a little hurt, and a little pissed off, but you're getting there.  Now, read through your original submission (the one that got rejected), and try and read it as if you're a reviewer and that is not your own writing.  As soon as you're done reading what you submitted, then read the rejection letter and reviews again.


Having read what you submitted, do the reviews seem a little more legitimate in their critiques? Oh really, your submission was perfect?  THEN READ IT AGAIN.  Ok, did you find something that might be worthy of revising a little?  Good.  So, how do you feel towards the reviewers/editor now?

Notice the smile?  That's the spirit!

Ok, that's good!  So, you probably noticed that your masterpiece of a manuscript had a few places that you could improve.  If so, great!  Get to work revising the manuscript, trying to incorporate into the revised version as many of the valid critiques from the reviewers and/or editor as you feel comfortable with.  Eventually, you'll have a finished revision that you can submit to a different journal, hopefully with a better outcome this time (if not, start back at the beginning of this post).

Regardless of whether your coping strategies are similar or different (or the advice above works for you), you should at least be willing to revise the manuscript and send it somewhere else.  Don't let a single rejection result in tossing the manuscript in the garbage (maybe after 3 rejections, but not after the first...).  Also, you have to keep in mind that any decent journal is going to reject more manuscripts than they accept, so chances are if you send them something else, they won't hold your prior rejection against you, and so you shouldn't hold the prior rejection against them (eventually... give yourself some time to be pissed with them).

-----------------------------------------------------------
Image Credits: Finger Old_Finger Young_Finger Tony Cookie_Monster Einstein

Having a creative outlet outside of academia

Yes, when I get stressed, I basically
turn into Tweak from Southpark.

One of the things I think is incredibly important for making it through grad school (and hopefully through the tenure track) is to have a creative outlet that doesn't involve academia.  These outlets let you get away (even temporarily) from thinking about classes, about the dissertation, about teaching, about publications, about the job market, and all of those other things that caused my eyelid to twitch from thinking about them too much. (Seriously, the eyelid twitch is God's way of saying "Look, you've got to calm down before I give you a stroke.")

When I was in grad school I had a number of different hobbies, some short-lived, some not, that let me escape from the grad school, homework (when I still had classes), research, and academia in general.  Due to my self-diagnosed hobby A.D.D., the assortment of things I used as creative outlets and general stress relievers was pretty varied. Here's a list of three things I did (and may still do from time to time), along with the reasons I found them to be enjoyable, and things to consider if you wanted to take any of these up in the future.

This is why I hate swimming.
Thanks Speilberg.

1. Kayaking
My wife and I took up kayaking after her parents took us kayaking a couple times during a summer visit. While I am not a strong swimmer, and generally am terrified of swimming in natural bodies of water (holla back if you saw Jaws as a six year old as well!), I found I actually enjoyed floating on top of the water, and being able to easily control where the kayak went.

The best part of kayaking, I think, is being forced to go out into nature and actually do something.  Kayaking gets you out of the house, let's you see and explore the natural surroundings in your area, and is very relaxing. Note that I am not talking about whitewater kayaking, which requires a helmet, fast moving water, an updated will, and the ability to not freak the f*ck out when you flip over and are strapped into your death trap of a boat. I don't think I would find that relaxing, and since I'm allergic to drowning, I don't think I'll be trying it anytime soon. Rather, I'm talking about kayaking on slow moving rivers and calm lakes.

Yeah... not this kind of kayaking...

The worst thing about kayaking is it's very weather dependent, and in the northeast that means about six months of the year it is too cold (or frozen) to go, and is also a no go when it's raining.  The startup costs are also one of the downsides, at least initially.

Kayaking has pretty high startup costs, especially if you want to buy your equipment and not rent it every time from someplace like L.L. Bean (which gets expensive quickly anyway).  A good entry level kayak will start at about $250, and you'll need a paddle ($50), a life vest ($25+), a carrier of some type for your vehicle ($50-$200), associated straps and ropes ($25), and things like dry bags ($10-$30) and other accessories.  It will probably cost at least $500 to get into kayaking (per person, if you go in as a couple like my wife and I did).  The nice thing is once you've bought your stuff, the actual kayaking trips will be virtually free, minus gas to drive places and maybe admission to state parks.

2. Iaido
My friend Matt convinced me during my first year in the Ph.D. program to join him for an iaido class. His sales pitch was basically "Do you like swords? You get to swing them around in Iaido."  Since I do, in fact, like swords, and wanted to swing them around without someone calling the po-po, I decided to give it a try.

Thanks to Iaido, I can theoretically
defend myself in this situation.
Very useful on the college campus.

Iaido means 'the art of the sword,' and involves learning the appropriate techniques for drawing a samurai sword (an iaito), making clear and precise cuts against imagined opponents, while portraying a level of zanshin (roughly a conciousness and energy), for certain scenarios (kata).  These scenarios stemmed from various attacks that a samurai may have encountered and had to defend against in his daily life in Japan during the Edo (and similar) period.  It does not involve attacking actual people.  For that, look into kendo, which uses body armor (bogu) and bamboo 'swords' (shinai).  Kendo also comes with a lot more bruises than Iaido, so keep that in mind.

Iaido is like this, but with
more 'wooshing' sounds

The best part of Iaido, in my opinion, is that it's very easy to let non-Iaido things slip out of your mind when you're doing the kata.  You have to have a LOT of focus to do the moves correctly, as it's not just where you swing the sword, but how you swing it, your stance, your 'energy,' etc.  It's also a decent low impact workout, and the people who do it are generally interesting people who are fun to hang out with.

The worst part is probably that it requires constant training if you want to be any good at it.  It's not really something you can pick up for the month, put down for six months, and pick back up for a month.  Like any martial art, to learn Iaido you likely need to find a school to join (run by a sensei), attend regularly, pay school dues, and practice outside of class.

Iaido has low initial startup costs, as you can practice in gym clothes (no shoes) and use a wooden sword called a bokken ($15?).  Many schools will let you practice with these for the first few classes until you decide that Iaido is something you want to continue to pursue.  When you get more into it, you can spring for the metal iaito sword ($100 - $1000+), and the samurai clothing (a gi and a hakama, $100+ for both).  The main expense from week to week will be the class fees, which could be free (unlikely), or could be  more than $50/week.  Finding an instructor may also be a challenge depending on where you live.

3. Home-Brewing Beer 
So let me say up front that I don't drink a lot of beer.  I drink beer, I just don't drink a lot of beer.  This, apparently, is not a good trait for being a home brewer, as you will quickly end up with hundreds of bottles (literally) of beer that you won't be able to possibly drink yourself, and with friends who are tired of drinking your homebrew beer.  That said, I attended a party where someone brought some homebrew beer, and I was intrigued.  It seemed like an interesting hobby where you could get creative and tweak recipes to brew what you liked.  When I got the opportunity some time later to purchase my own homebrew kit, I dove in.

It's best not to think
about what this looks
like and just appreciate
the finished product...

Homebrewing is a fairly simple process, but involves a lot of steps that are all equally important to getting a tasty (or at least drinkable) resulting beer.  The process starts by making what is essentially a hot soup of sugary liquid (whether extracting it from grain yourself, or buying pre-made sugar extract), bringing it to a boil with a lot of water, adding specialty grains to give flavor and color to the mixture, throwing in hops at certain times during the boil to give bitterness, taste, and aroma, cooling the mixture down, adding specialty yeast, and putting everything in a large glass bottle (carboy) or plastic bucket to sit for a few weeks while the yeast turns the sugary liquid (called wort) into beer.

The best part of homebrewing I think is the ability to customize your recipes to what suits your tastes.  For example, I'm a fan of big malty beers, so I tweak recipes for barleywines, porters, and stouts.  Other homebrewers love hops, and so they focus on brewing beers like IPAs.  Thus, you can make what you like, and in fairly large quantities, for about the same cost as buying beer in the store.  This gives you a creative outlet to make, tweak, and try out different recipes, and you have a tangible product at the end of it (albeit about a month later) that you can hold in your hand and say "I made this!"

The worst part of homebrewing, probably, is that you don't actually 'save' any money by homebrewing.  For instance, a good recipe for a high alcohol beer might cost upwards of $50 in ingredients, and produce about 4 cases (48 bottles) of beer, for a cost of a little over $1 per bottle.  That's not that much less than buying decent microbrew ($8/6 pack?) and is a lot more expensive than buying a 30 pack of cheaper beer.  Actually, if you want to brew a basic America Lager (think Budweiser or Coors), you're going to end up spending significantly more money than if you had simply bought the same type of beer in the store.  It also takes a lot of patience to brew the wort (5+ hours?), and then wait for the beer to ferment in the carboy or bucket (2-3+ weeks).  So if you're one of those 'immediate gratification' people... well.. you probably shouldn't be in a Ph.D. program for one, but you also will probably find it hard to homebrew, for two.

"After I had to mortgage my house to afford
all of my homebrew equipment, my wife left
me and took the kids.  BEST DAY EVER!"

Homebrewing has moderate startup costs, as you can get a startup 'kit' for around $100 (or more, depending on how fancy you want to make it).  The kit will come with a couple brewing buckets, hoses and other tools, and a basic ingredient kit (you supply a large pot).  You can also buy ingredient kits for a basic low alcohol beer for around $30-$40, more if you want something like a Barleywine, Stout, or Porter.  You can also customize your homebrew setup with larger brew pots, more carboys or buckets, higher quality tools, etc., so you can quickly drop a LOT of money into homebrewing if you're not careful.  If you stick with the basic setup, then you can figure somewhere between $30 and $60 per 5 gallon (~48 bottle) batch of beer.

"My 'hobby' is academic research... as well as
heart attacks... lots and lots of heart attacks."

In conlusion: There are plenty of other hobbies out there that range from costing almost nothing (e.g., hiking, volunteering, blogging), to being very expensive (e.g., collecting antiques, skydiving); that range from being relatively quick (swimming, working out) to relatively slow (knitting, reading).  The trick is to find something that interests you, fits in your budget (both in terms of time and money), and gives you a way to get away from academia, if only for a little while.  Having a non-academic-related hobby will help keep you sane, make you well-rounded, and give you perspective.  I highly recommend everyone taking up something of interest to them that is outside of the academic world.
--------------------------------------------
Picture Credits: Tweak Jaws Kayaking Iaido_1 Iaido_2 Carboy Beer_Dad Professor_Heart_Attack

Watch out for the post defense blahs!

So this post is a little more serious than some of the previous ones.  Around 4 months ago, I defended my dissertation.  I wasn't really aware of what to expect immediately afterwards, and hopefully this post helps some people who are in a similar boat.

My actual dissertation defense was fairly easy, with no curve balls from the committee or moments of doubt once the defense started.  While I stayed fairly calm in the weeks leading up to the defense, the day before I was ABSOLUTELY. FREAKING. OUT.  I felt that I was going to forget some key piece of my dissertation (like one of my research questions or main findings), and/or that I would completely choke under the pressure.

Everyone I knew who had already gone through it (including my own dissertation committee members) said the defense would be fine, and would be more like a friendly chat about my research and future directions than an actual defense of the work I completed.  While I "heard" what they said, I wasn't able to really internalize it and overcome my self doubt.

I imagine I looked a little like this
during the defense.  I blame coffee.

In any event, I wasn't able to sleep much the night before the defense, and so I went into my defense running on about 2 hours of sleep and a pot of coffee.  Despite being tired to the point of almost falling over, the defense went fine, and I became a 'Dr.' (which, from what I've been told, is conferred upon successful defense, while the 'Ph.D.' usage comes after successful conferral of the degree itself.)

Immediately after the defense, I felt pretty damn good. I was finished, and even if I got hit by a car and killed walking back to the parking lot, I'd go out as a Doctor (sorta).  After going back to my (on-campus) office and receiving some congratulations from coworkers, and calling my wife and my mom, I went home early to crash.  I slept for about six hours, got up, and then felt... well... nothing much.  Maybe a little relieved, but otherwise just sort of empty.  I knew I was basically finished (barring some minor formatting things required by the University beancounters), and was already a 'doctor' in title, but I still just felt a little... blah... if that makes sense.  I had a 'so what' kind of feeling about the successful defense, the culmination of what I'd spent the last 5 years pursuing, and the last 2 years directly working on.

If you DO see a sad
little bubble following
your every move, you
should probably call
a real doctor...

Unfortunately, this 'blah' feeling stuck with me for probably a month or so.  I had occasional flashes of 'F*ck Yeah!' feelings that would last a few minutes or maybe an hour (like after walking across the stage at graduation), but nothing constant like I had imagined prior to the defense.  It wasn't full-blown depression by any means (at least, I didn't see a sad little floating bubble following me).  However, there definitely were occasional feelings of sadness at the 'race' of the dissertation being over, spiked with a little 'Oh Sh*t' feelings at what I needed to do next (i.e., get tenure... eventually...).

After a while, the 'blah' feelings subsided, and were replaced with other feelings and thoughts, like stressing about upcoming deadlines, travel plans, and work, my upcoming cross-country move, and day-to-day things like working on research projects. I began to get more and more excited about other things, too, like submitting (revised) pieces of my dissertation to journals for publishing, planning out my fall syllabus, looking into housing options where I would be moving to, etc.  The feelings that were at the forefront of my thoughts, namely my new status as a doctor, eventually became a feeling in the background of my thoughts, that I now forget about more often than not.  A somewhat similar experience happened after I finished my Master's degree, but it was on a much smaller scale compared to finishing my Ph.D.

I'm writing this blog post not as a review of what everyone always experiences after defending their dissertation, nor is it meant as a warning of impending post-dissertation depression.  I simply want to throw out the idea that you might feel a little 'blah,' a little sad, a little depressed, a little angry, a little scared, or feel nothing much in the days and weeks after you defend.  So, if you know this *might* happen, you can hopefully deal with it if it occurs, whether that's by surrounding yourself with friends and family, going on a relaxing trip, diving into your next project, rewarding yourself with a post-defense gift, or even talking with a professional if need be.

Now that it's been about 4 months since I defended, on a typical day I usually don't remember that I'm a Doctor/Ph.D. unless something or someone reminds me (like when someone refers to me as 'Doctor.')  While I'm very proud of my accomplishment, 'being a Doctor' doesn't define who I am as much as I thought it would. That might change the more time I spend in my new position as an assistant professor, but until that point, I feel I'm just someone that successfully survived the dissertation and made it through the post-defense blahs.

-------------------------------------------
Picture Credits: Beaker Zoloft_Bubble

How to drown out noises and concentrate

One of my previous posts noted that I like to use active noise cancellation headphones to help concentrate on writing tasks.What I didn't really emphasize is that even with the noise cancellation technology turned on, the headphones still don't always keep all of the various sounds out, especially if people are talking nearby.

This is what happens when I start listening to the
music playing through my headphones... it's not pretty

With some music playing through the headphones, it's much easier to drown everything out, but then I'm stuck listening to the music, which itself gets distracting since I start listening to the lyrics (I was never one of those people who could fall asleep with the TV on).  I then start tapping my fingers/feet to the music, and pretty soon I'm completely distracted.

Hanson actually created
what is widely considered
to be the first 'White Noise.'
Fact.

One way around this is to play white noise through the headphones along with the noise cancelling turned on.  White noise can also be used to help fall asleep, meditate, etc.  Here's a very useful YouTube 'video' of 12 hours of just white noise: http://www.youtube.com/watch?v=1KaOrSuWZeM

However, listening to YouTube requires internet, and sometimes you won't have access to it (like when you're on a plane with your laptop).  If you're somewhat technologically savvy, you can use a special program (or if you're not, use this website) to capture that white noise YouTube video as a (very, very long) .mp3 sound file. Then you can cut it down to ~5 minutes using any sound editing program, and set it to loop on your desktop, laptop, or portable music player.  Voila!  A small, non-internet-based white noise audio file on an infinite loop.

Enjoy!  (And remember to turn off the noise cancelling option when you're done with it!  On a related note, the dollar store is great for finding cheap packs of AAA batteries that you will no doubt burn through fairly quickly with the headphones.)

---------------------------------------------
Picture Credits: Headphones Hanson Jay_and_Silent_Bob