How to 'do' research (Part 1)

[Sorry about the long wait time since the last post.  The first-year-on-the-tenure-track mid-semester crunch really slapped the hell out of me these past month or so.]

I was chatting with a new Ph.D. student the other day about their research ideas and (eventual) dissertation topic.  They said that they had plenty of statistical training (they have two master's degrees), but haven't had much training (yet) in the way of research methods.  Midway through the conversation about their research interests and background, they asked me one hell of a question: "How do you actually do research?  Like, how do you find the literature, figure out a research question, write up a literature review, choose a dataset, that kind of stuff?"

I think 'doing research' is closer
to this than anything else...

The answer to that question, I think, is both very complicated and very personal.  I assume that everyone does their research a little differently from anyone else, and it likely depends on whose research methods courses you took in grad school, which articles and books about research you read (and understood/liked), your own personal preferences and habits, the kind of research you're doing, and a whole lot of other things that aren't immediately coming to mind.  So, I don't think it's as simple as "Do X, then Y, then Z, and viola!  Top-tier pub here we come!"

Still, it's a fair (and good) question to ask, especially as a new Ph.D. student whose largely expected to hit the ground running.  While my answer right then wasn't really well thought out, insightful, or likely even coherent, I've had a little more time to think about it.  Thus, this is the way that I typically conduct research (or at least part of it -- I'm sure there will be more to come, hence the 'Part 1' in the title).  It seems to work pretty well for me thus far, so feel free to steal anything that looks like it might help you, since I'm sure I stole most of this from other people along the way.

1. Read the literature for stuff that interests you.

Researching uninteresting stuff
is like watching Golf on TV:
Both make me want to stab my
eyes out with a spoon.

Note the second part of that statement -- 'for stuff that interests me.'  I think one of the biggest mistakes people make as Ph.D. students and new researchers is choosing research topics only because it's what's 'hot' in the field, or because their advisor does it, or because they can be funded through a project to do it, or for any number of other reasons that don't involve being genuinely interested in it. (Yes, there's obviously something to be said for doing research that will eventually get you a good job, but you should be able to balance the marketability of a particular research subject with your interest in it.)

There's a reason I left my stable, fairly well paying office job to earn my Ph.D.: I got to the point where couldn't see myself doing something, as my career, that I didn't absolutely love.  I still feel that way, but now instead of that unloved 'thing' being the 9-5 office job of shuffling papers, it is now conducting research that I really couldn't give two sh*ts about (or even one sh*t about). So I try not to do that on a day-to-day basis, and I think I've been fairly successful at it thus far.

So, I guess the precursor to step 1 would be 'figure out what interests me.'  This likely involves reading a lot of broader literature in your field to find the things that you want to read more about.  Once you've done that, and you know that you're really, really interested in X and Y, go ahead and look up more literature on X and Y and read it.

Keep in mind that this isn't simply reading the latest 3 years of top-tier journal articles that used X and Y as keywords, but also involves figuring out who the big names are (and were) in the field, what the major studies on X and Y are that get cited over and over again are, and what types of methods, data, etc. are generally used in those studies.  You should also read studies that might not be on X or Y, but relate to it somehow (similar methods, similar units of analysis, similar theoretical models, etc.).  All of this reading allows you to talk coherently in your research papers about what's been done before, who has done it, how they did it, and how your stuff is similar.

So, now that you're knee deep in the literature of the stuff that interests you, what's next?  Well, it's something that takes place while you're reading the literature...


2. Figure out what people have been doing poorly, or not at all, in those studies (i.e., find the 'gaps' in the literature).

Similar to the UK warnings,
if you don't mind the gaps in
the literature, you'll be hit by a
metro train called 'peer review'

This really only comes from reading study after study after study.  Invariably you'll be able to spot the limitations of studies that you read, whether it's generalizability, unsophisticated methods, not using Theory A, not studying area B, etc. etc. etc.  Sometimes the authors will do you a solid and list the 'real' limitations of their studies in their respective limitations section.  From what I've seen, however, most of these so-called limitation sections are really just a list of one or two of the most minor problems that barely constitute an actual limitation, with no mention of the more serious limitations of the study.  So read the limitations sections with a grain of salt, but if something keeps popping up over and over again (e.g., 'this study only used cross-tabs and not multiple regression', 'we couldn't study X, so we had to study a proxy of it, Y', 'we could only examine one county of data, so generalizability is a concern', 'historical/longitudinal data were unavailable for X, so this was cross-sectional in design'), then it's likely one of the main gaps in the current literature.  You want to figure out what these gaps are, because they can help you either craft a research question or craft your study's methods section.

For instance, if you find that prior studies have all examined individual states/counties/cities, and no one's compared multiple places, that could be the source of a new study.  Your research question might be as simple as "What are the state/county/city characteristics that influence X", and even though this exact same question might have been asked by multiple prior studies, if your study does something those other studies didn't (e.g., examine multiple places rather than just one), you'll probably be making an important contribution to the literature.  If studies have all ignored Theory A, and you think Theory A could be important in explaining X or Y, then your research question might be "Does Theory A explain the occurrence of X?"

Ok, so you've read (and are continuing to read) the literature on your topic, you've found some things that are missing in the current literature that you might be able to address, and you've incorporated that into your research question.  Now what?

3. Write a good, but concise, literature review. 

You should by now have a handle on who the big names are in your field, what the major studies are, and what the general findings of the literature are on topic X.  Now comes what is, for me, one of the hardest and most boring things: writing the literature review. There are a number of guides on this floating around the interview (see one example here), and I guess now there's one more...

Note that I'm talking not about the 40+ page literature reviews that get published as stand-alone manuscripts in journals. Rather, I mean the shorter, ~5-10 page reviews you see at the front of a 'regular' journal article.

The literature review helps lay the foundation for your study - it talks about what people have done, how they did it, and what they found, but it also connects those findings into a coherent narrative, as well as exposing the gaps that you will be filling with your study. That said, the literature review can vary tremendously depending on how your study is set up. I have most of my experience with quantitative, policy-oriented studies, so that's the type of literature review I'll be discussing. I'm sure this varies for qualitative, historical, theoretical, etc. studies, so buyer beware.

In any case, the point of your literature review should be to give the readers a crash course in what's already been done in the field that relates to your research. Sometimes this will be fairly easy and direct, since a lot of people will have done studies before you on a similar or same issue.

Another research idea:
The effect of access to hoodies
on rates of juvenile delinquency.

For example, suppose you're writing about the relationship between doing poorly in school and juvenile delinquency. If there were a number of studies, or even a specific theory, that can be easily applied to this issue (let's go with Strain Theory, since apparently everybody in my field wants to test this at one time or another), then much of your literature review might be discussing the theory, what it says, and how these studies have supported (or not supported) this theory. So you might want to set up your literature review by first discussing Strain theory generally, and then noting how it easily applies to your specific situation (i.e., doing poorly in school leads to delinquency). Then you can cite prior studies on school achievement and delinquent outcomes, and note whether they support Strain Theory or not. You would note the problems with the prior research, and how more research on this issue is sorely needed (although, really, it isn't in this case... so please stop doing these studies, folks...). This sets up your study by placing it in relation to the prior research and theory, and also helps distinguish it since you've addressed the limitations of prior research.

However, sometimes you're paving new ground, and need to discuss two or three different types of research and blend them together into a coherent story.  For example, maybe you're trying to examine whether access to a cell phone influences juvenile delinquency. It makes sense, since with it you can chat with your friends wherever you are, and you can more easily decide where to meet up to vandalize the school, or burgle the liquor store, etc. Maybe no one has previously looked at this specific issue (if not, I call dibs...). BUT, people HAVE looked at how communicating with friends (in person) can influence delinquency. So, you review this prior research. Then, you realize that socio-economic status (SES) is going to influence who has a cellphone in the first place, and so you need to review the research on how SES can influence delinquency. Now that you've done this, you finish your literature review by pulling together these two prior research fields (communication and delinquency, SES and delinquency), and relate this to juveniles communicating via cellphone, and support why you think this would (or would not) have the same influence on delinquency as face-to-face communication with friends, controlling for SES.

I can't be too much more specific on how to actually organize your literature review, because the organization is going to depend on a number of factors specific to what you're studying, how you're studying it, and what's been done in the past. I recommend reading a lot of articles, mostly from top journals, that have looked at similar things and see how those were set up. This should give you an idea on how to set up your own review section.

Side rant: Tone
Let me take just a minute to talk a little about tone when discussing the prior literature. I'm assuming that the ultimate goal of doing this research will be some type of scholarly journal article that will need to survive the peer review process. If this is the case, then the journal will send your work to other 'experts' in field. While this might include grad students that are just beginning their careers, it might also include some experienced professors whose past research you've cited and discussed in your literature review. How you discussed their research, then, matters a great deal...

Let's consider the following examples:

Example 1: Standing on the shoulders of giants.

Example 1:
"The prior research has been
tremendously important, but
has suffered from some minor
issues which I will address"

Imagine you're one of those distinguished professors reading an article for a journal. The author discusses your article in a neutral fashion, commends certain things that it (you) did well, and even-handedly notes what was missing or otherwise could have been improved upon. The author is acknowledging the contributions of your work, and indirectly suggests that they are 'standing on the shoulders of giants' with their present study. This sets the tone for your thoughts on the article. You think to yourself "Good call, sir! I should have used more than three years of data. I'm glad you're taking the ball and running with it, and glad my research could contribute!" as you sip your brandy and puff on your Meerschaum pipe while sitting by a roaring fire in your study (or where ever/however distinguished professors review journal articles...). The rest of the article is well done, so you send off a positive revise and resubmit recommendation to the editor, with encouraging words for the author.


Example 2: All my predecessors were idiots. 

Example 2:
"The prior work is total
doo-doo done by smelly
caca-heads. My research
is good because it isn't
poop based." 

Now, imagine you're that same distinguished professor reviewing a very similar article. The author cites your work, but does so in a dismissive, critical tone. It's clear that they think your prior research was garbage, and that you were obviously a drooling idiot and/or untrained monkey. While not specifically saying this, they imply that their research can 'save the field' from its past horrific, worthless studies conducted by lesser researchers. "Humph and habberdashery!" you exclaim as you quickly type up your review to the editor with the strong recommendation to reject this horrible, insulting manuscript on the spot. "Little sh*t thinks my research is garbage, does he? Well good luck getting into [top journal of the field], muahahahahaha!" you cackle as you fire off your recommendation to the journal and casually toss the reviewed manuscript into your fireplace.

So kiddos, what did we learn from these examples? Well, first, apparently there are still some people in the world who use the phrase 'habberdashery.' Second, and more importantly, the tone you use to discuss the prior literature can be very important. Yes, you need to point out the gaps in prior studies so you can set up reasons for why your research is an important contribution. However, praising the prior researchers for their contributions, and implying that you're using their research as a basis for yours to help further the field, is a very, very different story compared to one where you're bashing the idiocy of prior researchers to promote your (obviously) much more sophisticated study. You want your manuscript to come across as 'standing on the shoulders of giants,' rather than 'chopping the giants off at the kneecaps.'  So tone matters.

Hopefully what I've written so far will be somewhat helpful to those of you just starting the research process. I'll be posting more on this topic in the future, but this should get you started.

--------------------------------
Image credits: Golf, Hoodies, Old Scholar, Baby Scholar, Mind the Gap, Then a miracle occurs (Harris, 1977)