Posts Tagged ‘Research Methods’

For the Final Exam (12/18/15)

December 13, 2015

Before

  1. The final draft of the proposal is due by the end of the exam period, and attendance is mandatory. A few of you have spoken to me about scheduling issues, and I’ll deal with those individually; but generally speaking, it should go without saying that it’s a bad idea to schedule something during an exam period.
  2. Prepare a presentation on your proposal (a satisfactory presentation will count for a score of 100% on one of your previous assignments — I’ll choose the assignment — so think of it as extra credit). As I mentioned in class, I’m not asking for ‘I Have a Dream’-level speechifying; this should be a 5-10 minute presentation, with or without slides/visuals/charts (although the superior presentation would be with), addressing the following questions:
    1. Who are you and what’s the title of your proposal?
    2. What’s it about?
    3. What’s your research question and/or research hypothesis?
    4. What’s your research design? (Brief Overview)
    5. Why should this be funded? (IOW, What’s the contribution your study would make to the relevant scholarly literature on your topic?

During

  • Presentations (here’s some how-to advice, if you haven’t already seen the post above this one)
  • TBA: we’ll probably devote a little time to some topics we haven’t had a chance to really delve into, but think of this as a time to put the finishing touches on your proposals and accompanying materials, or to get my help.

After

  • If you’ve followed instructions meticulously, relax: everything should be okay.
  • If not…
Advertisements

Replication, Replication, Replication

December 7, 2015

I couldn’t resist the allusion to my earlier, lame little joke about participation. Anyway, everybody’s been having so much trouble wrapping their heads around the replication homework assignments that I thought I’d try to provide a bit more background as to why replication is such a core part of doing science. If you’re at all familiar with Milgram’s obedience study (discussed in our textbook and every other textbook that ever talks about research methods, or research ethics, or, well, obedience), you’ll know that his findings were surprising, even to Milgram. When you get surprising results, the most sensible thing to do is to stage a replication — a ‘do-over’, as Janet Ruane (2005) dubs it — in order to establish that the results the first time weren’t just a fluke. If  you repeat your study and get much the same results, you’ve now got grounds for more confidence in the findings. If other people replicate your study and get much the same results (after all, what if you really, really wanted to be surprised? what if you therefore saw what you wanted to see, and not what was actually there?), then you’ve got even more confidence in the findings.

Even after you’ve established some confidence in the findings, you might wonder how much they can be generalised to other settings, populations, and circumstances. And so you might seek to replicate the study but add some twists, in order to test the scope of the study’s external validity. All of these options — repeating the study step-for-step, having other researchers try to replicate the findings, tweaking the study’s setup — are alluded to in the replication HW assignments. The point is to digest the methods and findings of a previous study, and then to ‘reverse-engineer’ it in order to come up with your own version, which can be more or less faithful to the original (you should always give reasons for the changes you make, of course).

Why would anyone want to do that? you ask. Well, whenever someone ends up with surprising or controversial findings, we have good reasons to seek to replicate them. But often the most surprising ones or the ones that are only marginally significant, statistically speaking, are the ones that can’t be replicated: ‘Suppose you have two well-designed, carefully run studies, A and B, that investigate the same phenomenon. They perform what appear to be identical experiments, and yet they reach opposite conclusions. Study A produces the predicted phenomenon, whereas Study B does not’ (Barrett 2015). What we have here is a failure to replicate — but it’s a telling failure.

And thus we eliminate tempting but misleading bits of falsehood from the fund of human knowledge. Science can get things wrong, but this doesn’t mean we can do without science; in such cases, as one researcher argues in a recent, highly informative interview, our best hope for a corrective is … more science.

Further Reading/Notes Toward a Personal Canon

Patten, Mildred L. 2014. Proposing Empirical Research: A Guide to the Fundamentals, 5th Ed. Pyrczak Publishing. For advice on how to ‘reverse-engineer’ when you think you’ve got reason to introduce some changes in the original design, see Patten (2014:19, Topic A) on modified replication.

 

REFERENCES

Barrett, Elizabeth Feldman. 2015. ‘Psychology is not in crisis’. New York Times, 1 September, p. A23.

Ruane, Janet. 2005. Essentials of Research Methods: A Guide to Social Science Research. Blackwell.

For Friday, 12/4

November 20, 2015

Happy Thanksgiving! (But don’t forget that the Up Movies lab is due by midnight tonight!)

Before

  • HW due: In honour of America’s Greatest Holiday, and in keeping with our recent examination of open-ended interviewing techniques,  we shall take part in The Great Thanksgiving Listen. Interview a parent or grandparent about their lives (or some other elder, if your parents/grandparents are unavailable). If your research topic lends itself to it, you can incorporate questions that you might ask a participant in the research you’re designing. If your parents were not born this country,  you might begin by asking them about their experiences as immigrants, as in this much talked-about episode of Aziz Ansari’s new show. You might not cry as much as Ansari’s friends reportedly do in watching the show, but you will want to have a box of tissues available, just in case. Write up your findings in the ‘Qualitative Interview’ lab.
  • Schutt, Chapter 12 (‘Evaluation and Policy Research’)

During

  • Choosing a Data Collection Method
  • BREAK
  • Causation

After

  • HW: TBA

For Friday, 11/20

November 13, 2015

Before

During

  • Lab: Up Movies (Fifty-Six Up). This is due by 11:59 on the 20th!

After

  • HW: TBA

For Friday, 11/13

November 6, 2015

Here’s a glossary that explains some of my most frequent grading comments. (I’ll mention it in class as well.) As always, Friday’s agenda is below.

Before

  • HW due: This was supposed to be the Qualitative Data Analysis, but the links to the data are broken and I’m having trouble repairing them. This is a golden opportunity for you to catch up on HW you haven’t turned in yet. You should also start drafting your final proposal if you haven’t started already, paying special attention to the literature review.
  • Schutt Chapter 7
 (‘Experiments’)

During

After

  • HW: TBA

For Friday, 11/6

October 30, 2015

Here’s a link to the mid-term feedback survey I mentioned in class. You should fill this out anonymously; just sign out of Google before you click on the link.

Here’s our upcoming agenda:

Before

  • HW due: We’ve repaired the outline doc associated with the Design Your Sampling Strategy. I won’t assign any new HW this week in order to give you a chance to catch up on your homework. If you didn’t turn the sampling HW in before class last Friday, however, I will still consider it late. If you are unsure about how to complete an assignment (in this case, because the guiding outline doc had been altered), you should contact me before it’s due.
  • Schutt (2015), Ch. 11 (‘Qualitative Data Analysis’)

During

  • Lab: Exploring Sampling, cont’d. Do not go beyond Question #5 before class this Friday. If you fail to follow instructions, I will clear your attempt.

After

  • HW: TBA

For Friday, 10/30

October 23, 2015

Before

  • HW due: Design Your Sampling Strategy
  • Schutt, Chapter 10 (‘Qualitative Methods’)

During

  • Lab: Exploring Sampling

After

  • HW: Qualitative Data Analysis

When and Where I Enter: On Reviewing a Literature

September 14, 2015

Imagine that you enter a parlor. You come late. When you arrive, others have long preceded you, and they are engaged in a heated discussion, a discussion too heated for them to pause and tell you exactly what it is about. In fact, the discussion had already begun long before any of them got there, so that no one present is qualified to retrace for you all the steps that had gone before. You listen for a while, until you decide that you have caught the tenor of the argument; then you put in your oar. Someone answers; you answer him; another comes to your defense; another aligns himself against you, to either the embarrassment or gratification of your opponent, depending upon the quality of your ally’s assistance. However, the discussion is interminable. The hour grows late, you must depart. And you do depart, with the discussion still vigorously in progress — Kenneth Burke, The Philosophy of Literary Form, pp. 110-111.*

I’ve touched on the subject of literature reviews before in at least one previous post, but I’m going to try to centralize a number of disparate sources and bits of advice here. This is a post that should grow quite a bit over the next few days weeks months years…

Further Reading/Notes Toward a Personal Canon

Booth, Wayne, Gregory Colomb, & Joseph M. Williams. 2008. The Craft of Research, 3rd Ed. Chicago: The University of Chicago Press.

Chisholm, Chad. 2017. ‘The Rhetoric of an Excellent Essay’. Intercollegiate Review, 7 April. More on the research literature on a given topic as a Great Conversation, with an extremely helpful introduction to the fundamentals of Roman rhetoric to boot!

Google Scholar. As Rossman (2010) notes, Google Scholar is good at giving you a sense of the ‘invisible college’, but is not as helpful in directing you to good theory.

‘Literature Reviews’. n.d. Purdue Online Writing Lab. Retrieved 16 September 2015 (https://owl.english.purdue.edu/owl/resource/994/04/).

Mills, C. Wright. 1959. ‘On Intellectual Craftsmanship’. Appendix to The Sociological Imagination. Oxford University Press. Very tempting to interpret all of Mills’ talk about ‘craftsmanship’ and the ‘mature workman’ (p. 197) in terms of his blue-collar origins, but that’s a story for another time. A widely cited classic.

Patten, Mildred L. 2014. Proposing Empirical Research: A Guide to the Fundamentals, 5th Ed. Pyrczak Publishing. Topic 8 (‘Finding Ideas in the Literature’ will help you build on existing research; Topics 19 (‘Organizing Literature by Topics’), 20 (‘Evaluating Research Literature’), and especially 21 (‘Considering the History of a Topic’) are all chock-full of great advice; use Topics 22-26 to help you with the actual writing.

Rossman, Gabriel. 2010. How to Review a LiteratureCode and Culture: Stata, Sociology, and Diffusion Models, 7 September 2010. Retrieved 14 September 2015. See especially the tips on how to ‘snowball forward’ and ‘snowball backward’!

Sociological Abstracts. See again Rossman (2010).

REFERENCES

Burke, Kenneth. The Philosophy of Literary Form. Berkeley: University of California Press, 1941. *I found the Burke quote  here — but I only got there because of this great little essay on how the Trump administration might have avoided its recent immigration travel ban woes if someone had learned the basic readin’, writin’, and researchin’ skills that are a core part of any decent college course.

For Friday, 9/18

September 11, 2015

If you know that you’re coming late, or have to leave early, or have to take an important phone call/text message, please let me know ahead of time. That’s just common courtesy—and common courtesy is an important component of your class participation grade.

Before

  • HW due: Annotated Bibliography assignment. This is on Bb; click on the ‘Proposal Work’ sidebar.
  • Schutt (2015), Chapter 9 [Chambliss and Schutt, Chapter 8 and Appendix A]
  • Gabriel Rossman, ‘How to Review a Literature’
  • Patten (2014), Topic 8 (‘Finding Ideas in the Literature’)
  • Patten (2014), Topic 9 (‘Considering a Body of Literature’)
  • Patten (2014), Topic 10 (‘Considering Theories’)

During

  1. Review: Content Analysis. If I’m still busy setting up at 1pm, just begin the lab (see #2 below). You can pause and save your work when we stop to discuss class business (e.g., ‘Are you getting updates every time I edit a post on the blog?’) and review the method.
  2. Lab: Manifest Content Lab. You’ll need the five keywords that you generated from last week’s freewriting exercise.
  3. Presentations, cont’d: Model Proposals #3, 6, and 7
  4. Quiz: ASA Style

After

  • HW: TBA

Quotable

On Choosing a Research Topic

September 10, 2015

When it comes to advice on choosing research topics, opinions are like kittens—people give ’em away. (I never tire of quoting that line from Modest Mouse.) Here are a couple of the best:

  • I never met Charles Tilly; by all accounts, he was an extremely generous mentor as well as a world-class scholar. Here is one byproduct of that generosity. It’s a PowerPoint presentation on choosing a dissertation topic, but it applies to the choice of research topics above as well as below that level.
  • Here’s another link to Professor Elin Waring’s advice on the subject.
  • More to come…