Rough Theory

I’ve mentioned previously that I’ve found myself reading much more draft student work this term than I normally do. While this has been a somewhat sudden development, the work involved is continuous with work I’ve done in other academic contexts – I don’t think I’m anyone’s notion of a master of English prose, but I have done a lot of thinking and teaching on academic writing, and believe I can provide at least passable assistance to most students who are struggling with the genre.

What has been more surprising this term has been the number of requests I’ve been receiving for consultations on research methodology. I realise it sounds a bit odd to be surprised by this, given that I’m teaching a research methods course. And I do love teaching into this course – it’s my favourite “subject” to teach, specifically because I enjoy the process of workshopping the logical connections between students’ broad interests, their narrower research questions and their methodologies. It’s one of the most creative teaching processes I currently engage in – an intrinsically unpredictable, decentred, energising form of teaching practice that would be very difficult to replicate in other contexts.

Still, before being invited to teach the course, I had never previously thought of myself as any kind of methods “expert”. Having taught the course for a year now… I still don’t… And yet here I am, sketching on scraps of paper and whiteboards, trying to help people map out connections between intellectual interests, research questions and methodologies… And, since I like the work and want to continue doing it, I’m engaged in a process of trying to increase my skills so that they begin to seem somewhat proportionate to the faith people are placing in them… Problem is, I’m not sure that all of this effort is getting me any closer to any kind of methodological expertise – instead, I mainly seem to find myself refining ways of communicating some fairly straightforward dimensions of academic practice, such as (in no particular order):

– a research question is not the same as a topic or an intellectual interest: you want to connect your research question to some kind of broader intellectual interest, particularly for a larger project such as an MA or PhD, as this broader interest will help sustain your energy and focus through the more taxing parts of the research and writing process. For most people, though, the actual research question will not come close to directly addressing the entirety of their intellectual interests. Instead, the research question will more likely involve locating some small window that lets you peer onto some narrow slice of your broader intellectual project;

– a research methodology is not a “to do” list or some kind of descriptive account of the actions you plan to perform while undertaking your research – instead, there should be some clear, logical, explicit connection between your methodology and your research question, such that your reader can grasp why this particular methodology is, if not necessarily the only possible choice, then clearly a good choice for answering the question you have raised;

– your literature review is also not a list of everything you’ve read on the topic: it is a strategic discussion of key literature, written to convince your reader that someone really ought to answer your question, using your methodology;

– in your thesis, as in all academic writing, your claims must be commensurate with your evidence – and therefore your methodology shapes, and is shaped by, the sorts of claims you want to make;

– small claims, well supported, can be very valuable; large claims, poorly supported, usually aren’t;

– claims do not always have to be generalisable to be interesting and useful, sampling techniques are not always the best way to choose sites or participants in a research process, etc.: before designing your research methodology around normative concepts borrowed blindly from statistical (or any other kind of) analysis, look first to your research question to see whether these concepts apply to your project;

– academic research requires critical agnosticism about the results you might obtain – your methodology must allow you to be open to the possibility that you might discover something unanticipated, and your methodology should communicate what sorts of findings might cause you to doubt your premises, approach or conclusions;

– practical and personal considerations inevitably shape methodological choices – most of us would do different kinds of research given an infinite budget; many of us stumble into promising lines of research via personal networks and autobiographical accidents; etc. There is no need to hide these idiosyncratic dimensions of your research design. There is, though, a need to explain to what degree, and for what purposes, someone else can trust and rely on the validity of your research results. For most kinds of academic arguments in the social sciences, this explanation requires a movement beyond autobiographical storytelling, and into some engagement with the reasons other people might still find your research meaningful or useful;

– don’t be afraid to be methodologically promiscuous – your questions may be best answered by a combination of methodological techniques, each designed to cast light on different dimensions of the problem.

I could go on – there are some methodological issues, for example, that commonly trouble students trying to do historical or theoretical theses, and students doing these kinds of work must often learn specific strategies for explaining how they meet standards of academic rigour, etc. But the basic point is that much of this generalisable information falls more into the category of socialising students into academic culture, than it does into something that is truly “methodological”. The rest – all the properly methodological work – is often highly specific to individual projects (and heavily shaped by the practical constraints of student research, such that methodology consultations often become exercises in realising that the student doesn’t have the time or money to do x, and then redrafting the research question and methodology with these constraints in mind). I actually like this aspect of socialising students into academic “discipline”, as I also like the idiosyncratic and creative dimensions of individual project design. But I’m not sure this is what students expect to receive from their methods course – my sense is that most students come in expecting, e.g., to spend a lot of time practising the hands-on mechanics of the research process, rather than conceptualising the underlying logic of their project…

At the same time, I’m finding that, while I really do enjoy this kind of work, there are some kinds of methodology consultations that, like some kinds of draft student writing, elicit a certain degree of dread. I struggle slightly, for example, with students who come into methodology consultations with strange, unmovable fixations about what methods they must use, particularly when these are combined with an equally unmovable commitment to a research question that really cannot be addressed through the preferred methodology. Some students like this behave as though they have a sort of reset button: I’ll think that we’ve made good progress in a particular meeting, only to find, next time around, we’re back to where we started. I can usually go through a few rounds of this before the dread actually begins to kick in, and I start dropping in on their supervisors to enlist outside support…

In methodology consultations, as in most other contexts, I am generally dismayed when students resist the notion that they need to engage with data or theories that might challenge their core assumptions. To me, this is such a fundamental dimension of academic work that I don’t really understand how someone decides to write a thesis without some basic commitment to the principle that research is a process through which we test our ideas. These sorts of situations also sometimes give rise to ethical dilemmas for me, particularly when the level of consensus on an issue within a particular discipline makes it highly likely that the project would not trouble a supervisor or an examiner, but when I am aware of the contentious character of a claim when viewed from the standpoint of other disciplines. If a student is resistent to the prospect of measuring their claims against evidence from outside their discipline, am I meant to assist them in designing a methodology that would be acceptable within their field, but about which I have deep personal misgivings? My general approach has been to keep pushing the need to engage with conflicting evidence, and this usually ends up being persuasive, in the end (at least until they escape from my class, at which point they may simply breathe a sigh of relief, and move back to more comfortable terrain…). But I have, on occasion, had students outright refuse – and I’m aware that, once they leave my course, they’ll be back within the confines of their own discipline, with no immediate requirement to engage more broadly…

The most difficult situations for me, though, are the occasional students who seem to be looking for something that I’ve taken to calling “research porn” (I wonder how long it will take before I regret posting that word on the blog…) – students who seem to expect only climaxes from the research process – who are intolerant of the hard slogs, unexpected results and periodic dead ends that arise en route to interesting data. This kind of problem actually doesn’t usually manifest when I’m discussing the original research design (although I occasionally get a hint that this might become a problem when the original methodology is a bit over-literal, e.g., when someone is seeking evidence for phenomena that would intrinsically need to be observed somewhat indirectly, but proposes a methodology that strongly suggests that they believe they will be able to find direct and explicit evidence). The problem usually hits, though, the first time the student begins to undertake research – and realises, as all of us eventually do, that research involves a lot of rather boring, tedious activity, that research subjects don’t always say what we expect, that our data doesn’t fall neatly into the pattern we predicted, etc. While these experiences can potentially lead anyone to redesign their research project, for some students these sorts of frustrations kick in early and often, leading to a seemingly endless research redesign process that seems to be motivated by the desire for some sort of unattainable, idealised vision of unadulterated research success. At some point, I start feeling like I’m facilitating work avoidance, rather than providing methodology advice – and I have to fend off impulses to reach for my copy of Freud’s “Analysis Terminable and Interminable”…