My worry is that I’m redesigning blind – I’ve been involved in math curricula designs before, but it’s been a very long time, and I’ve lost the sense of what students don’t know, of where they fall down… So there’s a large risk of pitching a conceptually-based course too high, and provoking some kind of repeat of my advanced politics course, where I had students approaching me in tears about the course requirements… ;-P

I tend to agree with you on mechanics, but am not sure it can be excised from the course this year (I will also be involved in a longer-term course redesign process, which is aimed at rethinking both the quantitative and qualitative methods courses, and this process will provide some scope for rethinking issues like this). My off the cuff recommendation was that they distill out software and mechanics style training into a separate intensive course, as this will probably give a sufficient orientation for students to drill in on their own.

That said, it is a recurrent complaint about the more advanced Research Strategies course that we don’t provide more hands-on software and similar training – that we focus too exclusively on the logic of research design. I don’t personally agree with the complaint – I tend to think that research design is actually very difficult, and am always worried that the one-term Research Strategies course isn’t sufficient to communicate how to do this to a reasonable level of competency – but I have to acknowledge that a decent percentage of even our more advanced students disagree…

Back to the quant course: I’ve received some informal feedback from a few of my first years (who are no doubt looking forward nervously to the course…) that they don’t really understand why they have to do any “methods” training at all, if they have no intention of doing Honours. I realised in the course of these conversations that my “stock” answer from the US doesn’t really apply in Australia – in the US, the undergraduate degree is fairly generalist, and so you could explain to someone that they were being credentialled as generally competent in the social sciences – a credential that meant, among other things, that they could make competent evaluations of social science research. In practice, this is the motivating idea behind our common course architecture courses here – that all students coming out of vaguely social science disciplines need a certain base of common knowledge, whatever their specialisations. This motivating idea, though, seems still to be very alien to most students – whether because it has not been articulated, or because the vocational nature of Australian education weighs so heavily against it that the concept isn’t easily absorbed…

I think I keep finding it so shocking that people wouldn’t want to use their undergraduate experience to… er… learn something… ;-P I seem to fumble unconvincingly in these discussions… I actually do have a response – really, if you plan to work in any professional field, you need to be able to read and assess research. And this is actually much easier to do if you learn the underlying concepts that are generative principles for understanding everything else…

So let’s say I target the lectures toward major concepts – would it be better to leave out the “research design” issue (this course is advertised as a course on research methodology, even if it moonlights as a course on statistics), and focus on core statistical concepts (and perhaps a bit on developing a basic sensitivity to mathematical plausibility)?

I still have the inherited structure with the labs, so I’ll have to work out something to do with them. After writing the original post, I had an opportunity to read through the lab manuals – they’re clearly written and well-organised, but do things like toss out vocabulary very quickly – including some vocabulary, to be honest, that would actually be fairly unusual in the “wild”, and more easily expressed using everyday terms – but are very much oriented to “do this, then do that” and magic occurs at the end… My problem is that we don’t currently offer any other structure for software training other than this course – and until this has been resolved through a more fundamental redesign of the methods courses, the software training component apparently must remain.

And I have to figure out what the main assessment tasks would be – and how to produce adequate supporting materials to guide students through them (again, my major problem is that I don’t have a “feel”, sitting here now, for where students are going to fall down). Your notion of restricting to a dataset is good. Ideally, in a course like this, you want multiple assessments so that students can make mistakes and still recover – then the question becomes whether to offer multiple types of assessable tasks, or to have each task contribute a different portion to a final overarching project (which is, sort of, the approach in Research Strategies)…

Many thanks for responding to this…

Your biggest problem is that most social science students are petrified of numbers. Rather than seeing the statistical concepts as just another language for expressing some relationship, it is a mess of meaningless figures and mostly meaningless operations. I don’t really see the value in teaching software in either of the research courses – you can run most of these tests in any basic spreadsheet – but my background might make me an outlier in that assessment.

Nor am I sure that teaching the mechanics is actually terribly useful. Social scientists who can run regression tests but don’t know how they work are a menace. Most of it will be forgotten quite quickly anyway. I’d rather see a course that worked through major concepts – averages and variance, correlation and causation, significance etc. – in the lectures and tutorials, focusing on the what and why, and using examples/theoretical readings, and only mentioning in passing that “oh, there is a test for this”. You seem to be leaning in that general direction anyway.

Actually, I am not sure you even can teach mechanics. It is the sort of thing a weekly/fortnightly exercise sheet and instructions for how to go away and do it work best. Perhaps allied to a drop-in lab time for students who just don’t get it?

A lot of the project complications will be data related. If you restrict it to a single (largish) dataset – say ABS statistics for Local Government Areas – you could get students to formulate a question based on available variables, do a small lit. review, choose and justify a statistical method, and then run their analysis. That sounds easier than it probably is.