Rough Theory

Theory In The Rough

Category Archives: Teaching

Wide Margins for Error

In the latest round of marking, I’ve noticed that one of my students has submitted an essay with these almost ludicrously wide margins. It’s a good essay, and comfortably long enough for the assignment, so I don’t believe they were trying to make their work look longer than it is – my guess is that they’re trying to provide more room for my comments, which have a tendency to scrawl around the borders and dwindle to illegibility when the margins are small. Wide margins as an iatrogenic consequence of a pedagogical – or, in this case, feedback – style…

This round of marking has also gotten me thinking about random patterns in student assessment: you occasionally get these long runs of papers at a similar level of quality – this time around, my marking started with a very long run of papers that had really struggled with the assignment; last time, my marking started with a series of excellent papers. I know from experience that, as you move through the bulk of the marking, the results eventually begin to distribute more normally. Nevertheless, it’s hard to prevent some anxiety when you hit these long runs at the beginning – was the assignment massively too easy? much too difficult? There’s a strong temptation to search for meaning in the data before the picture is complete…

On a more administrative note: the site is open again for comments, but may still be periodically up and down for the next little bit. If anyone notices any odd issues, feel free to post about them here.

Marking Blind

Whenever possible, I blind mark student assessments, so that I have no idea whose paper I am marking. This also has the side effect that I rarely remember what particular students have made in my course, since I just quickly transfer all the marks into a spreadsheet at the end, and grades tend to blur together. I occasionally guess that I must have given someone a bad mark from how they, e.g., won’t make eye contact and hurry along when they pass me in the hallway… Absent such social clues, though, I generally don’t know whether someone was a “good” student, in terms of their formal assessment performance. I do, of course, remember things like how well someone engaged with the material in the classroom, but this never correlates strictly with marks in the course.

Some students, of course, have such a strong and individual voice in their written work that it is essentially impossible not to recognise whose work I’m grading. These students tend to be on the better end, since finding your “voice” in academic work is fairly difficult to achieve, so I’m not generally too worried about recognising who they are as I’m grading. And in courses like Research Stratgies, we spend so much class time discussing students’ work before they submit assessable material, that it’s essentially impossible to disguise authorship.

When I can blind mark successfully, though, I tend to get strong reactions from students who do well on an assessment or two, and then suddenly receive a poor mark. The reaction isn’t always negative – I’ve occasionally had students admit to having concluded that, after I gave them a high mark on multiple assessments, I must now think they are an “HD” student, and would therefore give them a high mark regardless of what they turned in. They might wince at the low mark, but still seem strangely pleased that I noticed that the work hadn’t been up to par…

Not all students share this view: I’ve had a number of students feel that early high marks were a sort of promise of what they would make in the course, and who therefore feel very betrayed by a sudden low grade. Sometimes there is a substantive objection – i.e., the student feels that they did a similar kind of work across two assessments, but received a very different grade – and then we need to work through why a strategy that might have been appropriate to one assessment, might not work for another (I’ve been tending recently to mix assessment types, on the theory that this would give students a greater chance of having at least one assessment draw on skills in which a student has a relative strength. I may drop this strategy, though, as on balance I think it may be causing more work for all students, as they have to readjust to new assessment styles as well as learn new content, and I suspect it particularly disadvantages students who, as a matter of temperament or skill, need multiple opportunities to practice and perfect their work within the constant structure of a particular assessment style…).

Sometimes, though, the objection to a sudden low mark does not involve a contestation about the quality of the work, but about something more like the social compact involved in marking – the student’s expectation that, since they have demonstrated their abilities to me, they should therefore receive a mark for the course that is indicative of their potential, even if they occasionally fail to live up to this potential in an individual assignment. Blind marking renders this impossible – forcing students to demonstrate their skillls on each assessment if they want a high mark for the course.

I generally do design in some failsafes to protect students from truly cataclysmic assessments – I usually offer some option to revise and resubmit, or I count the best x of y marks, or similar. In my experience, though, students rarely avail themselves of the failsafe options, and the existence of failsafes doesn’t seem to mitigate reactions to poor marks…

Acknowledgement Website

I just wanted to put in quick plug for the Acknowledgement project – a joint endeavour between the University of Melbourne and Monash University to develop plagiarism and academic integrity materials designed for academics, rather than for students. I attended a brief presentation about the project yesterday, and have just been playing around with the Acknowledgement website this morning (note that the current website is still in demo form, but will apparently continue to be available to the general public even after the website has been finalised; also note that the link above goes to the University of Melbourne, but the resource is apparently also available via the Monash University website).

Aside from providing the standard assistance with, e.g., developing student assessments to minimise opportunities for plagiarism, or managing cases of plagiarism once they occur, the site also provides resources on the thornier academic integrity issues confronting established academics – asking us to explore how we feel about “self” plagiarism; investigating the academic integrity responsibilities of an academic reviewer or editor; wondering how we should acknowledge more intangible forms of intellectual influence over our own work; etc.

The website provides quite clear and well-organised materials, including extensive references to further resources. It also includes a series of videos – under the “academic stories” sections within individual topics – that are based on interview material conducted as part of the research for this project, gathered into narratives that, to preserve anonymity and confidentiality, attempt to pull together and express the major points from a range of interviews. These stories generally attempt to provide a sense of the range of views present within the mainstream academic community on specific issues. Occasionally, the videos (which star actors apparently normally used to act out specific medical complaints for real-time medical simulations at one of the partner universities) veer a bit into camp: I particularly enjoyed this video on the virtual university, for example, which begins with a man ranting:

I know that in the 21st century we are supposed to be all about the virtual university and so on. But I have opted to use the Internet as little as possible; I am not a fan of email or Google; in fact, I don’t watch television; I avoid “news” in all its forms.

Why? Because I believe that the mass media erode the kind of originality that I am bound to strive for as an intellectual. Academic freedom to me means, in part, freedom from constant superficial chatter, and from overloading with what passes for “information” these days.

The humor-value, I should note, is mainly in the delivery: I have known people who espouse very similar views, so I can’t argue that the content doesn’t represent a certain approach to academic work. I’ll pass over, for present purposes, what I think about this approach… ;-P

The website also includes some very interesting self-tests – like, for example, the self-test on how to “proof” assignments against plagiarism – that enable you to measure your own thoughts about academic integrity, against findings from the broader research literature. The site also works hard to provide sic et non links to conflicting opinions in the literature on various topics.

My brief look suggests there is some very good material here – for students as well as academic staff. The website is open access, so anyone can have a browse around.

Becoming the Teachers We Didn’t Have

Like parents who want to spare their children the worst experiences from their own childhoods, academics often choose pedagogical strategies in the hope of sparing their students from their own worst university experiences. A post from See Jane Compute reminded me of this issue. Jane reports:

The story of how I teach intro courses has to start with my own first experience with a programming class in college. In short, it was a complete disaster. Now, I was not a complete newbie–I had taken a few computer courses in junior high and in high school, although none on the “serious programming content” or AP level, so I at least knew a bit about how computer programs worked. And for the first few weeks, everything was fine. But about halfway through the class, we were introduced to a concept–and I don’t even remember exactly what it was anymore–that I just could not understand. Unfortunately, programming courses build heavily on previous material, and so this pretty much sealed my doom. Also, the class picks up steam and more material is covered in the second half, so I quickly found myself drowning. On top of everything, the professor was one of those brilliant types that have no business teaching undergrads (i.e., couldn’t teach you if you weren’t already a programming genius), the TA spoke little English, and I was intimidated by the fact that all of my classmates either (a) seemed to get things much faster than I did, or (b) cheated their way through the class. By some miracle, I got a B in the class–to this day, I have no idea how I pulled it off. But the damage was done: I now *hated* programming, an activity that had only brought me joy in the past.

She then goes on to say:

This experience, more than any other experience I had as an undergrad, colors the way I approach teaching. My number one goal in teaching intro courses is to make sure that my students’ first experience with programming is overwhelmingly positive (or as positive as possible). I remember the despair I felt, and I keep that in mind as I introduce concepts, make up homeworks, and talk to my students in class and in office hours. I try to make sure that my interactions with students, and what I do in the classroom, encourages them rather than discourages them.

I have a similar undergraduate memory that haunts my teaching practice. Read more of this post

You Don’t Know You’re in Trouble When…

Seeking reasons to procrastinate in the face of the mound of marking that sits on my table, I’ve been spending a leisurely Saturday morning reading various studies of cognitive bias. (Note to self: this is probably not the best way to prepare for marking first-year undergraduate work…)

In the process, I stumbled across a very entertaining article by Justin Kruger and David Dunning, titled “Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments” (1999, Journal of Personality and Social Psychology, Vol. 77, No. 6., p. 1121-1134). The article begins with the story of the hapless McArthur Wheeler, who in 1995 robbed two Pittsburg banks in broad daylight with no disguise. Arrested within an hour of the broadcast of the bank security footage, Wheeler expressed shock that he was identifiable on the security tape, protesting “But I wore the juice!” Apparently Wheeler believed that rubbing lemon juice on his face would render him invisible to security cameras… (Kruger and Dunning 1999: 1122).

While Wheeler most likely would meet anyone’s definition of “incompetent”, what Kruger and Dunning are primarily interested in are… er… relative incompetents – folks like you and me, who may be quite skilled in some areas, but are likely not so skilled in others. In those areas where we aren’t so competent, Kruger and Dunning ask, do we know that this is the case? Their hypothesis is that, below a certain level – they isolate out the bottom 25% in the specific skills (logical reasoning, grammar, humour) they test – we may be so poorly skilled that we actually don’t know enough to realise how far off the mark we are – we may, in fact, not know enough to recognise competent behaviour, so that we can begin to model it to improve our own performance.

The study is worth a read (any lurking methods students might particularly enjoy the discussion of how to test whether someone is in the bottom 25% in terms of their sense of humour…). I make no specific claim about the broader validity of the study’s findings, but some of the specific results have a certain intuive plausibility: The authors cite, for example, findings that study participants in the bottom 25% tend to overestimate their skill level considerably, and that they do not tend to revise this positive self-assessment, even after being provided with samples of higher-quality work to “grade”. Interestingly, the authors also mention that participants in the upper skill levels tend to systematically underestimate their compentence – until they are given an opportunity to view others’ work, which then allows them to revise their self-perceptions in a more accurate direction.

Both of these observations track reasonably well with my teaching experience. Struggling students often view suggestions for improvement as unfair and as impositions of impossible standards; they need assistance to get a very concrete sense that better work really is possible – and that it is realistic to expect them to produce such work. At the other end of the spectrum, extremely talented students are generally acutely aware of how much more is possible in ideal circumstances – and can come to measure their work against a standard of perfection that would make anyone depressed, causing their self-perception to become inappropriately low… This can actually be a bit more difficult to manage, since you wouldn’t want to lower someone’s sensitivity to how their work could be improved, but an unrealistically harsh judgment of one’s own work can also be counter-productive…

I’ve been experimenting recently with types of assessments that provide students with an opportunity to view and edit one another’s work, in part to address these sorts of issues (and also to decentre the teaching process a bit – particularly in advanced courses where it’s quite reasonable to expect at least some students to know more in many areas that I do). I’m not completely happy with these experiments to date, so I’m continuing to tweak, but I think there is value in providing students at various skill levels with an opportunity to see what kind of work is possible – and also what kind of work is common…

Betting on Wikipedia

Two of my first-year students approached me this week to play Solomon in a bet they had apparently made with one another, over whether Wikipedia were an acceptable resource for university students. The provocation for the bet was apparently an interaction with another tutor, who had instructed them strictly that Wikipedia was unreliable and therefore had no value for university-level work.

Since many students in fact do find Wikipedia valuable, this advice contradicted their practical experience, and was therefore disregarded: its impact was essentially a social one, causing the students to feel that it is vaguely disreputable to admit to their Wikipedia use in polite circles – or at least in front of teaching staff… The result is a strange social situation that I’ve also seen in some of my other classes, where students with an unusually high respect for teaching staff dutifully stop consulting Wikipedia, while others continue using it clandestinely (one imagines them looking over their shoulders in computer labs), while disguising their use by never citing it…

I’m obviously not present in other classes and tutorial sessions when this anti-Wikipedia advice is meted out, so I don’t know exactly how other staff try to justify that Wikipedia is somehow a pariah resource for anyone with academic pretensions. My sense from what I get second-hand is that the objection involves one of two issues:

(1) doubts about how reliable Wikipedia content could be, when it has not gone through peer review and {{shudder}} can in fact be edited by anyone; or

(2) a more general objection to encyclopedias of any kind, on the theory that the use of encyclopedias as source texts for student writing encourages students to think of knowledge as a static given that they must learn from other people, rather than as a dynamic construct they must actively participate in creating.

I’ll address each of these objections in a moment, but I wanted first to mention in passing that these two objections actually sit in tension with one another, and rely on conflicting notions of how students should position themselves in relation to knowledge construction: the first objection relies on the notion that there are certain sources of information that students can and should treat as authorities – as materials that have been appropriately vetted so that students can cite them as true – while the second relies on the notion that claims about knowledge are arguments, and that students should therefore assume an active, critical relationship to all of their sources. So, from one perspective, Wikipedia is “bad” because it’s not a good enough source while, from the other, Wikipedia is “bad” because students might be tempted to treat it as a source. Read more of this post

Overheard in a University Coffee Shop II: Ashes to Ashes

Student 1: “I really want to do a dissertation, but I have no idea what to do one on!”

Student 2: “Oh, you can do a dissertation on anything – just interview twelve people and do a qualitative something-or-other on their… I don’t know…”

Student 3: “Dust. Do a dissertation on dust.”

Student 1: “Dust?”

Student 3: “Yes: The History of Dust.”

Student 1: “The History of Dust?”

Student 3: “Oh, well, you know, it might need to be A History of Dust…”

What Happened Next

My high school “world history” teacher began her class with a line I’ve (unfortunately) never been able to forget:

The earth formed in a ball of gas and dust. It cooled. It rained. Humans evolved. This year, we’re going to study what happened next.

Without going into specifics, can I just say that somehow her spirit has found its way to Australia, and is now haunting the essays of many of my otherwise talented undergraduates…

Just a small plea, if anyone is reading: yes, I have suggested that you use your first couple of sentences to link your essay to a broader context, so I realise I’m the proximate cause of this particular problem. So let me explain what I mean. The strategic point of contextualising your paper is not to demonstrate that your argument is of world historical import. References to world history, human evolution, global thermonuclear holocaust, or other Powers and Principalities are unecessary and – trust me on this one – usually counter-productive.

The goal, instead, is much more modest: write for the sorts of people who will eventually read your professional or academic writing – busy people, who might have a general background in what you’re discussing, but who aren’t intimately familiar with what you are about to say. Use your first couple of sentences to ease your reader into your thought-space, and to prime them for what you are about to argue. And, most importantly, take pity on your instructor, and don’t give me traumatic flashbacks to my world history class…

On Branching Out

Kerim from Savage Minds has made an interesting post on the need to preserve a space within the academy for “large questions in their entirety”. Among other points, Kerim draws attention to the common advice given to graduate students to focus on narrow specialisations until they are well-established in their academic careers, at which point they can “branch out”. I’ve posted at greater length over at Savage Minds, but I thought I would reproduce part of the exchange here, since I’ve been meaning to mention some of these same issues on this blog. Within my initial response, I commented:

Weber’s view is fairly commonly expressed in advice to graduate students, in my experience – often accompanied with a kind of historical just-so story about how it used to be possible to theorise large issues, but the world has now sadly become too complex… I’ve always felt it contradictory, at best, for academics to say things like this, while continuing to assign classic theory to graduate students: if the world is now too complex for us to develop our own grand theories, surely the ones from the 19th century should now appear hopelessly naive…

I’ve also personally always questioned the notion that students should be apprenticed to narrow specialisation in the theory that, at some later point, they can “branch out”: surely it is more common for us to develop into deeper and more refined versions of what we actually practice, so that an exclusive focus on narrow specialisation will predictably generate experienced and practiced specialists – not big picture theorists… If the “big questions” aren’t somehow there from the beginning, I’m not sure how a lifetime of attention to small questions is supposed to generate them…

Kerim picks up on this, and comments critically on the tendency to teach graduates brief critiques of the principal “big picture” theoretical traditions, a position I second:

What I always think when I hear one of those one-line critiques is something along the lines of “false consciousness is also true”: the really interesting aspect of any critique is precisely that it casts into clear relief how very strange it is, that we have managed to make sense of the world through the theories we create, in spite of their (occasionally large) empirical weaknesses. Critique should be the beginning of the puzzle, not the end – the real trick is to uncover why the theories were plausible, not just to tick off why they are wrong.

Elevator Evaluations

At least once a term, I end up crushed in the back of a crowded elevator, while some of my students gossip about my class in a… somewhat more candid manner than they would if they knew I were there. The gossip doesn’t bother me and, by the time I realise what’s happening, there’s usually no graceful way for me to warn the students that they’re doing this in front of me. But there’s always an awkward situation when the elevator clears out, or everyone gets off on the same floor, and the students suddenly realise that I’ve been there all along. My current strategy for minimising their concern is to pull out a book, a piece of paper, or something else to stare at, and try to look very absorbed and absent-minded professorish. Now I’m waiting for the moment when I find myself at the back of an elevator, overhearing students talk about how I’m so out of it that I didn’t even notice when they were gossiping about my course in the elevator the other day…

Design a site like this with WordPress.com
Get started