[Ed. 9 September: Now that events are unfolding a bit more slowly, and people have had a chance, for the most part, to learn about the basic facts, I’ve moved my on-the-fly updates to the bottom of the post, so that the original text is easier to find. I will try to update all the broken links next week.]
Original Post
A couple of people have sent me the link to thisdebacle of two researchers attempting to study what they call the “Cognitive Neuroscience of Fan Fiction” (further historical background here and here, collated links there, and information about the original research (which somehow doesn’t get aroundto mentioningthat the research is designed – not for academic publication – but for a popular book whose working title is Rule 34: What Netporn Teaches Us About the Brain) in the researchers’ background information).
Kruger and Dunning are interested in whether, below a certain level basic competence, it becomes very difficult for people to improve their skills – because they are, in fact, too incompetent to be able to tell the difference between competence and incompetence in the first place. They take as their point of departure the story of hapless bank robber McArthur Wheeler who – some of you will remember from my previous post on this article – robbed two banks in broad daylight without any disguise and, when arrested almost immediately based on the bank security footage, burst out: “But I wore the juice!” Mr. Wheeler was evidently under the impression that, by rubbing lemon juice on his face, he could conceal himself from security cameras (Kruger and Dunning 1999: 1122).
Assuming this mess is not some sort of elaborate research-themed performance art, or the result of a revenge-fuelled identity theft, researchers Ogi Ogas and partner Sai Chaitanya Gaddam are trying their best to demonstrate to the world that they are something like the academic research equivalent to Wheeler. They have blundered into an online community whose members write and read, among other things, erotically-themed fan fiction, and have presented community members with a poorly–designedquestionnaire (now taken down, but for a while being modified on the fly as people lined up with complaints about the research design – participants have posted screenshots and a text version of the survey after its initial modifications – note that a number of the final option responses and some other warnings and qualifications seem to have been added in response to criticisms of the survey in its original form – the modifications are often palpably different in style from the original text).
Among many other problems, the questionnaire asks respondents to provide sensitive information about sexual habits, desires and fantasies, in a setting where the questionnaire could be accessed by minors, without – as far as I can tell – having vetted the research design with their university’s IRB (the researchers are currentlybeinghoundedacrossseveralwebsites with demands to answer the question of whether they did, in fact, submit the project for ethics review – while answering other questions, they have steadfastly ignored this one: quick suggestion that, if the researchers don’t mean to imply the answer is ‘no’, then they should probably address this question very explicitly, very soon). [Side note: there’s a nice critical discussion of the limitations of IRB’s that’s been sparked by this whole mess: here.]
In the ongoing discussions now sprawled across a number of sites, the authors continue to dig this initial hole deeper by using terms regarded as offensive by members of the community (and, in one case, defendingthis because these are the terms that are standard in the sex industry – as Marx might say: !!!), by blithely demonstrating their own participation inwidely–criticisedassumptionsabout sexuality and presuppositions about gender, by demonstrating ignorance of basic facts about the community that could be gleaned from a quick skim of community sites, and by insisting on knocking back well-reasoned and absolutely on-target critiques by arguing that they are not doing “social research” and are not actually interested in the communityanyway, other than as an example of a much more generalphenomenon (these last, the researchers seem to believe, get them off the hook on ethical and basic research design requirements).
I’m not going to write my own critique of this mess: the community has already done this, eloquently, thoroughly – and, given the circumstances, with admirable patience. I am always warning my students when I teach research methods that something like this can happen – that this is why I’m so harsh on their research designs. Welcome to my new case study. I’m serious. I’m thinking of assigning parts of this trainwreck when I teach research methods next term.
I’m posting on this mainly because I’m wondering why the researchers have not apologised far more abjectly for having blundered into a community so ill-prepared – and possibly having ignored basic legal requirements and professional ethical standards governing their research. I am wondering if they are simply failing to register how devastating are the critiques being made of their work – perhaps because they are assuming these critiques have arisen defensively, due to strong affective attachments and loyalties within this particular community – or perhaps because they have “othered” this community so much that they aren’t sufficiently open to how badly they are being schooled here. Sai Gaddam’s university website suggests a potential vulnerability in this regard – let me quote from the source (apologies: I owe a poster in the original discussion a hat-tip for drawing attention to this, but unfortunately I’ve lost track of the comment – if you want to make yourself known, I’ll add a link):
My research interests have evolved over the years I have spent in the Ph.D program, but my derision for my subjects remains a constant. Well, not really, but this quote does make me smile.
The individual I chose as my principal subject for the experiments … was an old toothless man, with a thin face, whose features, without being absolutely ugly, approached ordinary triviality, and whose facial expression was in perfect agreement with his inoffensive character and his restricted intelligence.
The Mechanism of Human Facial Expression — Guillaume de Boulogne
So, for what it’s worth: I don’t belong to this community, but the criticisms being made of your ill-conceived research are excellent. Listen to them. You have tried wearing the juice. They’ve seen through it. It wasn’t the disguise you hoped it might be.
Updates:
[Ed. 7 September: Still no time to update the broken links below, but wanted to point to the discussion at metafilter, for those interested. ETA: and Neuroanthropology weighs in! – Twice!]
[Ed. 4 September: If people aren’t aware, Ben Goldacre from Bad Science has referenced SurveyFail on Twitter, linking here and also to Alison Macleod’s fantastic overview at The Human Element. Rushing at the moment – apologies for not responding yet to comments.]
[Ed. 4 September: Another day, a few more broken links. Ogi Ogas and Sai Gaddam seem to have had their websites removed from Boston University – not surprising, given the report that they are not affiliated with the university for purposes of this research project. Gaddam’s blog has also been made private. The links I have below off their names therefore no longer point anywhere. Again: my schedule’s too hectic to fix this right now, so just noting the problem. Some limited information about Ogas is included in his Wikipedia page, as a backup link… If the old Boston University pages end up being included in any of the screencapscollections currently being collated online, I’ll restore links to those once I have time.
For folks interested in legs, this post has been picked up at Josh Jasper’s blog at Publisher’s Weekly, as well as at Alison Macleod’s the human element. Macleod’s blog has a very clear overview of how the whole thing unfolded, as well, for folks new to this whole mess and trying to get a sense of what happened.
Broken link clean-ups still days in the future, I’m afraid…]
[Ed. 3 Sept: Folks, just a note that the researchers have taken down their site – after an amazingly offensive final blowup that, honestly, must be seen to be believed… This will break a lot of the links I’ve posted below. I’ll try to clean these up later, but for the time being, there’s are a number of good summaries of the whole incident – now christened SurveyFail – see especially Yonmei’s post at Feministsf.net, as well as a report of a response from the IRB at their university, which has disclaimed any affiliation with the project and asked the researchers not to use their uni emails or web addresses in conjunction with this activity. (My favorite part of the linked IRB discussion was the report that, when the IRB office was contacted directly: “Their exact words ‘I had a feeling it would be about that.'”) Links cleanup might have to wait a couple of days – schedule is awful at the moment…]
So it’s been a somewhat gruelling couple of weeks, getting everything together for the two methods courses I’m covering this term. The undergraduate course, since it’s new and we were designing it from scratch, took the lion’s share of the time – although it was nice to be able to work alongside the ultra-competent (and ubiquitously sardonic) L Magee in pulling everything together.
The course is large enough, and massive lecture halls scarce enough, that we have to deliver two different iterations of the lecture – so we had one go at the lecture Monday morning, and will get another try on Friday. Most of the first lecture, necessarily, dealt with housekeeping and course mechanics. I’m currently trying to gather my thoughts from the lecture on Monday – I always learn something about the tacit logic of my own stuff, when I present it – there were elements of the tacit logic underlying the structure of the lecture, elements of grasping why I wanted to organise my bits the way I did, that I sort of “got” only when listening to myself give the lecture… We’ll see if this improves the reiteration of the lecture to be delivered on Friday.
I thought I’d post a few notes here on the lecture and course concepts – with the caveat that I’m always a bit cringe-y when I expose pedagogical material publicly. There’s a strong exhortative dimension to teaching – things get simplified, not simply or straightforwardly to make them easier to learn, but with the goal of trying to rouse something, trying to pass along a certain contagion about why this stuff can actually be exciting (er… realising that what I’m about to write may not… er… have that effect on terribly many people – I don’t claim to be a rousing lecturer – quite the contrary (really strongly the contrary) – my skills lie much more in leading discussions – but there is still an element, in lecturing, of wanting to communicate affect, and not simply content, of wanting to share, somehow, that the very abstract sort of material that I generally teach, can be deeply meaningful in its strange way – and something about what I do, to try to communicate this, never seems to me to translate well when I write about what I did – instead, what comes through is the simplicity of the content, all the ways I would qualify it, all the ways I disagree with it… And yet… There’s a reason that stuff gets left out of the lecture in the first place… A reason that doesn’t prevent me from being fairly self-conscious about reproducing lecture concepts outside the shelter of the lecture hall…)
The course is titled “Social Research: Qualitative”, and the structure of classes here gives you twelve weeks to somehow meet whatever expectations such a title engenders. Last year, the staff member who took this course decided that twelve weeks simply wasn’t enough to give a meaningful introduction to something as broad as “social research” and so decided to drill closely down into one research method (discourse analysis), to try to give the students some in-depth experience with mastering a particular method, from which they could hopefully extrapolate when orienting themselves to other methods in the future. In earlier years, the course has been taught with a heavy NVivo focus, with all the students doing the same research project – again with the thought that students could extrapolate their experience with that project, into other sorts of research they might conduct in the future.
We’re equally dismayed, I suspect, by the jarring disjoint between the expansive course title, and what can reasonably be covered in twelve weeks in an introductory methods class for second-year undergraduates. Something – lots and lots of somethings – have to “go”, to make the course possible.
We’ve channelled our dismay, however, in a slightly different direction: while it is certainly true that any individual student is going to do some specific form of research project, and neglect others, we’ve decided not to pre-dictate either the method or the project itself. There will no doubt be plenty of project-deflection over the term, as students choose topics that are too vast – or illegal, dangerous, or inappropriate in other ways… ;-P But in principle we’ve left it to the students what they want to study, substantively, and how they intend to study it. Although we will do a bit with “method” in the “to do” list sense in this course, we’ve decided instead to focus on the most basic elements of the research design process – becoming curious about something, asking a question, looking around to see whether anyone else has ever asked something similar, trying to figure out what you need to do, to answer the question you’ve asked, and then being accountable for your question, what you’ve done to answer it, and the answer you’ve put forward, in a public sphere.
This approach means that we can’t dictate method, because we’re telling the students in a very strong way that their method has to derive in some quasi-logical way from their question. And we can’t dictate question because… well… we’re telling students that research is about straddling that strange space between personal curiosity and public accountability – and it’s a bit out of place to tell other people what they ought to be curious about… ;-P
So we bookended this first lecture with two videos, designed to mark out two possible extremes in conceptions of social research. After some brief transitional comments, we opened with the first six minutes of this video of the Milgram experiments:
What the students saw, was a man in a white lab coat take an authoritative role in a highly artificial experimental setting, where the stated purpose of the exercise was to test a hypothesis in carefully controlled circumstances. I did warn the students there was more to this experiment that met the eye (we’ll return to this video again later in the course) – but the parting image they were left with was of what looked to be a research subject with a heart condition, strapped to a chair, awaiting progressively nastier electric shocks if he failed in a memorisation task… (They laughed… Hmmm… I responded by telling them we would trial this method in their tutorials…)
So this is one extreme – not, in this case, for the distressing nature of the experiment, but for the highly artificial, controlled, hypothesis-testing orientation of the study. The video with which we ended the session was this one, on Sudhir Venkatesh’s anthropological work on a Chicago gang (the embedded video below is only an excerpt – the full program is here):
Venkatesh’s piece was chosen for a sort of maximal contrast to the fragment of the Milgram video that we showed: a research scenario in which the field strikes back, takes its researcher captive in the most literal possible sense, rejects the researcher’s “expert” knowledge, and tells the researcher how to conduct the (radically uncontrolled) study.
We will do other things with these and other video materials through the course but, for purposes of this introductory lecture, the point was simply to mark out two extreme points, suggest that there is a continuum of possibilities between them – and that all of this, the whole continuum, could be defended as some form of “social research”. A continuum of social research along which the students would have some opportunity to begin situating themselves in the course of the term.
In terms of other content, this blurb from the course guide gives the gist of how we are approaching the course:
Many people, when they think about research, think of something done in a special sort of place, like a laboratory, a library, or a “field site”, by a special sort of person, like an academic expert who has spent years acquiring a vast specialist knowledge of what they are studying, and on a special sort of topic, which is important enough to count as a “research question”. Thought of this way, research can seem a bit intimidating and removed from our other concerns: we can struggle to think of ourselves as being the kind of people who might do research – surely we aren’t qualified or we don’t “know enough”? We can struggle to imagine what research might look like, if carried out in the sorts of settings where we spend our personal and professional time – surely research doesn’t tackle the sorts of experiences we have in our everyday lives? We can doubt whether our questions and concerns are “important” enough to count as research questions – surely research investigates something more removed from our everyday experiences or personal passions?
While it is very common to think of research as this kind of specialised, rarefied expert activity, this image of research is highly misleading. Research, at its most basic, involves cultivating the very opposite of expertise: it entails a process of opening ourselves to what we don’t know – of taking seriously our own curiosity and desire to learn more – of asking questions. Because research crystallises around a question, the research process is driven precisely by our lack of expertise – by what we need to learn. In the research process, we all position ourselves as explorers and investigators, rather than as people who already possess some kind of mastery over our subject matter. Because research operates in this space of exploration and uncertainty – because it takes the form of quest to learn something new – it is impossible to have all the skills and knowledge you will need for the research process, before you undertake the research itself.
While some of us may have a bit more practical experience with research than others, all of us have some experience with the core skills required for the research process: we have all been curious, asked questions, set about finding answers, and debated with other people about each step in this process. On one level, then, we are all “researchers” in at least an informal sense. At the same time, no specific research project – formal or informal – begins with a special creature called a “researcher” who already possesses all the skills and knowledge required to carry out a research project, before they start asking questions and working out how to answer them. Researchers are created, not born. And what creates them is nothing more than the process of actually doing research. You become a researcher: you do this by carrying out research. All the skills that research requires, and all the things you need to know to do research successfully, are learned through the research process itself.
This doesn’t mean that formal study is not essential to the research process: it is. It means that this formal study more closely resembles the process of apprenticing in a craft, than it does the process of committing to memory some fixed body of information. Research is a practical activity – an art, albeit one undertaken with a scientific spirit. Every question, every method, every researcher brings something subtly different to the research process – meaning that research is never learned abstractly, as a skill that could be pursued separately from its various practical applications. Instead, your research question is what drives your formal study, providing a meaningful context within which you can work out what sorts of formal knowledge and skills you need to have, why you need to have them, and how you can learn them most efficiently. Your research question therefore grounds other sorts of study you undertake – which is why we will start this course, not with an abstract set of knowledge or skills we think you need to memorise, but with activities that will help you work out a research question that can organise the rest of your work in this course.
From this starting point, we will then guide you as you undertake a quick apprenticeship in the major stages of the research process. There are of course many different types of research. The research carried out by a journalist, an activist, a market researcher, a government, or an academic researcher will differ in significant ways, for example, due to the different end goals and audiences for the research. Nevertheless, certain elements are common to any sort of research process. Those common elements will provide the focus of our work in this course.
Off the wall question: I keep encountering a particular formulation in the work of quite good students who are trying to outline their methodology for research projects. It’s not unusual for someone to say something like: “My method is discourse analysis”.
Now, I react to this statement, sort of the way I would react to someone saying: “My method is statistics”. It gives me a rough ballpark sense of what sort of thing the student wants to do, but is nowhere near specific enough (to me) to indicate what the student actually plans to do (and the formulation itself also strikes me as awkward, as if it doesn’t quite express the habitus for how the term would be used in academic writing).
My question is: I see this so often, that students must somewhere be being taught that this is okay – that “discourse analysis” is a specific enough term that further clarification or specification of their method is not required. I’m curious whether I’m running into a strange displinary issue – whether this term actually does have a quite narrow and specific meaning, such that it would be intuitively clear to anyone who doesn’t scuttle around across disciplinary boundaries as often as I do? I always end up writing things like: “But what kind of discourse analysis? What do you plan to do?” – and the sheer repetition is starting to feel crotchety and pedantic, as if I’m asking students to explain the obvious. Am I?
Okay. Below the fold is the preliminary draft of the chapterised version of my post from the other day on indeterminacy as a form of determination (doesn’t that line make you wanna peek beneath the fold?). The last third of this chapter is still very undeveloped – basically, if you’re reading, once you get to the point where I start talking about the relationship of all of this to Hegel’s essence/appearance argument, the text from that point gets really sketchy and dubious. If it helps, I’m aware of this, but wanted to write at least a set of placeholder notes for things I want to discuss, when I’m able to revise that section properly. I may not be able to get back to this draft for some days, however, and so I thought I might as well toss it up in its current form. The main line of argument – which relates to how you can provide a socially-immanent explanation for certain categories that appear transhistorical in Marx’s work – is (I hope!!) sufficiently clearly developed for the moment. The points that remain undeveloped will always – in this chapter – be sort of foreshadows of material I can’t discuss in great detail until I’ve set up a few more layers of this argument.
Those who read the version of the previous chapter posted to the blog may notice that the transition at the end of the previous chapter draft doesn’t “work”. That’s because, partially in response to feedback received here, I significantly expanded the previous chapter – to the point that it got a little bit cancerous, and so I split it into two chapters, dividing off the programmatic bits, from the discussion of Marx’s relationship to Hegel, and adding more material to both of those discussions. So I suppose I can now say I’m working on the draft of the third chapter of my thesis. 🙂
Usual caveats apply to the content below the fold – with the additional caveat that, for some reason, I’ve found sleep almost impossible for the past several days, and so I’m really not in the position where I can “hear” this text. I think it’s still okay, but it may be much rougher than I realise. 🙂 Here goes…
Okay. Below the fold, one substantially – substantially – revised version of my previous attempt to develop a sort of programmatic chapter, outlining the broad brush-strokes of how I’m attempting to approach Capital in the thesis. This version sucks much less than the previous version – it’s decent enough that I would even post it to the main page, except that it’s simply too long (@12,000 words, for those tempted to peer below the fold). This time around, I managed not to forget my main argument while writing the piece. Hopefully this version comes a little closer to addressing some of the fantastic questions Alexei raised in relation to the previous iteration – it’s impossible for me to express how valuable such thoughtful, sympathetic critiques are in the formation of this project, particularly when, as Alexei did, someone takes the time to offer such criticisms with reference to an incredibly crude and… er… speaking frankly, deeply problematic version of the argument I was trying to make.
There are elements with which I’m still fairly uncomfortable. I’ve used, for example, a language of “embodied cognition” in some programmatic bits of the text. While this is a useful shorthand for some of what I’m trying to say about Marx’s argument, it’s also not completely accurate – at least, I don’t think it is… But for the moment, it’s somehow sneaked its way into the text, perhaps to be replaced by something more adequate later on.
There are also elements that are still, essentially, placeholders – the discussion of Phenomenology of Spirit, for example, may well be replaced by a discussion of the treatment of essence and appearance from the Logic – but I haven’t decided yet, and I’m ready to write on these issues in relation to Phenomenology, whereas I’m not quite ready to do this in relation to the Logic, so I’ve written the version that I can include now, in order to give readers at least some sense of what I want to argue in that portion of the chapter.
There is a lot of stylistic chaos in the piece – particularly in the final half, which I still find myself substantially revising each time I look at the text. A few parts have survived relatively unscathed since the previous version: the first two pages are similar, as is the summary of Hegel’s “With What Must the Science Begin?” Everything else is completely new, and therefore as raw, in its own way, as the draft I tossed onto the blog last time. I think this version has a clearer sense of what it’s trying to do, and I hope the internal structure is adequate to render the connections between the various sections clear, and that the piece provides sufficient background along the way that readers aren’t having to struggle to figure out what I am trying to argue. We’ll see…
The text loses something from having the footnotes excised: I write a lot of footnotes, often make substantive points in them, and engage with other literature primarily in this apparatus. It’s unfortunately clumsy to reproduce such things on the blog. As with the previous version, there are heavy debts here to Patrick Murray (for his work on Capital as a Hegelian “science”), Derek Sayer (for his work on Marx’s methodological eclecticism), and Moishe Postone (for his work on Capital as an immanent critical theory), as well as passing references to many others. I’m happy to clarify these sorts of debts in the comments, if anyone is curious.
I owe a very different sort of debt to certain people who have been putting up with my various thesis-related freakouts off the main page 🙂 Everyone who walks within range at the moment gets an earful of speculation about how Marx understands the relationship between essence and appearance. I suspect somehow that most folks don’t find this topic quite as enthralling as I seem to at the moment. I’m therefore particularly grateful to the ones who haven’t yet started running the other direction whenever they see an email from me 🙂 Such support is more deeply appreciated than you can know. You’re welcome to “out” yourselves here if you’d like, but otherwise I’ll keep under wraps that you get sneak peeks of ideas that are too ill-formed even to toss up on the blog. ;-P
Below the fold for the piece itself… Although I am still revising this piece, and working on the following chapter, there is a real sense in which working out what I’ve posted below really has been the “greatest difficulty” for me. I’m going to take a break from the blog and from all forms of writing for the rest of the day, but I will hopefully find time tomorrow at least to update the list of posting related to the Science of Logic reading group, which has seen a burst of inspired reflections over at Now-Times during the period when I’ve exiled myself from blogging to get this other writing done. Read more of this post
*sigh* This is awful. But I’m tired of looking at it, I need to move on now and write other things, and dumping it on the blog seems the best way to draw a bright, embarrassing line under it, and force myself to move on. Some version of this piece in the near future will be much better. It has to be. But that’s not going to happen this week. So below the fold this goes – a sort of framing mini-chapter, intended to do roughly the same work that the “Fragment on the Textual Strategy of Capital“ post did for the blog series on Capital, now that I’m finally ready (as I had mentioned wanting to do in the blog series) to outline this argument a bit more adequately, with reference to the work I’ve been doing on Hegel’s Science of Logic. My problem with this piece isn’t so much how it reworks these specific arguments – it’s more with everything else that somehow sneaked in along the way, with how many unintegrated layers this text seems to have acquired in its very brief life, and with the many sections where I know – please trust me, I know – I need to develop further what I have said, but where every time I add something, it just seems to make everything that much worse…
So below the fold it goes. Good riddance, for the moment at least…
I’ll try to write something substantive again later in the week – at the moment, I’m absolutely drowning in marking, which leaves me no time to have interesting thoughts, let alone pull them together into something others might want to read… For my own reference as much as anything else, I’ve tucked below the fold a sort of “Research Proposals for Dummies” piece I wrote this week for my quant methods students. It’s very, very, very simplistic – among other things, because it’s written for second-year undergrads, many of whom have no intention of going on to research careers – but some of my Research Strategies students also found the material helpful as a very basic breakdown and explanation of the strategic intent of the sections of a proposal. The piece might be useful for someone needing similar material for their own students, and not wanting to start utterly from scratch, but wanting to riff off of someone else’s basic structure.
Note that, because this piece was written in relation to a specific assessment, much of the material is obviously not relevant to a standard proposal (and I’m too lazy and too busy – hmm… can one be both? Evidently so… – to rewrite this as a more general piece right now). Note also that I wrote this at 3 a.m. – caveat emptor.
If anyone does convert this into something less assessment-specific – or improve it in all the various other ways it needs to improved – I’d consider it a great kindness if you’d share a copy of your revised version with me. Read more of this post
For those who have been curious about L Magee’s project, particularly if you’ve had a look and are still wondering what it is all about, I note that an introduction has been now posted over at schematique. Armed with the new information this introduction provides, I logged in to have a play, and am currently contemplating what to enter into my profile. Like the (sorely missed) “destroy” button, the profile screen offers all kinds of outlets for my anarchic impulses. There is a very large free response space, for example, where I can list my “ontology interests”. I’m wondering whether the appropriate answer for someone like me should be (with a nod to rob) “that there be none” or “prefer epistemology, myself”…
I also love the help information on this page: it’s not every day you see help for a profile that explains:
Only the username and password fields are obligatory. Other fields are used to add metadata to your ontology
But what if my ontology interest is “avoiding metadata”? What if I like my ontologies neat?
Also, although this seems somehow oddly appropriate, given my interest in self-reflexivity, should I have been able to do this: Read more of this post
orange. continues to hold this blog’s methodology slam title, offering a new interpretation of what’s really going on, for all who have been confused by L Magee’s ontology-matching experiment: orange. suggests that it’s later than we think. What LM has been calling a “pre-alpha” software development phase is no such thing: in reality, the software is fully developed, and the experiment’s on us!!! The cat’s out of the bag now, LM: admit it – you’re just studying what academics do when you throw them out of their comfort zone, and place them in a state of confusion: do we bluff, nod sagely, and pretend that we know what’s going on? Do we lash out and start tossing citations? How confusing does the environment actually have to be, before we fess up and admit we have no idea what’s going on?
Rough Theory 