Rough Theory

Someone sent me an email link to Richard Hamming’s (1986) “You and Your Research”, which I have read previously, but not for some time. The piece analyses why a few scientists manage to make significant contributions to their field, while the rest of us… not so much… ;-P

A few people have suggested to me recently that my trajectory seems to be pointing in the not-so-much direction, giving the receipt of this link at this moment a somewhat timely quality that might not have been intended… ;-P Perhaps re-reading will give me a sense of what I’m missing… ;-P

For those who haven’t read it, the piece begins with a dismissal of easy explanations for significant scientific discovery – luck, for example, or base intelligence. Hamming focuses instead on traits like courage, confidence, emotional commitment, drive, hard work, salesmanship, ability to work strategically within established institutional and cultural contexts, etc. – all of which one would expect to have a substantial impact.

He also, though, analyses some less expected factors, such as the interesting dynamics that follow on from early fame: the way in which recognition gained too quickly creates both external and internal pressures that undermine someone’s ability to work on the small and obscure problems that often lay the groundwork for grand concepts.

He talks about the role of frustration – of the absence of “ideal” circumstances – as a driver for creative thought:

…many scientists when they found they couldn’t do a problem finally began to study why not. They then turned it around the other way and said, “But of course, this is what it is” and got an important result. So ideal working conditions are very strange. The ones you want aren’t always the best ones for you.

And draws attention to the issue of how tolerance for ambiguity factors into research:

There’s another trait on the side which I want to talk about; that trait is ambiguity. It took me a while to discover its importance. Most people like to believe something is or is not true. Great scientists tolerate ambiguity very well. They believe the theory enough to go ahead; they doubt it enough to notice the errors and faults so they can step forward and create the new replacement theory. If you believe too much you’ll never notice the flaws; if you doubt too much you won’t get started. It requires a lovely balance. But most great scientists are well aware of why their theories are true and they are also well aware of some slight misfits which don’t quite fit and they don’t forget it. Darwin writes in his autobiography that he found it necessary to write down every piece of evidence which appeared to contradict his beliefs because otherwise they would disappear from his mind. When you find apparent flaws you’ve got to be sensitive and keep track of those things, and keep an eye out for how they can be explained or how the theory can be changed to fit them. Those are often the great contributions.

The point for which this talk is most famous, however, is one of these beautiful, obvious, simple points:

If you do not work on an important problem, it’s unlikely you’ll do important work. It’s perfectly obvious. Great scientists have thought through, in a careful way, a number of important problems in their field, and they keep an eye on wondering how to attack them. Let me warn you, “important problem” must be phrased carefully… It’s not the consequence that makes a problem important, it is that you have a reasonable attack. That is what makes a problem important. When I say that most scientists don’t work on important problems, I mean it in that sense. The average scientist, so far as I can make out, spends almost all his time working on problems which he believes will not be important and he also doesn’t believe that they will lead to important problems.

I spoke earlier about planting acorns so that oaks will grow. You can’t always know exactly where to be, but you can keep active in places where something might happen. And even if you believe that great science is a matter of luck, you can stand on a mountain top where lightning strikes; you don’t have to hide in the valley where you’re safe. But the average scientist does routine safe work almost all the time and so he (or she) doesn’t produce much. It’s that simple. If you want to do great work, you clearly must work on important problems, and you should have an idea.