I’ve mentioned this before, but there is this great Tim Harford lecture at the LSE that is worth a listen if you’re into podcasts. Here’s a link: How To See Into The Future – By Sound. There’s also an (excellent) article in the FT if you’re a fan of the written medium: How To See Into The Future – By Sight.
The Main Points
- Philip Tetlock is the author of “Expert Political Judgement”.
- That book was the result of an 18 year long experiment, where he collected almost 28,000 predictions, made by 284 commentators, across multiple fields of expertise, and then observed the accuracy of their outcomes.
- Mostly, he was motivated by his irritation with political commentators making predictions about rise of Gorbachev and the future of the Soviet Union in the 1980s (the right-wingers thought that what Reagan was doing was deeply dangerous, and idiotic; the left wing pundits were convinced that the Soviet Union was with us to stay).
- In particular, he was annoyed by commentators saying this sort of thing after being proved wrong:
- “It almost went my way”
- “It was the right mistake to make under the circumstances”
- “I’ll be proved right – mark my words”
- “The main thing to focus on is the moral evil of what is going on here.”
- His main findings:
- There is almost no difference between chimpanzees flipping a coin and human experts making a prediction.
- And crude extrapolation models were significantly better than human experts. The quote: “whereas the best human forecasters were hard-pressed to predict more than 20 percent of the total variability in outcomes…, the generalized autoregressive distributed lag models explained on average 47 percent of the variance.”
- He then tested for whether some human forecasters are better than others:
- Your background and accomplishments are irrelevant. Unless you’re a bit famous – in which case, you’re likely to make more wrong predictions (awkward).
- Belief systems made little difference – libertarians were wrong as often as Marxists, capitalists as often as communists, etc.
- But your reasoning style did make a difference. And Philip Tetlock used Isaiah Berlin’s “The Fox and the Hedgehog” analogy to explain it.
Hedgehogs and Foxes
The world has two types of expert:
- Foxes – who are pluralists and tend to cherrypick their ideas from multiple places.
- Hedgehogs – are puritans, committed to a particular world view. The die-hard Keynesians, and die-hard Libertarians, and the die-hard Socialists, etc. Like Jesse Colombo from Monday’s post…
A summary of their reasoning styles:
Here’s the version according to Nate Silver (of The Signal and The Noise fame):
And the basic summary is this:
- Foxes are right more often than Hedgehogs, especially in short-term forecasts.
- But Hedgehogs get more of the far out predictions right (albeit at the expense of getting so much else wrong in the interim).
So in general, foxes make better calls.
But if you want to make the best calls, be a crude computer model. Because regardless of whether you’re a Hedgehog or a Fox, the crude computer models are better forecasters.
However!
Not all hope is lost. Because the world also has super-forecasters…
#ToBeContinued.
For more on this topic, other articles you should check out:
- Tetlock: Why Foxes Are Better Forecasters Than Hedgehogs
- Quiz: Fox or Hedgehog (ignore the quiz though – yoh, it’s useless)
- Foxes vs Hedgehogs: Predictive Success
Rolling Alpha posts opinions on finance, economics, and the corporate life in general. Follow me on Twitter @RollingAlpha, and on Facebook at www.facebook.com/rollingalpha.
