Experts can’t predict yet investors must plan: What, then, to do?

In financial markets, predictions are ever-present and unavoidable. Yet investors worthy of the name don’t allow forecasts, including those of experts, to determine – or even influence – their decisions. The evidence has long been compelling but is almost universally ignored: authorities generally don’t (because they can’t) provide reliable guides to the future. Indeed, the more confident is the expert, the less accurate, on average, will be his prediction – and therefore the poorer will be any resultant decision. On the other hand, investment necessarily reflects assumptions about the future. More than mere guesses or wishful thinking must justify it. What, then, to do? 

The key is to bear in mind that investment is a process that generates decisions; accordingly, and as part of this procedure, the investor must understand that how she thinks about the future in general (rather than what she might think about possible variations of it) can provide justifiable means to navigate through the fog of risk and uncertainty.

Systematically Debunking Experts’ Predictions

Generally speaking, to what extent can experts predict the future? Anecdotally, it’s easy to cite numerous examples of prominent authorities whose forecasts have repeatedly failed miserably; conversely, it’s not hard to find a few that occasionally hit their targets. What does rigorous analysis tell us? In 1980, J. Scott Armstrong surveyed experts’ forecasts in a variety of fields, and concluded that “expertise and accuracy are unrelated.” And in 1984, Philip Tetlock designed and launched what he immodestly described as “the most comprehensive and rigorous assessment of expert judgement in the scientific literature.” He compiled a sample of almost 300 experts in economics, politics, science and technology, and then devised questionnaires which elicited testable predictions. Over the next quarter-century, he and his team collected and analysed more than 80,000 predictions. It was by far the biggest exercise of its kind ever – and its results were clear:

“Most experts,” Tetlock concluded, “would be far more accurate if they’d just toss a coin.” Most of the time, he added, “a dart-throwing chimpanzee” would produce more accurate forecasts. The blunt truth is that, generally speaking, predictions of experts are less reliable than random guesses; moreover, they’re no more accurate than non-experts’. 

Authorities presumably know plenty about their area of expertise; but nobody can reliably divine the future. As Warren Buffett astutely observed years ago, a forecast tells us much about the forecaster’s state of mind today – but very little about tomorrow’s state of reality. It’s amusingly ironic: intelligent non-specialists would never swallow an elixir sold by a huckster from the back of a wagon at the Royal Easter Show; yet they routinely accept the predictions of experts who allege, in effect, that they possess – and can accurately read – crystal balls!

Of course, astrologers and psychics can toss a coin as well as any expert; hence it’s tempting to infer from Tetlock’s research that, if one wants to predict (say) the price of Brent crude or the number of covid deaths, etc., exactly one year hence, it’s just as well to consult fortune cookies, palm readers or the loudmouth in the pub. But that’s not Tetlock’s conclusion. It’s certainly true that we should regard experts’ predictions very sceptically; but just as important as the dismal collective result is the wide variation of predictions among individual experts. “There’s quite a range,” he found. “Some experts are so out of touch with reality, they’re borderline delusional. Other experts are only slightly out of touch. And a few are surprisingly nuanced.”

Foxes and Hedgehogs

What distinguishes the impressive few (whom Tetlock dubs “superforecasters”) from the risible many? It isn’t whether they’re free-marketeers or socialists, or optimistic or pessimistic; it doesn’t matter whether they have PhDs (or if they do, from which university it comes); nor are experts in some fields (apart from physicists predicting eclipses, tides, etc.) better predictors than others; nor does long experience or access to classified information make any difference. If anything, high IQ and arcane statistical models beget worse predictions: “delusions can fool anyone,” says Tetlock, “even the best and the brightest – perhaps especially the best and the brightest.”

What does make a difference, Tetlock has discovered, is not WHAT but HOW people (expert or otherwise) think. The worst forecasters – those whose predictions would improve greatly if they’d simply toss a coin – disdain the world’s complexity and the future’s uncertainty. They seek to simplify prediction to some model, theory or theme – and they use it exclusively and repeatedly. To a small boy with a hammer, everything looks like a nail; similarly, the worst class of expert – which is by far the largest class – looks inflexibly at the future through the prism of this single idea, model or theory. The worst predictors are also the most confident. They’re certain that their One Big Idea is correct; hence their predictions, too, must be right. Except, of course, they’re not.

A poll conducted in June 2009 showed that business executives are, if anything, even more confident than other experts. Asked what the price of oil would be in five years, only 5% of the sample answered “I don’t know.” Asked to predict its price in 10 years, less than 10% pleaded ignorance; and when asked about their powers of divination 20 years hence, a mere one-third doubted their ability to predict! Hence the quip commonly attributed to one of Margaret Thatcher’s chancellors of the exchequer: the typical expert is “always wrong but never in doubt.”

In sharp contrast, experts whose predictions are somewhat better than random guesses think very differently. They acquire information from multiple (and conflicting) sources, seek to synthesise them – and accept that often they can’t. They’re self-critical rather than self-confident. When Tetlock’s researchers showed these experts that they’d erred, they – unlike the poorer predictors – didn’t try to avoid, deny or minimise their mistakes: they simply acknowledged them. Most importantly, they accept that world is forbiddingly complex and that the future is innately contingent upon innumerable actions and events. They’re at ease with uncertainty in the fundamental sense that they doubt the ability of anyone – including themselves! – to predict the future. Hence the paradox: the most accurate experts (perhaps “least inaccurate” is more apt) tend to be the least confident that they’re right.

In an essay entitled “The Hedgehog and the Fox,” the British political philosopher, Isaiah Berlin, recalled a fragment of an ancient Greek poem. “The fox knows many little things,” the warrior-poet, Archilochus, wrote, “but the hedgehog knows one big thing.” In Berlin’s honour, Tetlock dubs the less- inaccurate and more diffident experts “foxes” and the woefully inaccurate and laughably overconfident ones “hedgehogs.”

Tetlock’s results couldn’t be clearer: foxes’ predictions beat hedgehogs’. Cautious thinking that accepts complexity and uncertainty trounces the confidence that attempts to deny, ignore or greatly underestimate them.

The implication of Tetlock’s research is clear: if a hedgehog makes a prediction, it’s almost certainly wrong; you should therefore treat it very sceptically – and probably ignore it. The sort of expert who thrives in popular blogs, newspaper articles and columns, television panels, etc., is typically self-assured and dramatic. The mainstream and social media love him (men are typically brasher than women) because he delivers compelling stories and snappy sound bites – and doesn’t burden his audience with caveats, complications, doubts and uncertainties. Hence another amusing irony: the sort of “expert” who’s typically cited in and sought by mainstream and social media is precisely the sort who’s most likely, time after time, to be dead wrong.

Indeed, one of Tetlock’s most startling findings is that the more prominent is an expert, the more wildly inaccurate his predictions are likely to be – and, apart from his entertainment value as a buffoon, the less seriously you should regard him. The next time you encounter an expert self-assuredly predicting – which, I confidently envisage, won’t be long from now! – ask yourself: “why am I listening? Why am I not chuckling?”

But that’s not Tetlock’s most significant (for investors) result. Rather, it’s this: projections that emerge from simple, valid and reliable rules of thumb beat foxes as well as hedgehogs. One such principle is “project the long-term average;” another is “project the long-term rate of change.” Perhaps the best is “regress to the mean.” It depresses the priests on those rare occasions when they seriously analyse their own incantations – and amuses the laics who penetrate the priests’ mumbo-jumbo: experts’ “crystal balls,” that is, their arcane statistical models, generally produce significantly worse predictions than random guesses or tosses of a coin.

Intelligent people don’t take seriously the prognostications of astrologers, palm readers and the like; they should be equally sceptical of experts’ predictions. That presents a big problem. As investors, as in our careers, personal lives, etc., we must plan for the future. But if experts can’t foresee future developments in their chosen fields, then what hope have generalists? Are our plans pointless?

If we plan properly – that is, flexibly, humbly and using time-tested rules of thumb – then certain approaches to decision-making and particular styles of thinking can help us to navigate a path towards the future. Tomorrow is opaque, so some mistakes are inevitable; equally, logically valid and empirically reliable rules of thumb, applied rigorously, will help us to mitigate the number and severity of our errors. We must take care that we become neither Gnostics nor hedgehogs: no single idea, insight or model – and certainly no “secret” or special technique – can or will ever eliminate risk or uncertainty. Murkiness will always cloak the future. Only if we accept and embrace this fundamental and unalterable reality can we prepare ourselves for – and perhaps even profit from – the inevitable surprises that lie ahead. 

Conclusion

A panel of specialists, when it confidently attempts to divine the future, forms a parade of fools. “State a case to a ploughman and a professor,” said the author of the U.S. Declaration of Independence and its third president, Thomas Jefferson, in 1787; “the former will decide it as well, and often better than the latter, because he has not been led astray by artificial rules.” Experts’ predictions are typically less accurate than random guesses; but the experts, unlike tosses of a fair coin, regularly succumb to hubris. Hence it’s wise to discount their predictions heavily – better yet, apart from their value as slapstick entertainment, ignore them entirely. If their consequences weren’t so costly, confidently-uttered but wildly-inaccurate predictions would be amusing. Less comically, the passion to prophesy often reflects libido dominandi – the desire to dominate. Fortunately for the ruled, rulers’ (and wannabe rulers’) failure to forecast belies their clay feet.

Some experts (“foxes”), says Philip Tetlock, are somewhat less bad at prediction than others (“hedgehogs”). They can, in some instances over short periods, forecast better than the toss of a coin. At the same time, however – and when it really matters – the ability of these “superforecasters” has very severe limits: as a group, they failed to anticipate the bursting of the Dot Com bubble, eruption of the Global Financial Crisis, etc., even a few weeks (never mind months) before they occurred. Yet foresight – that is, prudent judgement amidst the thick fog uncertainty – is real. What’s more, it isn’t a mysterious gift bequeathed at birth: it’s the learnable result of particular ways of thinking, of gathering and analysing information, and of forming and updating conclusions. Although mainstream economics and finance is mostly claptrap, punditry is totally babble and many scientists egregiously overegg – and politicise – their findings, I don’t say that genuine expertise is bunkum. Clearly, however, the typical authority’s desire to grasp the future will always exceed his reach.

Fortunately, investors don’t need to predict accurately in order to decide sensibly. They can’t foresee the future; they can, however, acquire foresight (that is, prudence and wisdom). Our purpose isn’t to predict the future: it’s to prepare for a wide variety of conceivable futures. “The truth is,” a consultant told Dan Gardner, “in the oil industry today the most senior executives don’t even try to pretend they can predict the price of oil.” John Browne, BP’s former CEO, agrees: “I can forecast confidently that it will vary. After that, I can gossip with you. But that’s all it is because there are too many factors which go into the dynamics of the pricing of oil.” Rex Tillerson, once the CEO of ExxonMobil and later the U.S. Secretary of State, adds: “We’ve never been good at predicting these cycles;” hence “we don’t spend a lot of time even trying. How the future is going to look, we take no particular view on it, other than to recognise that whatever is today it will be different sometime in the future, and after that it will be different again.”

Here’s my confident prediction, which will certainly be 100% correct: during the next week, month, year, decade and beyond, the prices of stocks, bonds, commodities, etc., will fluctuate; further, at some point their gyrations will surprise, elate and terrify the crowd (and the experts who mostly follow the herd). Ignore their volatile emotions; instead, keep your head and prepare accordingly. At times like the present, when others are greedy (or at least not apprehensive), be cautious; and during the next bear market, bust, etc., whenever it comes (and when others – including experts, who yet again failed to see it coming! – become fearful or even panic), be bold. Until then, keep Tetlock and Gardner in mind:

There is no evidence … economic forecasters can predict anything … beyond the excruciatingly obvious … and the lucky hits are inevitable whenever lots of forecasters make lots of forecasts … Plan for the future … for adaptability and resilience. Then assume reality will give you a kick in the shin and think about dealing with that.

Not already a Livewire member? Sign up today to get free access to investment ideas and strategies from Australia’s leading investors.

Enjoy this wire? Hit the like button to let us know.

 Further Reading

1. J. Scott Armstrong, “The Seer-Sucker Theory: the Value of Experts in Forecasting,” Technology Review, volume 82, issue 7 (June/July 1980), pp 16-24
2. Dan Gardner, Future Babble: Why Expert Predictions Are Next to Worthless, and You Can Do Better (McClelland & Stewart, 2010)
3. Chris Leithner, The Intelligent Australian Investor: Timeless Principles and Fresh Applications (John Wiley & Sons, 2005), Chap. 7
4. William Sherden, The Fortune Sellers: The Big Business of Buying and Selling Predictions (John Wiley & Sons, 1999)
5. James Sirowiecki, The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations (Doubleday, 2004)
6. Philip Tetlock and Dan Gardner, Superforecasting: The Art and Science of Prediction (Broadway, 2016)