Why we mostly ignore market sentiment

Overview

“So far, so good!” Ken Fisher recently exulted (“To forecast financial markets, get sentimental,” The Australian, 16 April). “My 2024 outlook (issued in January) called for strong gains down under and worldwide – but with Aussie stocks lagging early before soaring late.” What underpins Fisher’s ebullience? “Sentiment. Stocks always move most on the gap between expectation and subsequent results, so gauging the former is crucial. There are hard ways and easy ways to do it ... Whether you choose easy or hard methods, tracking sentiment is key. Today it all signals more (market) gains ahead.”

Unlike Fisher, in this article I explicitly distinguish and analyse subjective (based upon investors’ unobservable, hard-to-measure but undoubtedly existing feelings) from objective (based upon directly-observable and easily quantifiable market prices) measures of investor sentiment. Further, and unlike his principal source of evidence, I analyse publically-available data. Finally, decades-long series substantiate my conclusions; Fisher’s, in sharp contrast, rely upon a handful of recent anecdotes.

I conclude that in three key senses Fisher is incorrect. First, he greatly overstates the importance of investor sentiment. In the short term, markets’ fluctuations are largely random; their systematic variation is thus a relatively small portion of their overall variation. Secondly, Fisher mistakes the nature of sentiment’s limited influence: it’s a “contrarian” correlate (but not cause) of tomorrow’s returns, such that higher sentiment today typically means lower returns tomorrow.

Finally, and bearing in mind markets’ largely but not completely random short-term fluctuations, both objective and subjective measures of sentiment presently signal the opposite of what Fisher expects: either mediocre gains or outright losses.

Except on the very infrequent occasions when it reaches extremes, Leithner & Company ignores investor sentiment: our actions stem from our own rigorous analyses rather than others’ transitory moods; moreover, our actions acknowledge that returns always transmit information – but recognise that often they don’t emit sensible signals.

As conservative-contrarians, and like Benjamin Graham, we derive no pleasure from acting with or against the crowd, and take no comfort because prominent, vocal or great numbers of people agree – or, more likely, disagree – with us. Adherence to passing fashion is no substitute for timeless principles, reasonable premises, valid reasoning and hard evidence.

What Is Investor Sentiment?

Investor sentiment, also known as market sentiment, refers to the mood of investors toward the stock market (as measured by an index such as the S&P 500) and its short-term prospects – that is, their belief that prices will rise or fall.

The many speculators who follow sentiment seldom express their key assumption explicitly: the more positive is today’s sentiment, the greater is the likelihood that stocks’ prices will rise tomorrow and in the near future. If so, speculators exert positive momentum. Conversely, the more negative is today’s sentiment, the more likely it is that stocks’ prices will fall. That’s negative momentum.

Several difficulties – and thus risks – accompany the incorporation of sentiment into investment decisions.

First, its measurement is hardly an exact science. To cite but one example: drawing representative samples of investors’ sentiments is always difficult and often prohibitively expensive. Second, what’s influencing what? Does today’s sentiment influence tomorrow’s returns? Or do yesterday’s returns influence today’s sentiment?

In other words, is the sentiment of today’s market participants merely a by-product of their recent returns, such that strong returns generate high sentiment and poor returns produce weak sentiment? For this and other reasons, it’s easy – as, I’ll show, Fisher does – to misstate and overestimate sentiment’s ability to anticipate changes of major market indexes.

Finally, and adding to the complications, assuming that you possess a valid and reliable measure of sentiment and have clarified what’s doing the influencing and what’s being influenced, what’s the direction of the relationship?

Specifically, is sentiment a “momentum” indicator, such that positive and rising sentiment today begets increasing returns tomorrow? Or is it a “contrarian” indicator, such that positive and rising sentiment presages decreasing and even negative returns, and negative sentiment signals stronger results?

Fisher’s Assessment of Investor Sentiment – and My Criticism

Ken Fisher asks: “how do you assess sentiment? One tough way my firm does you probably can’t replicate ... plotting professional sentiment bell curves.” Not unreasonably, he assumes that the sentiments of “Wall Street professional” investors at a given point in time follow, at least roughly, a “normal” (bell-shaped) distribution.

The distribution’s (bell’s) top quantifies its central tendency – that is, where most professionals’ sentiments cluster. Sentiments are distributed symmetrically around the mean (some professionals sentiments are above-average, others’ are below-average), creating downward-sloping curves on each side of the peak. The wider is the distribution of sentiments, the larger is its standard deviation; in particular, the distribution’s “tails” (the bell’s bottom) represent the relatively few professionals whose sentiments lie far above and far below the mean.

Fisher doesn’t say so explicitly, but (again not unreasonably) assumes that the distribution of the accuracy of pros’ sentiments – that is, their mood today vis-à-vis their subsequent returns – isn’t normally distributed; in particular, its tails are “fatter” than those of a normal distribution.

If so, then the actual likelihood of extreme events – subsequent results that are much better or much worse than today’s expectations – is greater than a normal distribution implies. In other words, the unexpected is more likely than the consensus anticipates. That’s a crucial insight.

Fisher elaborates: “an easier sentiment-tracking tool is to watch how economic (and financial) data compare to (consensus forecasts) ... Are subsequent results worse than expected? If so, sentiment is likely too optimistic. Are they beating (expectations)? Too dour. Low expectations mean weak results don’t doom stocks.”

Yet he omits to ask: do high expectations tend to beget disappointment – and consequent low returns? He also ignores what I recently detailed: a pillar of the mainstream’s obsession about “consensus forward earnings” – namely that markets boost the shares of companies whose earnings “beat the consensus estimate” – is generally false.

Specifically, investors don’t consistently punish companies whose earnings “miss” expectations. Moreover, companies which provide earnings guidance typically aren’t valued more highly than those which don’t; nor does their “forward guidance” tend to tamp the volatility of their shares’ prices (see Everything the mainstream says about earnings is wrong, 13 March 2024).

But let’s not let mere disconfirming evidence spoil a good story! In support of his claim, Fisher cites two examples (I’m tempted to say “anecdotes”):

1. In early-2018, professionals expected that during the year to come (he doesn’t say, but I assume American) stocks would rise 5.3% (excluding dividends). The lower tail of the distribution captured the actual result: a decrease of 6.2%.
2. In 2019, the median professional prophesied a rise of 15.8%, yet the actual result (28.9%) was in the upper tail.

At the beginning of this year, Fisher says, the median professional forecast that stocks would rise 1.8%. As he notes, that’s not very optimistic. Of the 54 professionals he tracked, the expectations of 40 clustered in the range 2.9-9.0%. Another nine envisaged market declines of 3.0% or worse, and one foresaw losses exceeding 17%.

Expecting the unexpected, Fisher therefore predicts a good 2024: “the relative void above 10% (at the beginning of this year) suggested above-average gains were quite possible, even probable. That is happening now.”

For two reasons, I doubt it. First, he looked at the wrong investors – or, at least, an overly restricted stratum of investors. Indeed, in this respect Fisher’s previous research – which to my knowledge he hasn’t recanted and others haven’t repudiated – undermines his current glib assertions! In “Investor Sentiment and Stock Returns,” Financial Analysts Journal, (vol. 56, no. 2, 2000), Fisher and Meir Statman “show that the sentiment of Wall Street strategists is unrelated to the sentiment of individual investors or that of newsletter writers, although the sentiment of the last two groups is closely related.”

Fisher’s current expectations of market gains rest exclusively upon professional investors. What about semi-professionals and non-professionals? Does he ignore their sentiments because he believes that professionals are superior forecasters? If so, he’s clearly mistaken (see, for example, Experts can’t predict yet investors must plan: What, then, to do? 23 November 2020).

Secondly, Fisher’s contention that professionals’ sentiment will extend the past year’s strong returns is, in light of his previous research, highly questionable. He and Statman “found a negative relationship between the sentiment of each of these three groups and future stock returns, and the relationship is statistically significant for Wall Street strategists and individual investors.”

In other words, if past is prologue and contra Fisher’s assertion, the recent sharp rise of bullish sentiment among non-professionals (see Figure 1, Figure 4 and Figure 6 below) will likely beget modestly positive and perhaps even negative returns tomorrow. In his and Statman’s words: “the sentiments of both small and large investors are reliable contrary indicators for future S&P 500 Index returns.”

Two Measures of Sentiment

In this article I’ll analyse two indicators of market sentiment. The first, the American Association of Individual Investor’s weekly survey of its members, is subjective: it gauges respondents’ opinions or feelings about the S&P 500’s prospects. The second, the CBOE Volatility Index (“VIX”), is objective: based upon today’s prices in options markets, it measures expected price fluctuations of S&P 500 Index.

AAII’s Investor Sentiment Survey

Each week since 24 July 1987 – that’s more than 1,900 weeks! – the American Association of Individual Investors (AAII) has asked its members: “Do you feel the direction of the stock market over the next six months will be up, no change or down?”

Various organisations and media outlets, including Barron’s and Bloomberg, follow and publicise its results. Birinyi Associates (“We are unique in that we do not analyze the economy, have little interest in corporate developments and fundamentals, and have little use for traditional, technical, quantitative or other market indicators. Our approach is to understand the psychology and history of the market, and most importantly the actions of investors”) has been following AAII’s survey for years,

According to AAII, the higher is the percentage of respondents that say the market will rise over the next six months, the more confident (“bullish”) is sentiment. If so, then high levels of confidence likely signal overconfidence.

Conversely, the larger is the percentage of respondents that say the market will fall over the next six months, the more fearful (“bearish”) is sentiment. Finally, the higher is the percentage of neutral responses, the more uncertain sentiment becomes.

The AAII’s average member is male, aged in his late 50s and holds a tertiary degree, and over half of its members own an investment portfolio of $500,000 or more. Its survey thus represents a small group of active and very well-to-do investors. Clearly, the views of respondents to its survey may well differ from those of investors as a whole.

The Wall Street Journal (“Investor Survey Says: Bet Oppositely,” 9 December 2010) goes much further: “what ... few on Wall Street know is that the (AAII’s) survey’s sample size is typically so small, and its methodology fraught with holes, as to render it statistically worthless.”

WSJ elaborates: “Just 200 to 300 investors respond each week ... From a strict, statistical perspective, the survey is ‘pretty much useless,’ said David Madigan, professor and head of the Department of Statistics at Columbia University, who is particularly troubled by survey’s reliance on voluntary self-reporting. ‘The thing you worry about is the bias of the people who volunteer ... But maybe the opinions of the 200 who are motivated enough to respond is predictive of what the markets are going to do.’”

Madigan’s criticisms are valid. Yet they apply just as much or even more to Fisher’s sample of ca. 60 Wall Street professionals, The Wall Street Journal’s long-running survey of ca. 80-100 economists, etc.

The AAII’s sample is small; accordingly, its margin of error is wide (aggregating weekly samples into monthly ones mitigates this defect). Moreover, the survey’s sample is likely unrepresentative of the general population of investors.

Why, then, consider the AAII’s survey? Although it doesn’t pass professional statistical muster, it’s nonetheless very useful. As WSJ acknowledged, “despite its formidable statistical limitations, ... over the past two decades it has proved a compelling contrarian indicator: if (its) reading is overly bearish, for instance, it is often a sign the market will rally.”

CBOE’s Volatility (“VIX”) Index

The Chicago Board Options Exchange (CBOE) is America’s and the world’s largest. VIX, which CBOE created and CBOE Global Markets maintains, measures the implied volatility, based upon the bid and ask quotes of a range of near-term call and put options traded on CBOE, of a hypothetical option on the S&P500 Index with 30 days to expiry. (Until September 2003, VIX’s formula was very different. According to Macroption.com, “obviously, the two methods produce different index values, although the differences are relatively small ...”)

Volatility, says Investopedia, “is often seen as a way to gauge market sentiment, and in particular the degree of fear among market participants.” Hence VIX is widely known as “the fear index.” The higher VIX rises, the greater, by implication, is investors’ fear. The lower it falls, in contrast, the lower is their fear and the greater is their confidence – and at extremely low levels, overconfidence and even greed. 

Investopedia concludes: VIX “is an important index in the world of trading and investment because it provides a quantifiable measure of market risk and investors’ sentiments.”

Results: AAII’s Investor Sentiment Survey

Figure 1 plots the percentage of “bullish” respondents to AAII’s survey; that is, those who believe that stocks will rise during the next six months. In order to tamp a considerable amount of random week-to-week fluctuation, I’ve expressed these percentages as 26-week moving averages (MAs). From 24 July 1987 to 21 January 1988, for example, an average of 39.4% of respondents reckoned that stocks would rise, and so on until the average of 44.3% from 12 October 2023 to 11 April 2024. The overall mean of these 26-week MAs is 37.5%.

Figure 1: Percentage of “Bullish” Responses, Six-Month Moving Average of Weekly Data, January 1988-April 2024

The MAs are only approximately normally distributed. If their distribution were perfectly normal, then 95% of the observations would lie within two standard deviations of their overall mean – and thus 5% of observations would lie more than 2 SDs from the mean. In reality, only at the height of the Dot Com Bubble at the turn of the century was the percentage of bulls exceeded two standard deviations above the overall mean.

Similarly, only rarely and very briefly has the MA fallen more than two standard deviations below its overall mean. Most recently, between December 2021 and July 2022 it repeatedly crossed this threshold. Finally, the data’s trend is parabolic: before the GFC, bullishness mostly waxed; since then, with some brief exceptions it’s mostly waned.

Indeed, the sharply rising bullishness over the past two years, from a very low to an above average level (19% in April 2022 to as high as 49% in December 2023, and 45% in April 2024), ranks among the strongest and most rapid on record.

Figure 2 plots the percentage of bearish respondents to AAII’s survey; that is, those who believe that stocks will fall during the next six months. This percentage has trended weakly upwards, and the 26-week MA has never been more than two standard deviations (2 × 6.5% = 13.0%) below its overall mean (31.0%). On four occasions, in contrast, the MA has risen more than two standard deviations above the overall mean.

Figure 2: Percentage of “Bearish” Responses, Six-Month Moving Average, January 1988-April 2024

The first commenced in November 1990 and concluded in March 1991, i.e., coincided with the recession of the early-1990s; the second occurred from January 2008 to August 2009, i.e., encompassed the Global Financial Crisis; the third occurred from July to October 2020, i.e., in the wake of (rather than during) the COVID-19 panic; and (perhaps in anticipation of a recession which, at least according to its semi-official arbiter, the National Bureau of Economic Research, hasn’t materialised) the fourth and most recent from June 2022 to March 2023. Since this latter month, bearishness has plunged.

Figure 3, which plots the “bullish” and “bearish” series as 52-week MAs, reveals four key points that Figures 1 and 2 obscured:

1. from July 1994 to February 2008, bulls outnumbered (by an average of 14.3 percentage points) bears;
2. from the nadir of the GFC to 2016, the percentages of both bulls and bears fell;
3. since 2016, these percentages have fluctuated sharply but erratically;
4. presently, investors are among the most bullish (12-month MA on 11 April was 44.4%) they’ve been since before the GFC.

Figure 3: Percentage of Bulls and Bears, 52-Week Moving Average, July 1988-April 2024

Figure 4 plots the “bull-bear spread,” that is, the percentage of bulls net of the percentage of bears, as a 52-week MA. A spread greater than 0% indicates that the percentage of bulls exceeds the percentage of bears, and a spread less than 0% indicates that bears outnumber bulls. The mean of the spread’s 12-month MA is 6.4%; since 1988, bulls have outnumbered bears by an average of 6.4 percentage points.

Figure 4: “Bull-Bear Spread,” 52-Week Moving Average, July 1988-April 2024

On just two occasions – the height of the Dot Com Bubble (September-December 2000) and before the GFC (February-September 2004) – did the spread rise two standard deviations above its mean (25.6%). Similarly, just twice (January-February 1991 and November 2008-July 2009) has it sunk two standard deviations below its mean (-12.8%). But only once – from September 2022 to June 2023 – has it breached this threshold.

Does the spread’s subsequent sharp rise (to 11% in April 2024) actually indicate a genuine increase of bullishness? Or does it reflect the rebound of a heavily mean-regressing series from its previous extreme – indeed, unprecedented – low?

Finally, for the sake of completeness Figure 5 plots the percentage of AAII’s respondents which believes that stocks will neither rise nor fall during the next six months. The mean of the 26-week MAs is 31.4% and their standard deviation is 8.2%. From 1988 until the GFC, the moving average fell by more than half to 20%; it then rose steadily and doubled to more than 40% in 2016; since then, it’s gradually sagged to 30%.

Figure 5: Percentage of “Neutral” Responses, 26-Week Moving Average, January 1988-April 2024

What Explains Short-Term Changes of Bullishness and Bearishness?

Do responses to AAII’s survey during a given week reflect the market’s direction and degree of volatility during the previous week? In order to answer this question, for each week since 24 July 1987, I computed the percentage change of the S&P 500’s closing level from the previous Friday (or the nearest business day in the case of holidays). I also computed

• the S&P’s volatility during the week (defined as its weekly maximum minus its minimum divided by the minimum);
• the weekly change (in percentage points) of the survey’s bullish percentage;
• the weekly change (in percentage points) of the survey’s bearish percentage;
• the change (in percentage points) of the survey’s “bull-bear spread” (that is, the percentage of bulls net of the percentage of bears vis-à-vis the previous week).

Finally, I sorted the data by the Index’s one-week percentage change, divided these sorted data into quintiles (five groups of equal size net of rounding) and computed relevant ranges and means for each quintile. Table 1 summarises the results.

Table 1: Effect of S&P 500 Index’s Change and Volatility upon Investors’ Bullishness and Bearishness, Weekly Data, July 1987-April 2024 

It demonstrates that the Index’s percentage change from the previous week, as well as its volatility during that week, systematically affect investors’ sentiments.

In quintile 1, the 20% of weeks in which the market falls as much as 15.7% and volatility averages 3.8%, bullishness decreases by 2.9 percentage points from one week to the next (for example, from its overall weekly mean of 37.6% to 34.7%), bearishness rises by 3.1 percentage points and the “bull-bear spread” decreases 5.9 percentage points.

Reading down the columns, as the S&P’s weekly return changes from hefty losses to strong gains, three things occur:

1. the Index’s volatility initially falls (from quintile 1 to 3) and then rises (quintiles 3 to 5),
2. bullishness rises (from -2.9 percentage points in the lowest quintile to 2.3 in the highest) and bearishness falls; and thus
3. net bullishness increases.

This effect also occurs over periods of six months (Table 2) and 12 months (details omitted for the sake of brevity).

Table 2: Effect of S&P 500 Index’s Change and Volatility upon Investors’ Bullishness and Bearishness, Weekly Data, July 1988-April 2024 

Why over the past 1-2 years have respondents to the AAII’s weekly surveys become more bullish and less bearish? Why has the bull-bear spread soared? It’s not least because over the past year the S&P 500 has zoomed more than 25% (versus its MA of 9.5% since July 1988).

What Do Short-Term Changes of Bullishness and Bearish Sentiment Imply?

Do today’s sentiments presage tomorrow’s returns? In particular, are today’s sentiments a “contrarian” or “momentum” indicator of tomorrow’s returns? In other words, does rising sentiment foretell decreasing returns, and does declining sentiment portend increasing returns? Those are the patterns that’d we’d observe if current sentiments are a “contrarian” indicator of subsequent returns.

To answer these questions, I conducted an exercise similar to the one that underpinned Tables 1 and 2. For each week since 15 January 1988 I calculated the S&P 500’s percentage change during the previous 26 weeks and the next 26 weeks. I then sorted the data by the current week’s bull-bear spread, divided the data into quintiles and computed summary statistics for each quintile. Table 3 summarises the results.

Table 3: the “Bull-Bear Spread” as a Contrarian Indicator, Weekly Data, January 1988-April 2024

Two are paramount. First, the greater is the bull-bear spread during the last week of a 26-week interval – that is, the larger is the percentage of bulls net of the percentage of bears – the greater, on average, has been the Index’s return during that interval.

Second, the greater is the spread the lower, generally speaking, will be the Index’s return during the next 26 weeks. Similar albeit weaker results obtain over 52-week periods (not shown).

What’s happening here? As we’ve seen, the more positive (negative) is the S&P 500’s return, whether over one-week or six-month intervals, the more bullish (bearish) investors become. Moreover, and as I’ve detailed elsewhere (see, for example, Stock tips are for patsies: Are you a patsy? 12 February 2024), over the short term (intervals of at least six months and perhaps up to 12-18 months) the levels of major Indexes like the S&P 500, S&P/ASX All Ordinaries, etc., as well as the prices of individual stocks, mostly fluctuate randomly.

The longer is the period analysed, however, the more returns tend (“regress”) towards their long-term means: over six-month intervals, they regress mildly; over periods of one year, the regression becomes stronger; and over periods of five years or more, returns become heavily mean-regressing.

What, then, does Table 3 tell us? An unusually low (below-average) or high (above-average) return at one point in time subsequently regresses towards the mean return: today’s stellar return will wane and become tomorrow’s more average result – and, at the same time, bullishness will abate and bearishness rise. Similarly, today’s abysmal return will become tomorrow’s more average result – and at the same time, bearishness will wane and bullishness wax.

Investor sentiment is thus correlated with but doesn’t cause tomorrow’s returns. Yesterday’s returns affect today’s sentiment, and tomorrow’s regress to their long-term mean. Hence the higher returns (and sentiment) rise today, the lower returns (and thus sentiment) will tend to fall tomorrow.

Investors’ sentiments per se don’t affect investment results. That’s simple common sense: if sentiment influenced results, then great returns would be a mere matter of extremely positive thinking! Yet sentiment’s role isn’t completely absent: it intervenes between results in a past period and results in the next period. Past results, in other words, influence subsequent results; but that influence depends to a degree upon investor sentiment.

From the AAII’s survey data for each week I calculated the S&P 500’s 52-week returns, the bull-bear spread and the Index’s returns’ four and 52 weeks hence. I then sorted the data by 52-week returns, divided the data into quintiles, sorted the data in each quintile by the bull-bear spread and divided each quintile into two groups of observations: those above and those below the bull-bear spread’s median. For each quintile and sub-quintile I then computed relevant summary statistics. Table 4 describes the results. (Roughly similar results also occur over periods of one and six months; for the sake of brevity I’ve omitted them.)

Table 4: the “Bull-Bear Spread” Intervenes Between the S&P’s Past and Subsequent Returns, January 1988-April 2024

The primary mechanism is the regression of returns to their means. Regardless of past returns but very much secondarily in importance, bullish sentiment tamps subsequent returns and bearish sentiment improves it.

In quintiles 1-4, returns in observations whose sentiment is above the median are lower than those in observations with below-median sentiment (in quintile 5 the returns are identical). Table 4 thereby corroborates a key result: today’s investor sentiment exerts a weakly “contrarian” – and not a “momentum” – influence upon tomorrow’s return.

This result implies a fundamental one: on both the upside and downside, market participants generally overreact. When returns lift and the bull-bear spread rises above the median, this overconfidence crimps ensuing returns (see also Why you’re probably overconfident – and what you can do about it, 14 February 2022). And when returns sag and the spread falls below the median, their overreaction boosts subsequent returns.

In other words, investors’ and speculators’ expectations of the future aren’t merely incorrect: they tend to be systematically mistaken (see also Everything the mainstream says about earnings is wrong, 13 March 2024 and How experts’ earnings forecasts harm investors, 11 July 2022).

Results: CBOE’s Volatility (“VIX”) Index

The week-to-week and month-to-month fluctuations of VIX, which I’ve omitted for reasons of space, are erratic (standard deviations of 10.2% and 4.1% respectively) but largely random (means of 0.0%): the change during one interval bears little relation to the change during the next.

As an indicator of sentiment, on weekly and monthly bases VIX is more noise than signal. To the extent that it measures investor sentiment, on a very short- term basis this sentiment is to a considerable degree random.

Over longer intervals, however, the ratio of signal to noise rises. Figure 6 plots VIX’s 26-week MAs since July 1987 (the series commenced in 1993; I’ve reconstructed a proxy for 1987-1993). Apart from upward spikes during the Dot Com Bubble and aftermath, GFC and COVID-19 panic, it’s approximately stationary (that is, its mean and standard deviation don’t vary greatly over time).

Figure 6: CBOE Volatility Index, Monthly Data, Six-Month Moving Average, July 1987-April 2024

But it’s only roughly normally distributed: although it’s breached the upper bound of its 95% confidence interval on five occasions (the spike in August 2011 coincided with S&P’s downgrade of the U.S. Government debt rating from AAA to AA+), it’s never come close to its lower found (4.6). Its current level (18.0 on 18 April) is slightly below its long-term mean (19.5).

What Explains Short-Term Changes of VIX?

Considered on its own, VIX is more noise than signal; but on a bivariate (two-variable) basis, a systematic relationship appears. I conducted an analysis identical to the one that Table 1 and Table 2 described; Table 5 summarises its results.

It corroborates a key result from the AAII’s survey: the S&P 500 Index’s percentage change from the previous week, as well as its volatility during that week, systematically affect VIX.

Table 5: Effect of S&P 500 Index’s Change and Volatility upon Investors’ Bullishness and Bearishness, Weekly Data, July 1987-April 2024 

A sharp decrease of the Index (of -1.4% or more, as in quintile 1) from one week to the next begets relatively high (3.8%) volatility and an average increase of VIX of 0.43 points (for example, to 19.93 from its average of 19.5). Reading down the columns, as the Index’s return rises its volatility at first sags (quintiles 2-3) and then lifts (quintiles 4-5) – and VIX, on average, falls by ever-greater amounts.

As the Index’s weekly return rises, in other words, weekly sentiment (as measured by VIX) lifts. The same relationship, albeit stronger, appears over longer (4-week, 26-week and 52-week) intervals, which I’ve omitted.

What Do Short-Term Changes of VIX Portend?

Is VIX a “contrarian” or “momentum” indicator? Over one-month (Table 6) and six-month (Table 7) intervals, it’s clearly the latter.

Table 6: VIX as a Contrarian Indicator, Monthly Data, July 1987-April 2024 

If VIX falls sharply during a given interval (quintile 1), the S&P tends to rise relatively strongly during that interval – and weakly during the subsequent one. Conversely, if it rises sharply higher during an interval (quintile 5), the Index tends to fall during that interval – but much more weakly during the subsequent one.

The larger the Index’s return (whether positive or negative) during one interval, the greater is the effect upon sentiment during that interval – and the smaller is the Index’s return during the next.

Over longer intervals, VIX’s ratio of signal to noise rises: in particular, extreme changes of VIX’s value during a given six-month period tend to beget an opposite change during the next six months. If VIX falls sharply during a given six-month period, for example (quintile 1 in Table 7), it subsequently rises smartly; and if it soars in one period, it collapses during the next (quintile 5).

Table 7: VIX as a Contrarian Indicator, Monthly Data, July 1987-April 2024

Moreover, whether from month to month or from one 26-week interval to the next, VIX’s percentage change as well as its volatility are systematically correlated with – but, it’s vital to emphasise, don’t cause – the S&P 500’s returns.

Conclusions and Implications

Changes of investor sentiment, whether measured by the AAII’s weekly surveys or the CBOE’s Volatility Index, don’t cause markets subsequently to rise and fall. Markets’ fluctuations do, however, cause sentiment to ebb and flow. Two relationships are crucial. First, previous returns influence today’s sentiment; specifically, higher returns tend to boost sentiment and lower returns tend to lower it. 

Second, tomorrow’s returns regress to their mean: they do so relatively weakly in the short term (periods of up to 18 months) but much more strongly over longer periods: hence yesterday’s stellar returns (and consequent higher sentiment) presage tomorrow’s lower returns (and lower sentiment). Conversely, yesterday’s poor returns (and resulting lower sentiment) presage tomorrow’s higher returns (and higher sentiment).

Mostly as a consequence of correlation rather than of cause, therefore, is today’s sentiment a contrarian indicator of tomorrow’s returns. If so, then the much-improved sentiment of the past year or so, which are by-products of the S&P 500’s strong return, signal lower (but not necessarily negative) returns over the next 6-12 months.

What causes markets to fluctuate? Their short-term ups and downs are to a considerable extent random. Consequently, over short intervals market sentiment also fluctuates largely randomly.

“Stocks,” asserted Ken Fisher on 16 April, “always move most on the gap between expectation and subsequent results, so gauging the former is crucial.” That assertion overstates and misconceives the nature and direction of the relationship between current sentiment and subsequent returns: the nature is epiphenomenal and the direction is contrarian.

Why We Ignore Mr Market’s Moods

My results reconfirm, as have those of many others before me, Warren Buffett’s wisdom. “Charlie (Munger) and I never have an opinion on the market,” he stated at Berkshire Hathaway’s AGM in 1994, “because it wouldn’t be any good and it might interfere with the opinions we have that are good.” “I have nothing to add,” added Munger.

“The market is there only as a reference point to see if anybody is offering to do anything foolish,” he elaborated to Forbes magazine in 1988. “If we find a company we like, the level of the market will not really impact our decision.”

“Don’t try to figure out what the market is doing,” Buffett advised Forbes (18 October 1993). ”Figure out a business you understand ...” “For some reason,” The New York Times Magazine (1 April 1990), quoted him, “people take their cues from price action rather than from values. What doesn’t work is when you start doing things that you don’t understand or because they (allegedly) worked last week for somebody else. The dumbest reason in the world to buy a stock is because (its price is) going up.”

Buffett’s earliest – and arguably wisest – utterance on this subject appeared in Forbes on 6 August 1979: “the future I never clear,” he said; “you pay a very high price in the stock market for a (bullish) consensus. Uncertainty is actually the friend of the buyer of long-term values.”

Journalists, Experts and Fallacy of Post Hoc Ergo Propter Hoc

My results also highlight the vacuity of journalists’ and alleged experts’ standard blather. “Optimism among global fund managers has soared in the last month to its highest level in years,” reported The Australian (“Why we should be worried about a bull market,” 18 April). “It’s the highest point for bullish sentiment since January 2022, two months before the Fed began its sharpest interest rate increases in decades.”

“Bullish sentiment has been driving share markets over the past five months,” it alleged, with Wall Street touching record highs and investors piling into big-name tech stocks ... ‘Bullish sentiment is not quite at “close your eyes and sell” levels, (Bank of America’s chief investment strategist reckons), but risk assets are tactically much more vulnerable to bad news than good.’”

“Can anything dent the optimism of stock pickers?” asked The Australian Financial Review (“Fundies have gone ‘full bull.’ Is it time to sell?” 19 April). “Sentiment is now so strong,” it adds, “that we are getting perilously close to a reliable contrarian signal ...”

It’s true that when the percentage of bulls rises appreciably the S&P 500 Index subsequently tends to fall; it’s also true that when the percentage of bears sharply rises the Index subsequently rises. But investors’ sentiments don’t cause these changes: the Index’s fluctuation affects current sentiment, and its ensuing regression simultaneously influences subsequent sentiment.

Investor sentiment partly reflects recent market returns, but it doesn’t cause subsequent returns. If you believe otherwise, you’ve succumbed to the fallacy of post hoc ergo propter hoc (which is Latin for “after this, therefore because of this”). This phrase expresses the logical fallacy of assuming that X causes Y merely because X precedes Y.

The classic example, as Jeffrey Tucker recently wrote, concerns a rooster and a sunrise. Suppose that, every morning before dawn, you observe a rooster crowing. Shortly thereafter, light appears on the horizon. If you know nothing else, and you watch much the same event occur day after day, you might reasonably conclude that the rooster causes the sun to rise. That’s a testable proposition. You could kill the rooster and see what happens. You’d find that the sun continues to rise.

But, you reflect, the fact that this one rooster is dead hardly means that all roosters have perished. Perhaps, you infer, the collective influence of all roosters’ crows causes the sun to rise. If so, then your experiment hasn’t disproved your initial theory; you therefore conclude that its gist remains intact.

Similar – that is, fallacious – thinking underpins the assertion that the collective mood of today’s bullish speculators presages tomorrow’s market returns. If speculators believe, some seem to think, they can achieve: the more bullish they become, the higher their returns will rise!

A surprising – and amusing but concerning – number of people, including professionals like Ken Fisher who really ought to know better (perhaps they do, but prefer to tell people what they want to hear), believe, in effect, that roosters (investor sentiment) cause the sun to rise (support and beget higher returns on major indexes). That’s why, except when it reaches extremes, Leithner & Co. ignores – and we never incorporate into our investment operations – measures of market sentiment.

Click LIKE so that Livewire knows that you want more of this type of content. Click FOLLOW on my profile for notification when my next wire appears.