A Random Walk

Humans are hard-wired to seek order, patterns and certainty. The opposite of all this is randomness, the idea that things happen unpredictably. Random occurrences shape our daily lives, from the mundane to the lucky to the tragic. We don’t think about this much, and when we do, the idea can be unsettling. Randomness suggests a lack of control and undermines our confidence in knowledge, preparedness and intuition.

Beyond its importance to philosophers and mathematicians, randomness has tremendous relevance for investors. If the idea that stock prices move randomly is true, this presents a serious challenge to predictive techniques practiced in every mode of active investing. What’s the value of brilliant analysis if the future path of a stock price or the price level of the market is no more predictable than a series of randomly accumulated numbers?

In this article we introduce readers to the “random walk” theory and its doubters. We make the connection between the random walk theory and the efficient market theory. We raise questions about the usefulness of market timing. And we explain why the random-walk theory is one of many compelling reasons for investors to adopt a buy-and-hold indexing approach to investing. Throughout this article I draw from the work of Eugene Fama, “the father of modern finance” and a Nobel laureate best known for his work on efficient markets and asset pricing.

The Analysts Who See It Coming (Or Don’t)

There’s always a story in hindsight. You know, how it all went down. In retrospect, swings in stock prices not only appear to reveal what happened, but why it happened. “We saw it coming” is how analysts explain their successful predictions. Their explanations reinforce the belief that the right patterns or proprietary information can be exploited to produce superior results. But observing patterns in retrospect is nothing like calling the shots accurately, time after time, before an event occurs. Analysts who make calls on where the market or stocks are headed generally rely on two investment management and research disciplines: technical analysis and fundamental analysis. These investment approaches rely on vast amounts of research, data, and historic patterns for predicting the future direction of stock prices.

The basic assumption of “technical” theory is that history tends to repeat itself, that is, past patterns of stock prices in space and time will tend to recur in the future. Technical theorists, also called chartists, believe that price charts and previous price returns can be used to predict future stock prices, thereby increasing returns.

The basic assumption of “fundamental” theory is that at any point in time an individual security has an intrinsic value that depends on its earnings potential. In turn, this earnings potential depends on a variety of fundamental factors such as quality of management, outlook for the industry, the economy and more. Through a careful study of these fundamental factors, the analyst should, in principle, be able to determine whether the actual price of a security is above or below its intrinsic value. If actual prices tend to move toward intrinsic values, then attempting to determine the intrinsic value of a security is equivalent to predicting its future price. This is the essence of the predictive procedure used in fundamental analysis.

Enter the Random-Walk Theorist

Random walk theorists start from the premise that the major security exchanges are good examples of “efficient” markets. An “efficient” market is defined as a market where there are large numbers of well informed and price-sensitive investors actively competing with each other, each trying to predict future security prices and to find and profit from market imperfections. In efficient security markets, competition leads to a dynamic situation in which actual security prices reflect events that have already occurred and a consensus on events the market expects to occur in the future. In other words, in an efficient market the actual price of a security is a good estimate of its intrinsic value at any given point in time. Learn more about the Efficient Market theory here.

In an uncertain world, the intrinsic value of a security can never be determined exactly. This means there is always room for disagreement among market competitors regarding the actual  intrinsic value of an individual security. To quote Gene Fama: “This disagreement gives rise to discrepancies between actual prices and intrinsic values. If the discrepancy between actual prices and intrinsic values is systematic rather than random, then this knowledge should help traders better predict the path of future prices. But when traders attempt to take advantage of this knowledge, their moves tend to cancel each other out, while the actions of the competing participants cause actual security prices to wander randomly about their intrinsic values."

New Information 

Of course, intrinsic values can themselves change as a result of new information. To quote Gene Fama: “The new information may involve such things as the success of a current research and development project, a change in management, a tariff imposed on the industry’s product by a foreign country, an increase in industrial production, or any other actual or anticipated change in a factor which is likely to affect the company’s prospects. In an efficient market, on average, competition will theoretically cause the full effects of new information on intrinsic value to be reflected ‘instantaneously’ in actual prices. In fact, however, because there is vagueness or uncertainty surrounding new information, ‘instantaneous adjustment’ really has two implications. First, actual prices will initially over-adjust to changes in intrinsic values as often as they will under-adjust. Second, the lag in the complete adjustment of actual prices to successive new intrinsic values will itself be an independent random variable, with the adjustment of actual prices sometimes preceding the occurrence of the event which is the basis of the change in intrinsic values and sometimes following." In sum, Fama concludes, “A market where successive price changes in individual securities are independent is, by definition, a random-walk market.”

Fama and others caution against relying on the random walk hypothesis as “an exact description of the behavior of stock market prices.” A good deal of research supports the idea that price changes are independent while still allowing for some amount of dependence. But the actual degree of dependence in a series of price changes “is not sufficient to make the expected profits of any more ‘sophisticated’ mechanical trading rule or chartist technique greater than the expected profits under a naive buy-and-hold policy."

Empirical Evidence of Independence

How has the random walk theory been tested, and what do the tests tell us? Over the years, a number of empirical tests have resulted in a consistent answer. None of the studies using standard statistical tools “have produced evidence of important dependence in a series of successive price changes,” Fama reports. And “If the statistical tests tend to support the assumption of independence, than the corollary is that there are probably no charting techniques, based solely on patterns in the past history of price changes, which would make the expected profits of the investor greater than they would be with a simple buy-and-hold policy.” In other words, testing has upheld the theory of random walks.

Implications for Other Theories 

Earlier we described how technical theories implicitly assume that there is dependence in a series of successive price changes. The theory of random walks suggests just the opposite, that successive price changes are independent. If pricing changes are independent, how can the past possibly be a good predictor of the future, as chartists suggest? Jim Davis of DFA put this argument to rest with an exhaustive study of mean reversion, demonstrating that there is scant evidence that market timing works, or that past patterns of market behavior are a reliable tool for maximizing gains and minimizing losses. If, as the empirical evidence seems to suggest, the random-walk theory is valid, then chartist theories are akin to astrology and of no real value to investors. Read our piece about Market Timing to learn more about the Davis study and why market timing is risky and counterproductive.

Assuming that the random-walk efficient market is real, this does not negate the idea that fundamental analysis is useless. Fama notes that analysts who can quickly and accurately identify “non-negligible discrepancies between actual prices and intrinsic values” can reap hefty returns. However, if there are many analysts who are fairly good at this sort of thing, they help narrow discrepancies between actual prices and intrinsic values and cause actual prices, on average, to adjust “instantaneously” to changes in intrinsic values. Although the returns to these sophisticated analysts may be quite high, they establish a market in which fundamental analysis is a fairly useless procedure both for the average analyst and for the average investor.

What about the analyst who has deeper insights or information not known to others? Again, this analyst may earn a substantial return. But can he or she do it consistently? And does this analyst’s stock pick outperform other securities with the same general risk profile? If this were the case, mutual funds created by the best and brightest in the industry would generally outperform a basket of randomly selected securities. But separate studies repeatedly suggest that, even if mutual fund fees are ignored, on average, actively managed mutual funds do no better than a randomly selected set of securities. If one takes into account the higher fees of active management, however, on average, the random investment policy outperforms the actively managed fund.

The Ongoing Battle for Investors’ Hearts and Minds

The efficient market theory, the random walk theory, and evidence that market timing doesn’t work all have been challenged and subjected to tests that refute the truth of their claims. It’s true that the market doesn’t always behave according to these or any other theories. No theory is airtight. Sometimes stocks go through periods of significant mispricing. As we wrote earlier in this piece, it’s only in retrospect that we can tell ourselves a story about what happened and why we should have seen it coming. The dot-com bust. The housing bubble and mortgage meltdown. The Coronavirus. We need explanations, order, patterns and predictability. Randomness is so difficult to grasp because it works against our pattern-finding instincts. It tells us that sometimes there is no pattern to be found. As a result, randomness is a fundamental limit to our intellect and our intuition; it says that there are processes that we can't fully predict.

At the end of the day, the chartist and the fundamentalist need to prove their cases. The evidence that their strategies can outperform a buy-and-hold index investor is nonexistent. What needs to be proven? Persistence of results. Over the past 100 years, the stock market has returned an average of 10 percent. An indexer would have captured all these gains with very low cost. Only if market timing and fundamental schemes produce better returns than the market can they be judged effective. And to date, not one has passed that test. To learn more about the underperformance of the active portfolio management process read our piece on Active vs Passive investing here.

As an investor you need to find a consistent way to win; a reliable, proven formula that moves you toward your goals. Most investors, both institutional and individual, will find that the best way to own common stocks is through a diversified mix of index funds balanced to their objectives and risk tolerance. Those following this path are sure to beat the net results delivered by the great majority of investment professionals. Index funds offer a brilliant solution based on the facts of the market. They offer higher returns, lower fees, lower operating costs, lower taxes, lower risk of errors or blunders, and lower anxiety about when, how and where to invest. And yet it is true that some active managers will do better some of the time. But if certain active managers have been doing significantly better over a long period of time, particularly after taxes, fees, expenses and errors, don't you suppose we'd all know who they are? Investors would be wise to devote more attention to understanding the real advantages that index funds have to offer. Stock prices may be random, rising and falling, but your strategy should be sure-footed and steady. Randomness happens. Make it work in your favor.

Learn More About Us

Cardiff Park Advisors is located in San Marcos, 25 miles north of San Diego. We work with clients throughout the United States. We welcome the opportunity to discuss your financial goals and how we can help you reach them. You may reach us by emailing our principal at jgorlow@cardiffpark.com or calling our office at 760-635-7526.