Understanding the market: why passive investing is probably the best choice for the average investor

by

Introduction

It seems reasonable to assume that most retail investors share a common objective: to maximize long-term wealth while taking on as little uncompensated risk as possible. The prevailing consensus claims that for the vast majority of individuals, the optimal strategy is to buy and hold low-cost, broad-market Exchange Traded Funds (ETFs).

Until recently, I have accepted this doctrine without investigating the claim much further. To test whether that trust is justified, this post aims to start from first principles and go over the typical chain of arguments to evaluate if, and under what conditions, ETF investing is truly the most sensible path.

Finally, as a disclaimer, I am not a financial professional, and my knowledge of economics and finance is obtained through self-study. All claims in the post are backed by reliable and standard references that the readers can check out for themselves. Nothing in this post should be taken as financial advice.

The CAPM

The most natural starting point seems to be the famous capital asset pricing model (CAPM). CAPM is useful for deepening our understanding of the market, and the goal of this post would not be achieved without discussing it. In the following, starting from the efficient market hypothesis (EMH), the theory leading up to the CAPM is presented briefly and the conclusion for investors is stated.

The efficient market hypothesis

Usually, like in the introduction page of Wikipedia, EMH is introduced as the hypothesis that “asset prices reflect all available information”. To make this precise, we’ll borrow the notation presented in the Nobel prize lecture Two Pillars of Asset Pricing by Eugene Fama (2013) 1.

Let \(P_{t+1}\) be the vector of payoffs, that is, prices plus dividends plus interest payments, at discrete time \(t+1\). Let \(\Theta_{tm}\) be the set of available information on the market at time \(t\) used to set the prices \(P_t\) at time \(t\). Let \(\Theta_t\) be the set of all information available at time \(t\). The EMH then says that the implied payoffs at time \(t+1\) given the information used in the market pricing at time \(t\) (\(\Theta_{tm}\)) is equivalent to the implied payoffs given all available information at time \(t\) (\(\Theta_t\)), that is, \[f(P_{t+1}\mid\Theta_{tm})=f(P_{t+1}\mid\Theta_t),\] where \(f\) is the conditional distribution of payoffs given the information set. Often this is expressed in terms of expected returns: \[E(R_{t+1}\mid\Theta_{tm})=E(R_{t+1}\mid\Theta_t).\] Notice that we use \(P_{t+1}\) and not \(P_t\). This might seem counterintuitive at first glance if we take “prices reflect all information” literally. However, at time \(t\), the current price \(P_t\) is already fixed and observed. EMH is a statement about expected future payoffs given all available information and not about the current prices: it asserts that current prices are set such that no available information today can give an investor a superior forecast of tomorrow’s payoffs (\(P_{t+1}\)).

The EMH is a hypothesis regarding the expected returns, but it does not yet tell us what those expected returns should be. As noted by Fama in his lecture, for this we need an asset pricing model.

Modern portfolio theory and Tobin’s separation theorem

Before discussing asset pricing models, we should understand the mean-variance portfolio model developed by Harry Markowitz (1952) 2. In essence, the idea is that the investor chooses the required rate of return of the portfolio in such a way that minimizes risk (variance). The set of optimal minimum-risk portfolios for each given level of required returns defines a so-called frontier.

Let us make this idea more precise. The expected return of the portfolio \(g\) with \(N\) choices for assets is \[E(R_g)=\sum_{i=1}^Nw_iE(R_i),\] where \(w_i\) (which can be negative, i.e., shorting is allowed) is the weight of asset \(i\) in the portfolio and \(E(R_i)\) is the expected return. The weights must satisfy the relation \[\sum_{i=1}^Nw_i=1.\] The risk of the portfolio is characterized by the variance of the portfolio returns, which is given by \[\sigma^2(R_g)=\sum_{i=1}^N\sum_{j=1}^Nw_iw_j\,\mathrm{cov}(R_i,R_j).\] The frontier of portfolios is then obtained by solving the following optimization problem for each level \(\mu_0\) of required expected return: \[\begin{align*} \min_{w_1,\dots,w_N}&\sum_{i=1}^N\sum_{j=1}^Nw_iw_j\,\mathrm{cov}(R_i,R_j)\\ \text{subject to }&\sum_{i=1}^Nw_iE(R_i)=\mu_0,\\ &\sum_{i=1}^Nw_i=1. \end{align*}\] Now, what happens if we introduce a completely riskless asset, that is, an asset with some risk-free rate of return \(R_f\) and zero variance? The investor must then make a choice for how much of their portfolio they wish to allocate toward the risky portfolio \(g\) and the risk-free portfolio consisting of the asset providing the risk-free rate. This leads us to a portfolio \(p\) with the following characteristics 3: \[\begin{align} E(R_p)&=xR_f+(1-x)E(R_g),\\ \sigma(R_p)&=(1-x)\sigma(R_g), \end{align}\] where \(x\in[0,1]\) is the proportion of the portfolio allocated toward the risk-free asset. Let’s solve for the proportion \(1-x\) of our risky portfolio from the second equation: \[1-x=\frac{\sigma(R_p)}{\sigma(R_g)}.\] Plugging this into the first equation yields \[\begin{align*} E(R_p)&=\left(1-\frac{\sigma(R_p)}{\sigma(R_g)}\right)R_f+\frac{\sigma(R_p)}{\sigma(R_g)}E(R_g)\\ &=R_f+\left(\frac{E(R_g)-R_f}{\sigma(R_g)}\right)\sigma(R_p). \end{align*}\] We arrived at a linear relationship between expected returns and risk. The term inside the parenthesis is called the Sharpe ratio.

Remember the assumption from the introduction that investors want to maximize their returns and take on as little uncompensated risk as possible? An investor that follows these goals is called rational in this context. To maximize expected returns \(E(R_p)\) for any given level of risk \(\sigma(R_p)\), we see that the rational investor should choose the portfolio \(g\) that maximizes the slope in the linear equation above, that is, maximizes the Sharpe ratio.

The unique portfolio \(g\) on the Markowitz frontier that achieves this maximum slope is called the tangency portfolio 4.

This implication of this result is called Tobin’s separation theorem 5. It tells us that the investment problem separates into first finding the tangency portfolio \(g\) and then selecting for a suitable proportion \(x\) to match the investor’s preferences. Crucially, we see that an investor’s personal risk preference (\(x\)) is irrelevant to which risky assets they should hold. Every rational investor holds the exact same mix of risky assets \(g\) that maximize the Sharpe ratio.

Market equilibrium and the capital asset pricing model

So, now we know that a “rational” investor should always choose the risky portion of their portfolio to maximize the Sharpe ratio, that is, they should find the tangency portfolio. Let’s assume for a moment that all investors are rational. Let’s also assume that the efficient market hypothesis is true and continue to assume that a risk-free asset exists. What happens then? We just learned that every rational investor holds the tangency portfolio in some proportion according to their risk-preference. Since all investors have access to the same information by the EMH, and assuming they all process this information identically, they all arrive at the same portfolio. It follows that under these assumptions the tangency portfolio is in fact the value-weight market portfolio of risky assets 6.

Why? For the market to be in equilibrium, the total demand of all assets must equal the total supply. Since everyone has the same risky portfolio, call it \(M\) for “market”, it means that the portfolio \(M\) must hold each asset in proportion to its total value in the market, that is, the weight of asset \(i\) in the market portfolio must be \[w_i=\frac{p_im_i}{\sum_{j=1}^Np_jm_j},\] where \(p_i\) is the price of asset \(i\) and \(m_i\) the number of outstanding units of asset \(i\).

We are now ready to derive the famous equation of the capital asset pricing model (CAPM) introduced by Sharpe (1964) 7. The idea is to analyze how the market portfolio reacts if we add an incremental amount of some asset \(i\). Let’s consider some portfolio \(p\) consisting of the asset \(i\) in proportion \(a\) and the market portfolio \(M\) in proportion \(1-a\). The expected return is then \[E(R_p)=aE(R_i)+(1-a)E(R_M)\] and the variance is \[\sigma^2(R_p)=a^2\sigma_i^2+(1-a)^2\sigma_M^2+2a(1-a)\,\mathrm{cov}(R_i,R_M).\] If \(a=0\), the portfolio is entirely the market portfolio. To see what happens to the expected returns and risk if we add or subtract a bit of asset \(i\) to \(M\), we can take the derivative w.r.t. \(a\) and evaluate it at \(a=0\). For the expected return we get \[\partial_aE(R_p)\big|_{a=0}=E(R_i)-E(R_M)\] and for the standard deviation we find \[\partial_a\sigma(R_p)\big|_{a=0}=\frac{\mathrm{cov}(R_i,R_M)-\sigma_M^2}{\sigma_M}.\] The ratio \(\partial_aE(R_p)\big|_{a=0}/\partial_a\sigma(R_p)\big|_{a=0}\) is the slope of the risk-return curve at \(M\). As we already saw, at \(M\) this must be equal to the Sharpe ratio of the portfolio: \[\frac{\partial_aE(R_p)\big|_{a=0}}{\partial_a\sigma(R_p)\big|_{a=0}}=\frac{E(R_i)-E(R_M)}{\left(\frac{\mathrm{cov}(R_i,R_M)-\sigma_M^2}{\sigma_M}\right)}=\frac{E(R_M)-R_f}{\sigma_M}.\] From this we can solve for \(E(R_i)\) and we arrive at the famous CAPM equation: \[E(R_i)=R_f+\beta_i[E(R_M)-R_f],\] where \[\beta_i=\frac{\mathrm{cov}(R_i,R_M)}{\sigma^2(R_M)}.\] So, those many assumptions led us to an asset pricing model that tells us that the expected return of any asset is the risk free rate plus some risk premium, which scales in proportion to the covariance of the asset’s returns and the market portfolio’s returns, measured by the asset’s beta. Beta signifies so-called systematic risk. Note especially that the asset-specific volatility \(\sigma_i\), so-called idiosyncratic risk, is not present in this equation at all! This means that picking a volatile stock simply because one suspects it might have higher beta is not rational.

The conclusion of CAPM for investors, the arithmetic of active management, and SPIVA

Our conclusion from the theory section is clear: if the assumptions of CAPM hold, all rational investors should hold the value-weighted market portfolio. So we have our answer from at least one point of view.

The problem, however, lies in those assumptions. As noted by Fama and French 8, the model does not hold well under empirical observations, and one of the explanations for this is that the assumptions the model makes are too extreme.

But is the conclusion, to hold the value-weighted market portfolio, still sound? If we relax our assumptions, do we still arrive at a similar conclusion? To answer this, we turn to the article The Arithmetic of Active Management by William F. Sharpe 9. The article presents a simple and logical argument in favor of so-called passive investing without making almost any assumptions. We will now go over this argument.

Let the market be a set \(M\) of stocks. The passive investor is defined as the investor whose portfolio \(P\) consists of each stock \(s_i\in M\) with the proportion of \(s_i\in P\) given by \(w_i\) as we discussed above in the market equilibrium section. The active investor is any investor who is not passive. The market return is given by \[R_M=\sum_{s_i\in M}w_iR_i,\] where \(R_i\) is the return of stock \(i\). Let \(x\) represent the proportion of all value invested by active investors in the market. The market return is a weighted average of the passive and active segments, that is \[R_M=xR_A+(1-x)R_P,\] where \(R_A\) is the average return of actively managed capital and \(R_P\) is the average return on passively managed capital. Since \(R_P=R_M\), we get \[R_M=xR_A+(1-x)R_M\implies R_M=R_A,\] and thus the average return of the active segments is also equal to the market return. This gives us conclusion number 1 of Sharpe: “before costs, the return on the average actively managed dollar will equal the return on the average passively managed dollar”.

The second conclusion that Sharpe makes relies on the following assumption: on average, active managers pay more per invested dollar than passive managers. And, since we just showed that the average returns are equal for passive and active investors, we arrive at conclusion number 2: “after costs, the return on the average actively managed dollar will be less than the return on the average passively managed dollar”.

The argument and its conclusion are extremely clear: for the average retail investor it makes more sense to invest passively.

After going over this argument, one may still think: “OK, but I’m not average. I can get my long-term actively managed returns higher than those of the passive investor even after accounting for costs.” To answer this, one need only take a look at the SPIVA dataset, which shows that the vast majority of active fund managers in the US fail to beat the S&P 500 over 10- and 20-year horizons. These people have access to information, tools, and time that the average retail investor does not. The conclusion is again pretty clear: to beat the market you need some kind of serious “edge” that the average investor does not have.

Conclusion

We began this exploration with the Capital Asset Pricing Model (CAPM) and saw that under ideal conditions, a rational investor should always hold the unique portfolio that maximizes the Sharpe ratio: the value-weighted market portfolio. While CAPM relies on idealized assumptions that fail to hold up under empirical testing, William Sharpe’s arithmetic of active management shows that the conclusion still holds even when we relax the strong assumptions of CAPM: on average, passive indexing beats active management after costs. As datasets like SPIVA consistently demonstrate, the vast majority of active fund managers lack the lasting “edge” required to overcome their higher fees.

So, who, if anyone, can actually find this edge, and what does it take?

Recall that under a strict interpretation of the Efficient Market Hypothesis (EMH), asset prices always reflect available information, which makes it impossible to systematically earn risk-adjusted extra returns over the market returns, or so-called “alpha”. Yet real-world counter-examples exist, one famous example being Renaissance Technologies’ Medallion Fund. However, it is clear that such firms possess tools and information that far exceed anything a typical retail investor can dream of.

I leave it to the reader to decide if they have the necessary preconditions to attempt something similar.

Sources

  1. Two Pillars of Asset Pricing, Nobel prize lecture, Eugene F. Fama (2013)↩︎
  2. Portfolio Selection, Journal of Finance, Harry Markowitz (1952)↩︎
  3. The Capital Asset Pricing Model: Theory and Evidence, Journal of Economic Perspective (2004), Eugene F. Fama and Kenneth R. French.↩︎
  4. The Capital Asset Pricing Model: Theory and Evidence, Journal of Economic Perspective (2004), Eugene F. Fama and Kenneth R. French.↩︎
  5. Tobin, James. 1958. “Liquidity Preference as Behavior Toward Risk.” Review of Economic Studies.↩︎
  6. The Capital Asset Pricing Model: Theory and Evidence, Journal of Economic Perspective (2004), Eugene F. Fama and Kenneth R. French.↩︎
  7. Capital Asset Prices: A Theory of Market Equilibrium under Conditions of Risk, Journal of Finance William F. Sharpe (1964)↩︎
  8. The Capital Asset Pricing Model: Theory and Evidence, Journal of Economic Perspective (2004), Eugene F. Fama and Kenneth R. French.↩︎
  9. https://web.stanford.edu/~wfsharpe/art/active/active.htm. Accessed on 6ht, August, 2026. Sharpe’s original article is published in the Financial Analysts Journal in 1991 (Vol. 47, No. 1, pp. 7-9)↩︎