The normality of log-returns

A client asked me to look into Equity Option pricing formulas that do not assume normality of returns. That goes against the default stance in Mathematical Finance theory that claims that stock returns are normally distributed. Before delving into the industry literature to find research papers with these alternative models, I think it was in order to test the claim that stock returns follow a normal distribution. I’m skeptical by nature, so measuring if stock returns are normal or not, was just as useful for the client as myself. After all if the majority of stocks under analysis do follow a normal distribution then the skepticism would be misplaced.

In a previous post I briefly touched on the discrepancy between the theory in the academic world and how practitioners in the industry value financial instruments. This project will also explore that. One of the foundational blocks in Finance, and derivatives pricing in particular, is the Black-Scholes model (BS) model. I briefly cover the assumptions of this model in this blog post.

Dataset and testing Link to heading

The normality test I picked is the Kolmogorov–Smirnov test because it works well with large data sets. The test was performed at $ a = 0.01 $ significance level. The dataset is made of over 7,000 stock symbols from the US market, observed daily. Each series varies in length, based on when the company went public. Some will have “only” 5 years worth of data, while others 30 years. The dataset is large enough for statistical inference.

The dataset is available upon request.

Only about 100 stocks out of 7000+ show normal log-returns, so it’s safe to conclude that the assumption in the Black-Scholes model is not validated by reality, and thus by extension, the other claim that stock prices follow a log-normal distribution is also invalid.

The formula for log-returns is:

$$ R_t = ln( \frac {P_t} {P_{t-1}} ), $$

where $ t $ denotes the frequency of the data.

The Python driver code is:

# computing/log_returns_normality.py

import numpy as np
import pandas as pd

from pathlib import Path
from scipy.stats import kstest


# path containing the individual stocks OHLCOV data in .csv format
dataset_dir = Path.cwd() / "datasets" / "US_equities_OHLCOV"


def normality_test(time_series, var_name):
    """
    Perform Normality test based Kolmogorov-Smirnoff (KS) approach.
    The KS test works well with large data sets.

    :param time_series: the input time series -> list-like or pandas-like obj
    :param var_name: name of variable -> str
    :return: p-value of the normality test --> float
    """
    if time_series.empty:
        return None
    return kstest(time_series, "norm")


significance_level = 0.01
obs = 0
normal= 0
non_normal = 0
# dict below will have shape d = {"AAPL": No|Yes}, where No indicates non-normal log-returns
d = {}
for file_path in dataset_dir.iterdir():
    stock_ticker = file_path.stem
    stock_OHLCOV = pd.read_csv(dataset_dir / f"{stock_ticker}.csv")
    # log return = ln(P_t / (P_t-1) )
    stock_log_return = np.log(stock_OHLCOV["Close"] / stock_OHLCOV["Close"].shift(1))[1:]  # [1:] to ignore 1st Nan val.
    test_results = normality_test(stock_log_return, stock_ticker)
    if test_results is not None:
        obs += 1
        if test_results.pvalue < significance_level:  # reject null hypothesis of normality
            non_normal += 1
            d[stock_ticker] = "No"
        else:
            normal += 1
            d[stock_ticker] = "Yes"

# Summary statistics
print("Total observations: ", obs)
print("Normal log-returns: ", normal)
print("Non-normal log-returns: ", non_normal)
normal_over_total = round((normal / obs) * 100, 2)
print("Normal/Total: ", normal_over_total, "%")
# Convert dictionary d to pandas Dataframe
df = pd.DataFrame(d.items(), columns=["Symbol", "Normal log-returns"])

Results of the normality test Link to heading