Assumptions in Backtesting

A guide on the common unrealistic assumptions made during quant backtesting and simulation, and the risks they pose.

TL; DR

• Unaccounted Market Impact: Ignoring the effect of large orders on market prices can lead to overestimated strategy performance.

• Ignoring Transaction Costs: Overlooking fees, spreads, or slippage can lead to an overestimation of profitability.

• Underestimating the Impact of Liquidity: Failing to consider liquidity constraints may render a strategy non-viable in live markets.

---

Insights

Quantitative backtesting and simulation are powerful tools for evaluating the potential performance of trading strategies based on historical data. However, these methods can lead to misleading results if they rely on unrealistic assumptions. Below are examples of such assumptions and the risks they pose.

Unaccounted Market Impact

Assumption:

When a large order is placed, it does not significantly affect the market price.

Risk:

• Large orders can move the market, especially in less liquid assets.

• The strategy may appear profitable in simulation but fail in live trading.

Expand for more details

In reality, significant trades can cause price slippage, which occurs when the market price changes as the order is being executed. This can lead to higher costs and lower profits than anticipated in a simulation that does not account for market impact.

Ignoring Transaction Costs

Assumption:

Trading does not incur any costs such as fees, spreads, or slippage.

Risk:

• The profitability of a strategy is overestimated.

• In reality, transaction costs can significantly erode returns.

Expand for more details

Transaction costs can have a substantial impact on the net return of a trading strategy. Ignoring these costs in simulations can lead to an unrealistic assessment of a strategy's profitability.

Underestimating the Impact of Liquidity

Assumption:

Assets can always be bought or sold at the current market price regardless of the quantity.

Risk:

• The strategy may not be executable in live markets due to insufficient liquidity.

• The model's trades could themselves affect liquidity and pricing.

Expand for more details

Liquidity risk is the risk that a given security or asset cannot be traded quickly enough in the market to prevent a loss (or make the required profit). A simulation that does not account for liquidity constraints may show a strategy to be viable when it is not.

To mitigate these risks, it is essential to use realistic assumptions in quant backtesting and simulation. This includes accounting for transaction costs, market impact and liquidity.