Why past results mislead

A backtest shows what would have happened. It's easy to build one that looks great and then disappoints. These are the usual reasons, and what you can do about each.

Overfitting#

Every number you tune to the past, such as a 150-day average instead of 200, or one more condition, makes the past look better. Part of that improvement is the rule fitting noise that won't repeat.

Warning signs: the result changes a lot when you nudge a number, the strategy has many conditions, or a rule only makes sense in hindsight.

Trying too many ideas#

Try enough variations and one will look excellent by luck alone. The best of fifty attempts is not as good as it looks. Keep count, and be more skeptical the more you have tried.

Survivorship#

The ticker list holds what trades today. Companies that failed and funds that closed are missing, so a rule that picks individual stocks from today's list is picking only from survivors. Funds that track broad indexes are much less affected.

Hindsight#

Tactfolio's readings use only prices up to the day they are read, and trades happen at that day's close. Hindsight can still sneak in through your own choices: picking a fund because you know it did well last decade, or adding a rule because you remember a crash it would have dodged.

Changing markets#

A rule that worked while one kind of asset boomed may not work when that stops. Gold sits on the defensive side of three of the four starter patterns, and gold did very well in both decades they were tested on. Part of those results is gold, not the rule.

Costs and taxes#

Backtests charge a trading cost but not taxes or commissions. Frequent trading in a taxable account can turn a good backtest into an ordinary result.

How to guard against it#

  • Keep rules simple. Prefer few conditions and round, sensible numbers.
  • Test another period. The starter patterns were chosen on 2016–2026 and then checked on 2006–2016, a separate decade that includes the 2008 crisis. Only rules that improved drawdown and Sharpe in both were kept.
  • Hold out recent years. Tune on older data, then check once on the most recent years. AI assistants can do this for you; see Use Tactfolio from your AI assistant.
  • Nudge the numbers. If 150 or 250 days works much worse than 200, be suspicious of 200.
  • Raise the costs and see whether the edge survives.
  • Expect less. Assume real results will be worse than the backtest.