Overfitting and out-of-sample testing

Overfitting means tuning a strategy so closely to past data that it captures noise instead of anything that might repeat. An overfitted strategy has an excellent history and a poor future.

How it happens

• Testing many combinations of settings and keeping the best one.
• Adding filters that remove a few specific losing trades.
• Choosing the test period after seeing which period looks best.

Each of these can be done innocently. Together they can produce results that are almost meaningless.

Out-of-sample testing

The standard defence is to split the data. Develop and tune the rules on one period, the in-sample data. Then test the finished rules, unchanged, on a separate period that played no part in building them, the out-of-sample data.

If performance collapses out of sample, the strategy was probably fitted to noise. Walk-forward testing repeats this process across several periods to make the check stronger.

Fewer rules and fewer settings generally mean less room to overfit.

Educational content only. Not financial advice.