How many years of backtesting do you use for testing stocks?

[helpme_please]

How long should backtest historical period be is a tricky question.

If backtest period is too long, say, 30 years, the strategy may not be suitable for today's market because the market 30 years ago can be very different from today's market. For instance, U.S market used to trade in 1/8 increments. Later, stock prices became decimalised. I did not test this but I think the market structure will change quite a bit as a result and that will affect backtesting results. Strategy that works before decimalisation should not work as well after decimalisation.

If backtest period is too short, say, 3 years, and past 3 years are bull markets, strategy developed as a result may not work well in bear markets. Too little historical data used in testing can also lead to over-fitting.

Question to elitetraders here is what are the guidelines you use for choosing a suitable length of period for backtesting?

 

[JSOP]

How many years of backtesting do you use for testing stocks?

10 years. Contains enough of different market conditions to test the strategy and yet still relevant for today's market.

 

[globalarbtrader]

How long should backtest historical period be is a tricky question.

If backtest period is too long, say, 30 years, the strategy may not be suitable for today's market because the market 30 years ago can be very different from today's market. For instance, U.S market used to trade in 1/8 increments. Later, stock prices became decimalised. I did not test this but I think the market structure will change quite a bit as a result and that will affect backtesting results. Strategy that works before decimalisation should not work as well after decimalisation.

If backtest period is too short, say, 3 years, and past 3 years are bull markets, strategy developed as a result may not work well in bear markets. Too little historical data used in testing can also lead to over-fitting.

Question to elitetraders here is what are the guidelines you use for choosing a suitable length of period for backtesting?

It depends a lot on trading speed and the style of trading. So for example an HFT firm will use around 3-12 months of tick data for fitting, because the profitability of strategies is so high their alpha decay means the strategies can only be exploited for a few months. I'm doing medium speed trend following, and I use as much data as possible (up to 40 years of daily data in some markets).

The higher the 'true' performance of your strategy, the less data you will need. Here's a table from a book that some bloke wrote:

True SR    0.2    0.5     1.0     1.5      2.0
Years      45     20      6       3        1.4

SR= Sharpe Ratio (the actual realisable Sharpe, not just some inflated backtest nonsense)

Also, fitting across instruments can increase the amount of effective data points you have and so make better use of limited data. For example, the SR on a single instrument for one of my trading rules is around 0.24 - which means from the table above I'd need decades of data. But for 40 instruments the SR will be approximately doubled, so I only need 20 years of data to make fitting decisions.

If you do use relatively small amounts of data, make sure you use very robust fitting techniques that will account for this.

GAT

 

[Dicer]

How long should backtest historical period be is a tricky question.

If backtest period is too long, say, 30 years, the strategy may not be suitable for today's market because the market 30 years ago can be very different from today's market. For instance, U.S market used to trade in 1/8 increments. Later, stock prices became decimalised. I did not test this but I think the market structure will change quite a bit as a result and that will affect backtesting results. Strategy that works before decimalisation should not work as well after decimalisation.

If backtest period is too short, say, 3 years, and past 3 years are bull markets, strategy developed as a result may not work well in bear markets. Too little historical data used in testing can also lead to over-fitting.

Question to elitetraders here is what are the guidelines you use for choosing a suitable length of period for backtesting?

https://www.winton.com/research/experiment-and-observation-in-quantitative-investment-management

depends on your data frequency.

Usually, more is better. People always think the length of data must be relevant to today's market, but how to measure whether they are relevant or not? If you think you can choose the optimal timeframe which is relevant for today's market, it means you can quantify it and choose the longest time frame to do a walk-forward optimization at a given scenario.

but yes. you need at least 10 years of data. 7 years in sample, 3 years out of sample.

However, for me, I will choose at least 15 years in sample data. for 1-minute data, I will still use at least 15 years of data. (about billions data points)

Just remember: the backtest report just shows your strategy profitability tendency, it doesn't guarantee success. Using more data can let you know the weakness of your strategy and let you prepare for the potential big MDD.

However, if your strategy shows a good performance in the recent 15 years, your strategy is usually successful (assume your strategy is explainable).

 

[lindq]

Question to elitetraders here is what are the guidelines you use for choosing a suitable length of period for backtesting?

In my experience, the amount of time is less important than selecting for a specific list of market conditions, and accounting for those conditions in my trading rules.

Do I need to be concerned about volatility, range, momentum, bull or bear markets? Yes, then I look for the periods of time that the markets exhibit those conditions, and code for it. For me, that covers it.

 

[traider]

In my experience, the amount of time is less important than selecting for a specific list of market conditions, and accounting for those conditions in my trading rules.

Do I need to be concerned about volatility, range, momentum, bull or bear markets? Yes, then I look for the periods of time that the markets exhibit those conditions, and code for it. For me, that covers it.

Do you mean you run 1 set of rules for bull and another set for bear markets

 

[shatteredx]

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2308659

 

[lindq]

Do you mean you run 1 set of rules for bull and another set for bear markets

Absolutely. They are two different environments for equities, deserving of different approaches to trading.

Whenever I go sailing, I want to know which way the wind is blowing, and how hard, before I put the sails up.

If you can factor market direction, momentum and volatility into your strategies, that will take you a long way.

 

[helpme_please]

Thank you for the graph. From the graph, a good guideline is to have at least 600 trials in the backtest to avoid over-fitting. Unfortunately, I am not sure what exactly is meant by each trial. Does 1 trial refer to 1 trade?

So, if time frame is daily, around 10 years of data history is needed. Does it mean for intra-day backtest with time frame in minute, only weeks of data history is enough?

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2308659

 

[ph1l]

Unfortunately, I am not sure what exactly is meant by each trial. Does 1 trial refer to 1 trade?

So, if time frame is daily, around 10 years of data history is needed. Does it mean for intra-day backtest with time frame in minute, only weeks of data history is enough?

In his "Advances in Financial Machine Learning" book
Marcos Lopez de Prado (author of the paper in the Number of Trials vs. Minimum Backtest Length in hears graph) writes

3. Do not backtest until all your research is complete (Chapters 1–10).

Then, he defines trial as the number of backtests on a dataset.

4. Record every backtest conducted on a dataset so that the probability of backtest
overfitting may be estimated on the final selected result (see Bailey, Borwein,
L´opez de Prado, and Zhu [2017a] and Chapter 14), and the Sharpe ratio may
be properly deflated by the number of trials carried out (Bailey and L´opez de
Prado [2014b]).