Backtesting time period

dtrader98 said:
Some of you are focusing way too much on the statistics, without really seeing the forest from the trees.
If you have only a small subset of data, there is no reason to believe the same conditions will necessarily persist on a new small subset of data. Market properties change a lot, and therefore a lot of market conditions should be required to give you more confidence, not a lot of trades in a small subset of market conditions.
More...

Yes, good point. In addition to ensuring it generates enough trades, the historic data should also be as representative as possible of the market regimes you might expect to see going forward.
 
Sergio77 said:
...
I just saw this but I am not sure I fully understand it. Can this bound be used to determine the number of trades based on win rate?...
More...

OK… I think I am starting to get my head around what the Chernoff Bound does/doesn’t tell us (sorry … it has taken me this long to get there! … a little bit of knowledge is a dangerous thing …):

Here’s an example of how I understand it could be used … If you have a strategy with a win rate of say 55%, then the Chernoff Bound gives the minimum number of trades you will need in a backtest to be say 99% certain of getting backtest results with a win rate of greater than 50%.

That is not the same as the minimum number of trades required to give x% confidence intervals on the difference say between the average trade measurements. Therefore it does not answer the question of how many trades we need in a backtest to be able to compare In Sample and Out Of Sample results statistically.
 
You must log in or register to reply here.
Back
Top