I would like some expert opinions to throw against a wall with my amateur opinion. Is optimization on 20 years of daily data followed by 2 years of out of sample data enough? If not how much is enough? 20%? 50%?
SSB
SSB
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
How much out of sample testing is enough?
- Thread starter SideShowBob
- Start date
20 years? Useless. Most markets have a "memory" of maybe 5 years at most. Concern yourself with the last 8-10 years at most. Markets today are very different from 5 years ago. Would you take a Ferrari on a dirt race track and expect to win? Each market environment requires a different vehicle to participate in it.
OK, even if I narrow to the last 5 years, what would you recommend for the out of sample period?
SSB
SSB
Daily data is crap anyway, I wouldn't trust 100 years of backtesting.
Quote from SideShowBob:
I would like some expert opinions to throw against a wall with my amateur opinion. Is optimization on 20 years of daily data followed by 2 years of out of sample data enough? If not how much is enough? 20%? 50%?
SSB
How much out of sample testing is enough?
However much you need, to feel confident. Fill in the blank: If I tested _____ months out of sample, then started trading my system, then experienced big losses, I wouldn't blame my test procedures.
However much you need, to feel confident. Fill in the blank: If I tested _____ months out of sample, then started trading my system, then experienced big losses, I wouldn't blame my test procedures.
AaronCapps
Global Futures
the amount of out of sample data should depend on how many times a year your method trades. try to get a lot of trades in your out of sample data to get a better feel if the idea is holding up.
Quote from horribilicus:
How much out of sample testing is enough?
However much you need, to feel confident. Fill in the blank: If I tested _____ months out of sample, then started trading my system, then experienced big losses, I wouldn't blame my test procedures.
I don't have enough data to feel confident....ever. Then again on the other hand I've thrown away money on systems that tested well but stopped working the moment I started trading them....:eek:
Someone recommended using 1/3 of data to build the system, 1/3 to optimize then 1/3 as out of sample. That's actually a pretty good measure.
As for daily, I prefer intraday because it reacts much faster, but there's also a lot more noise which cancels out the some of the benefits of the faster action.
I prefer to trade some systems on daily and others on intraday. ...
SSB
FredBloggs
Guest
30 is supposed to be the least number of trials required to prove any degree of significance in statistical tests.
so, i would use this number given the ever changing nature of markets. look at the last 30 potential trades.
so, i would use this number given the ever changing nature of markets. look at the last 30 potential trades.