How much out of sample testing is enough?

[illiquid]

A bigger sample will just give you more excuses to keep trading something that doesn't work anymore -- another way to look at things

 

[risktaker]

I often use equity charts to eyeball and get a feel for how a system/equity behaved during the testing period and then will keep an eye afterward to see if something changes. If the equity chart looks very different from the testing phase, then it's a warning that *something* has changed. For example, attached chart "trade this" would be an equity curve along the lines of what I currently trade. The next chart shows a change in the character of another equity around trade 180, so I would have to reevaluate this system even though it's still profitable, but is showing some kind of warning.

 

[risktaker]

This equity curve is showing a change in the security's "character", which is an early warning sign to be cautious. Here, the equity curve was somewhat predictable until around trade 180, when it starts to zigzag quite a bit more than before. While it doesn't mean that you should stop trading it, it *does* tell you that whatever you were exploiting/profiting from before may now be disappearing and you need to reevaluate whether to continue trading this security/system. Often, the equity curve will still make new highs, but not with the same smoothness as before.

 

[Cheese]

Quote from SideShowBob:

I would like some expert opinions to throw against a wall with my amateur opinion. Is optimization on 20 years of daily data followed by 2 years of out of sample data enough? If not how much is enough? 20%? 50%?

SSB

This is an interesting question.
I would suggest not less than the last 12 months to date.
You may well need to apply the last 2 or 3 years.
But raw data for any one market will not yield a reliable or playable result for predictive purposes.
Of course it is entirely up to you.