How to pick the best walkforward based strategy ?

[KarthikK]

I am trying to build a strategy via walkfwd optimization. So roughly, I take last 6 months of data and will build a bucket of profitable strategies to play with the next 1 month and keep moving the sliding window. All looks good, only that the most profitable one from the bucket is really not profitable. I have not intentionally looked into other strategies from the bucket - because I am worried about data snoop.

It brings a question, even with walkforward testing, I suspect many of you may have multiple strategies that are potentially profitable. How do you go about picking the right one, without data snoop, hindsight bias etc.

 

[sysdevel99]

Karthik,
I hate to break it to you but in that case you may not have a profitable strategy at all, just a strategy that found something/exploited some edge based on the noise data from the past and that noise changes all the time.
Ignore the sliding window for a second, but if you have a strategy, optimize it on the "training data" and then let it go against new unseen data and it doesn't work, it's not going to be profitable (you won't have the signal in your strategy).

I've a similar challenge http://www.elitetrader.com/et/index.php?threads/would-you-trade-this-system.296196/ where my strategy is still profitable on the walk forward but only on a subset of the markets. It's a slightly better starting point though cause at least those markets remain profitable for now but it's also not as broadly profitable as I'd like in the walk forward but hopefully profitable enough to trade it.

 

[KarthikK]

@sysdevel99 - Yeah, you may be right. But what I am doing is also an experiment. I actually do not have a lot of indicators and system is mainly based on pure price action alone (Roughly, if the price breaks a certain point everyday, I trade with a certain risk/reward). But then if trends do develop and your parameters align with the trend, this might work (since you are training the "closest" sliding window). Again, hypothesis I think there is greater chance of overfit with this and that may be what is happening.

Why do you say your system needs to work on different markets ? After all markets are unique, so shouldn't they be different.

 

[sysdevel99]

@sysdevel99 -

Why do you say your system needs to work on different markets ? After all markets are unique, so shouldn't they be different.

I don't know are they aren't they all driven by fear/greed/supply/demand at the end of the day? I don't know that's the bottom line. I've other systems (not as smooth of an equity curve) that work on a much broader set, that's why I'm concerned.