"Improving" systems usually make them worse?!

sheepsucker · Mar 29, 2012

Usually when I try to "improve" one of my systems by adding some filter (eg. daily trend) to take a trade I end up making the system worse..

Frustrating..

It seems the idea should work almost straight away as it is without filters or improvements for eg. daily trend or then throw it in the garbage and try something new.

DT-waw · Mar 29, 2012

dont expect anything else having your nickname ;-)

magicz · Mar 29, 2012

this is what we call curve fitting when there might not be any correlations.

jeffalvinson · Mar 29, 2012

Quote from sheepsucker:

Usually when I try to "improve" one of my systems by adding some filter (eg. daily trend) to take a trade I end up making the system worse..

Frustrating..

It seems the idea should work almost straight away as it is without filters or improvements for eg. daily trend or then throw it in the garbage and try something new.

In the past 12 years I have developed 6 different mechanical trading systems by back-testing them with Index and ETF data.
You can improve a system if you have enough data (minimum 2 years) to confirm your improvements.
"But," even after developing ideal systems, they still seem to have a shelf life before there performance begins to degrade due to
the market personality changes.

dom993 · Mar 29, 2012

When evaluating filters, I actually look at the performance figures for the inverse filter (that is, only the trades candidate to be filtered in the final system).

It is even more difficult to come-up with good filter (say, a P/F for the inverse filter of 0.67 or less), than it is to get to a good system (P/F 1.5 or more).

The big reason for that, in my experience, is the "randomness" of the large winners ... in the past I have been able to identify pretty good filters when considering performance of my 1st target, but that was never translating into any performance increase for the runner - quite the contrary.

IMO, the tendency of big winners to "prefer" marginal setups comes from the surprise factor ... when the majority is caught on the wrong side ... you never get that if you are taking the most obvious or cleanest setups. But those marginal setups have a lower win%.

I hope this makes some sense.

abattia · Mar 30, 2012

Quote from dom993:
... The big reason for that, in my experience, is the "randomness" of the large winners ... IMO, the tendency of big winners to "prefer" marginal setups comes from the surprise factor ... when the majority is caught on the wrong side ... you never get that if you are taking the most obvious or cleanest setups. But those marginal setups have a lower win%...

+1
Great observation!

ocean5 · Mar 30, 2012

Quote from sheepsucker:

Usually when I try to "improve" one of my systems by adding some filter (eg. daily trend) to take a trade I end up making the system worse..

Frustrating..

It seems the idea should work almost straight away as it is without filters or improvements for eg. daily trend or then throw it in the garbage and try something new.

Keep your settings,but do not trade instead of ''imroving'',when you see it is not matching.

Cherrypicker · Mar 30, 2012

Kiwi

sheepsucker · Mar 30, 2012

Thanks, some good points here.

I dont think it could be classified as curve-fitting if you add a daily trend filter. eq. CloseD(1) > CloseD(3). If the strategy is on the long-side.

As for the surprise-factor yes, could play a role. But in my strategies the winners are quite constant whereas the losers wary in size.

It seems I am not able to filter out the conditions when the strategy works and not. But the big losers tend to come in clusters so looking to add a filter to take a 2-week break after a big loser.

HurricaneUS · Mar 30, 2012

Quote from sheepsucker:

Thanks, some good points here.

I dont think it could be classified as curve-fitting if you add a daily trend filter. eq. CloseD(1) > CloseD(3). If the strategy is on the long-side.

As for the surprise-factor yes, could play a role. But in my strategies the winners are quite constant whereas the losers wary in size.

It seems I am not able to filter out the conditions when the strategy works and not. But the big losers tend to come in clusters so looking to add a filter to take a 2-week break after a big loser.

track volatility / atr and maybe adjust stops based on volatility...consider standing aside when volatility gets too high