Adaptive System vs. Curve Fit

[TriPack]

Please provide feedback on the following method of making a system's parameter "adapt" to the most recent market behavior. Is this curve fitting? Is this a bad way to do it? Am I in danger of some kind of trading system purgatory for doing things this way? By the way, it has led to better overall results and a smoother equity curve (more consistency) in forward testing over the past couple of weeks which is my main goal.

I have an input that I change by +/- 25 each day that is the length used in an indicator. Here is the way I determine whether to increase, decrease or keep the value the same:

I run an optimization of the past day, and take note of the Length that would have given the largest $ profit. Let's say the old value was 150, and the optimized value is 275 (in the optimization I use minimum increments of 25). I would then add 25 to the next day's value, so tomorrow I would use 175 as my Length. If the optimized value is the same as the old value, then the Length stays the same. If the optimized value is less than the length, then I subtract 25 from the old value and use that for the next day's value.

What do you think about this method?

 

[DT-waw]

So, you assume that the most recent market behavior will be continued in the near future. It may be or not... Market's behavior is non-linear, you'll never know what's gonna happen!
Markets always change and are hard to predict. How can you adapt system's parameters to the future price curve? Impossible. Yes, you can "adapt" to recent, historical curve... only.

On the other hand, I think that every market has it's own character. Some ranges, sets of parameters will work better than the others. Look at daily price ranges, overnight gaps, etc. - their levels oscillate at some average value. There is an order behind the charts - to some degree, but also there's a lot of chaos and unpredictability.

I'll say it again: the best way to deal with market's non-linear behavior is to diversify, use many systems with many sets of parameters. This will smooth your equity curve ( b/c drawdowns will occur at different periods with different magnitude ). Plus you have to be discretionary when something special is going on. For example, if your system is short and Fed announces surprising rate cut - reduce your shorts or go long before system's signal will occur.

 

[jboydston]

Originally posted by DT-waw
... I'll say it again: the best way to deal with market's non-linear behavior is to diversify, use many systems with many sets of parameters. This will smooth your equity curve ( b/c drawdowns will occur at different periods with different magnitude ). ...

This is one of the most profound lessons I have learned about trading. Seems to me that some of the hardest, most profitable lessons to learn are the least exciting and "fun" to implement.

 

[DT-waw]

Regarding equity curve smoothness:
I looked once again at http://iasg.pertrac2000.com/mainframe.asp?

The best fund in terms of average return per year is Hanseatic - Discretionary. Their monthly performance ranges from -15.67% ( May 1997 ) to +40.05% ( March 2000 ). Yearly performance from -24.31% in 1999 to +308.94% in 1998.

One of the biggest funds in terms of assets under management: Dunn Capital WMA. Yearly returns: worst year 1985 ( -21.85% ), best year 1995 ( +98.63% ).
Forget about smoothness...

Maybe if you invest in basket of top 10 funds, you'll get close-to-consistent profits.

 

[metooxx]

Originally posted by TriPack
Please provide feedback on the following method of making a system's parameter "adapt" to the most recent market behavior. Is this curve fitting? Is this a bad way to do it? Am I in danger of some kind of trading system purgatory for doing things this way? By the way, it has led to better overall results and a smoother equity curve (more consistency) in forward testing over the past couple of weeks which is my main goal.

I have an input that I change by +/- 25 each day that is the length used in an indicator. Here is the way I determine whether to increase, decrease or keep the value the same:

I run an optimization of the past day, and take note of the Length that would have given the largest $ profit. Let's say the old value was 150, and the optimized value is 275 (in the optimization I use minimum increments of 25). I would then add 25 to the next day's value, so tomorrow I would use 175 as my Length. If the optimized value is the same as the old value, then the Length stays the same. If the optimized value is less than the length, then I subtract 25 from the old value and use that for the next day's value.

What do you think about this method?

I think if you are making money with it, stick to it.

It seems to make sense on the surface ...

 

[TriPack]

Originally posted by DT-waw
So, you assume that the most recent market behavior will be continued in the near future. It may be or not... Market's behavior is non-linear, you'll never know what's gonna happen!
Markets always change and are hard to predict. How can you adapt system's parameters to the future price curve? Impossible. Yes, you can "adapt" to recent, historical curve... only.

On the other hand, I think that every market has it's own character. Some ranges, sets of parameters will work better than the others. Look at daily price ranges, overnight gaps, etc. - their levels oscillate at some average value. There is an order behind the charts - to some degree, but also there's a lot of chaos and unpredictability.

I'll say it again: the best way to deal with market's non-linear behavior is to diversify, use many systems with many sets of parameters. This will smooth your equity curve ( b/c drawdowns will occur at different periods with different magnitude ). Plus you have to be discretionary when something special is going on. For example, if your system is short and Fed announces surprising rate cut - reduce your shorts or go long before system's signal will occur.

Well I guess I should have said "overly curve fit" (implying unprofitable into the future) rather than just "curve fit". But since in forward testing it is outperforming any given static parameter version of the system then I suppose I already know that it isn't overly curve fit.

As far as assuming that the market's recent behavior will be continued in the future I suppose by changing the length input the way I have been, I am assuming this to be the case. However, by not changing the length aren't I implicitly assuming that the market's cycles of high to low will stay static into the future (which I know is a false assumption in the first place)? Thus I don't see why making adjustments is necessarily a bad thing especially because by doing so, I'll at least have a greater chance of being close to the optimal cycle length for my indicator if the market's cycles last for several days (which it appears they do according to my system tests).

And as was pointed out by metooxx, if it is working no need to change but I'm almost surprised that he or someone else hasn't been using a strategy like this to adapt to recent market behavior.

By "many systems" do you mean many uncorrelated systems?

 

[dbphoenix]

Originally posted by TriPack


Thus I don't see why making adjustments is necessarily a bad thing especially because by doing so, I'll at least have a greater chance of being close to the optimal cycle length for my indicator if the market's cycles last for several days (which it appears they do according to my system tests).

I used to think that "curve-fitting" was a sin to be avoided at all costs. But over the years it finally dawned on me that "tweaking" the parms of an indicator, or even of timeframe, is a form of curve-fitting. This search for optimal results is partly a discovery of how the market works and partly a discovery of one's own psychological profile. Therefore, I don't look down my nose at it as I used to, though I do understand that the tweaking in and of itself is relatively useless and certainly misleading; one must take the next step and forward-test all this in real time. Which is why I don't pay a whole lot of attention to discussions of backtests unless they have at least some forward-testing.

As for having a diversity of tactics, if not a diversity of strategies, there is something to be said for being able to pull something else reliable out of the toolbox in case whatever one is doing isn't working and isn't likely to work given current market conditions. However, unless one knows in advance what the market conditions are going to be, he can't very well choose an appropriate strategy.

Therefore, it would seem logical to go a step further with what you're doing and have several options according to what the market tells you that day. For example, one might have a strategy that combines trend-following and range-bound trades and employ that strategy until the market shows its hand, then choose whatever strategy seems appropriate for that day (though the market may change from trending to range-bound - or vice versa - during the day).

--Db

 

[gladiator]

If all the entire parameter set is positive i see nothing wrong with optimizing. If on the other hand u have a bunch of holes, forget it !

"Curve Fitting" is forcing the system to perform on the data.

"Optimizing" should be looking for the best parameters out of a set of positive performing values.(this is often abused and turns into above).

 

[turgoveca]

last couple of months i have given a closer look on strategy optimization with varying parameters. the problem i could not overcome is how do you define optimized system probability of success? for instance if the system is optimized over 10000 bars of data and has given say 200 trades roughly half of them profitable what is the probability of further profits (and how much) over the next say 1000 bars of data? i realize that is quite a complex mathematical problem but i am pretty familiar with calculus and probability theory and i am willing to go there in depth. is there a book and/or articles that discuss those matters?
it has something to do with iterations but how many and how often?
any idea will be appreciated.
good luck (if that has anything to do with trading

 

[rcreal]

Adaptive systems are useful if they adjust to market conditions.

Instead of something like:

if c> 3 points above 10 day moving average then go long

use if c> 2*atr +10 day moving average then go long

the buy level will be dynamically adjusted with volatility.