Introduction
This post provides an overview of what I call 'static' versus 'walk-forward' curve-fitting. Trading software applications generally don't enable walk-forward adaptation of model/indicator parameter values by default, although with some coding it often can be done.
<B>Static Curve Fitting</B>
The following points describe a problematic and often frustrating scenario.
<ul><li>We build a trading strategy that makes use of a few technical indicators (data transformations, rules etc.) to output a trading signal.</li>
<li>We then run a curve-fitting process to determine what indicator parameter values have been optimal for a particular market over the last n years (modeling period).</li>
<li>We then monitor the performance of the system post construction and we are, more than likely, disappointed to see that the performance does not resemble the hypothetical performance over the modeling period.</li></ul>
For this scenario, the modeling period and the trade simulation period are one and the same. A small book could be written describing the pitfalls of this approach. The bottom line is that hypothetical performance over the modeling period is, in the majority of cases, unrealistic.
<B>Walk-Forward Curve Fitting</B>
Walk-forward curve fitting can be implemented using various trading software applications and produces more realistic hypothetical trading results. Some trading applications require a significant amount of programming effort to enable walk-forward adaptation because they were not designed for this purpose. Others were designed specifically for this purpose. The following points describe walk-forward curve fitting at an application independent level.
<ul><li>We use the same trading strategy as that in the static curve fitting scenario.</li>
<li>A lookback period or modeling period of 2 years is used, once again, to determine optimal indicator values.</li>
<li>A curve fitting algorithm is applied each trading day, or each X number of trading days, over a 5 year trading simulation period using the prior 2 years for optimization walking-forward. Assuming that our modeling period is 504 trading days, data from t (-504) to t(-1) is used to determine optimal indicator parameter values, where t is the trading day that the signal is being generated for. This is repeated for each trading day over the 5 year period. Signals are generated for the last 5 years, so a total of 7 years of data is required.</li></ul>
The walk-forward curve-fitting process produces more realistic hypothetical trading results over our trade simulation period. It is worth noting that curve/function fitting can occur at more levels than just the determination of model/indicator values. Therefore, there is potential for us to fool ourselves on many levels. Some examples of curve-fitting that do not involve indicator parameter values are:
<ul>
<li>Determination of the system parameter values such as the walk-forward modeling period and the performance metrics. If the walk-forward simulation is repeated over and over using different modeling periods etc. until optimal system parameter values are determined then we have curve-fit and potentially over-fit our system over our trade simulation period. i.e We have polluted our trade simulation period.</li>
<li> Selection of the trading strategies to apply to a given market. If we evaluate a large set of trading strategies over our simulation period and then select a subset to use then curve-fitting and potentially over-fitting has occurred.</li>
<li>Selection of the data to use for the walk-forward simulation. It is possible that the market dynamics that were dominant over the last n years will cease to be dominant over the next year. For modeling on the daily time frame, use 30+ years of data if that is possible.</li>
</ul>How can we avoid curve-fitting over our trade simulation period? Do everything on a walk-forward basis or at least as much as we can, and always be well aware of the danger.
Regards,
James
Thanks for the laughs