Quote from trackstar:
Back to RoughTrader...
So basically I should do an optimized backtest of say, 2006-2007, than see how those parameters do from 2007-today?
Or something of the like?
The basic idea of walk-forward optimization is this:
Your historical data has N sessions (days)
Choose a window of this data with K sessions, K < N. K is the in-sample data length.
Choose a length of data J to be your out-of sample data. In total,
K + J <= N
The process is iterative. You start by optimizing the strategy over the first K sessions. Once the optimal parameters have been found, apply them to the historical data starting from session K+1 until K+J. Record the PNL for each session K+1 -> K+J.
Slide the in-sample window forward by J sessions. Optimize, retrieve parameter values, then apply the values to obtain session PNL values for out-of-sample sessions K+J+1 -> K+2J.
Your in-sample optimizations should be performed over data windows:
1 -> K
1+J -> K+J
1+2J -> K+2J
1+3J -> K+3J
etc.
etc.
Your out-of-sample PNL values are extracted from the following windows:
K+1 -> K+J
K+1+J -> K+2J
K+1+2J->K+3J
k+1+3J->K+4J
you continue this iterative process until you have run out of data (i.e., K >= N).
Then the out-of-sample PNL values are sequentially stitched together to form a realistic, forward-looking sequence of trades that mimics how the system behaves with unseen price data.
The selection of K and J values is a subject of debate. I personally choose to keep J relatively small (5 sessions, reoptimizing every Friday after close), and do a loose ratio optimization to find the best K value.
If you have ANY parameters that the equity curve is sensitive to, IMO it is imperative that this analysis be done. Otherwise, you are just left with the result of optimization of historical data. Such a result is NOT forward-looking at all, and as such does not provide the information necessary to judge confidence in the system.
rt
