I have 13 months of intraday data that I have optimized a strategy on. I have been watching it real time for a couple weeks and it seems to be providing the results it should. Now Ive been reading about curve fitting and walk forward testing and Im not sure if Ive tested it correctly. I pretty much just did an exhaustive optimization....Can someone explain the walk forward optimization and how I should use it to test the system with the 13 months of data? I'm confused.....
Walk Forward Optimization?
- Thread starter mij
- Start date
Walk-forward testing is a specific application of a technique known as cross-validation. It means you take part of your data to optimize a system, and part of the data to validate.
So, suppose you consider a strategy around a moving average. You take the first 3 months of data, and find that for that period a 20-minute moving average was optimal (using tick data). You then validate this rule by assessing its performance for the 4-th month (i.e. profit, reward/risk or any other statistic of interest). Next, you repeat the optimization using data from month 2-4, and validate using month 5, and keep repeating this until you've reached the end of the data. The performance you get for the validation months (4-13) are your out-of-sample performance.
You don't necessarily have to restrict yourself to walking-forward. For example, you can also consider "leave-one-out" cross-validation, where your optimization sample is 12 months (not necessarily consecutive) and your validation sample is one month, and repeating this 13 times (each time leaving a different month out).
So, suppose you consider a strategy around a moving average. You take the first 3 months of data, and find that for that period a 20-minute moving average was optimal (using tick data). You then validate this rule by assessing its performance for the 4-th month (i.e. profit, reward/risk or any other statistic of interest). Next, you repeat the optimization using data from month 2-4, and validate using month 5, and keep repeating this until you've reached the end of the data. The performance you get for the validation months (4-13) are your out-of-sample performance.
You don't necessarily have to restrict yourself to walking-forward. For example, you can also consider "leave-one-out" cross-validation, where your optimization sample is 12 months (not necessarily consecutive) and your validation sample is one month, and repeating this 13 times (each time leaving a different month out).
Murray Ruggiero
Sponsor
TradersStudio has built in walkforward testing. Here is a link to the write up on our site
http://www.tradersstudio.com/Overview/tabid/68/Default.aspx?PageContentID=23
We also have a large PDF explaining it which is free , you just need to be register on our site. Here is a link to register.
http://www.tradersstudio.com/UserRegistration/tabid/56/Default.aspx
You will receive a verification code to use the first time you log in. After that you can access this document as well as other tutorials ,videos ect.
http://www.tradersstudio.com/Overview/tabid/68/Default.aspx?PageContentID=23
We also have a large PDF explaining it which is free , you just need to be register on our site. Here is a link to register.
http://www.tradersstudio.com/UserRegistration/tabid/56/Default.aspx
You will receive a verification code to use the first time you log in. After that you can access this document as well as other tutorials ,videos ect.
Murray Ruggiero
Sponsor
No , you take the best parameters from each training period and use that for a walk forward window. Then you move the window forward and reoptimize and retest on the new windows.
Please read the PDF I have added links to above. You then need to combine the results of each out of sample window. This creates several issues on the boundaries of the windows. All this is explained in the PDF which I linked to in my earlier post.