You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
BackTesting vs WalkForward
- Thread starter ButterMilk
- Start date
Quote from ButterMilk:
I hear allot about backtesting systems, including TV commercial from retail brokers. My experience, however, is that walkforward testing (with varying test periods) is significantly more effective than backtesting, when it come to consistence and profitability.
All comments and experiences would be appreciated.
I have used both with good results.
The issue with back testing is that you are prone to over optimization. Over optimization can produce a brittle system, one that fails when market conditions change slightly - as januson explained, you are curve fitting.
I have found that time-and-time-again, less than optimal settings produce a more stable trading system.
The Evaluation and Optimization of Trading Strategies by Robert Pardo is an excellent book that you should consider having in your library.
Quote from asynchronous:
The Evaluation and Optimization of Trading Strategies by Robert Pardo is an excellent book that you should consider having in your library.
Yeah, what he said...
A sytem that is back tested properly will hold up during walk forward analysis as it is tantamount to papertrading. A system that does not should be tossed. It's pretty simple.
A system that doesn't work after preliminary backtesting isn't likely to have robust parameters, or could have improperly sized optimization windows, etc.
I have the Pardo book, and it's pretty good, if pricey. I think it's pretty much the definitive work on the subject. What's interesting is that I basically came to the same conclusions as he does through trial and error.
I don't agree with his ratio of optimization to trading windows. But his idea of returns vs. "perfect profit" is interesting. Though I haven't incorporated it.
FYI - the in sample test period length should be a variable
www.automated-trading-system.com/walk-forward-testing/
www.automated-trading-system.com/walk-forward-testing/
Quote from iagainsti:
Your ignorance is showing, my friend. Your statement shows clearly that you do NOT understand out of sample testing, robust parameters, etc.
If you are curvefitting, then you're doing it wrong.
Curvefitting = Optimization !!
In my early days I actually believed in such crap, now I'm building on a system that simply just works without lagging indicators.
Quote from januson:
Curvefitting = Optimization !!
In my early days I actually believed in such crap, now I'm building on a system that simply just works without lagging indicators.
Back Testing;
Curve fitting does not necessarily mean optimization. The curve will exhibit an optimum fit over a data set if and only if you chose the optimum settings from your test.
The approach I have chosen, the approach I like for myself, consistently yields stable systems.
The results from my testing consistently run within 3% of real money trades.
Do what works best for you, but please realize that your solution is not the only successful approach to testing
Well, then, there's your problem, you were using lagging indicators.Quote from januson:
Curvefitting = Optimization !!
In my early days I actually believed in such crap, now I'm building on a system that simply just works without lagging indicators.
As for me, I use price data. Indicators are too subjective...