What is a long enough period for backtesting a breakout strategy for forex? It's an intraday strategy that enters about 10 days per month.
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
Backtesting
- Thread starter dunitlongpole
- Start date
Quote from Aaron:
not ones that have been optimized for profitability by adjusting your parameters or indicators.
I don't get this quote.
Isn't the whole point of backtesting strategies to deduct which strategies performed the best.
Let's say I'm testing a simple strategy. I want to buy 30 mins before the close and then sell MOC.
I run the test. Then I see that it may have been more profitable to buy 15 mins before the close but still sell MOC.
How does this effect the future trades I make 15 mins before the close? How is the new test statistically invalid?
Backtesting is all about optimization if anything at all. Not optimizing past parameters, you might as well just enter and exit the marker randomly.
Right?
Quote from Samspon:
He's saying the more trades the better to get an idea of how it's going to perform, but as many of possible should come from running the strategy on data that you did not use to optimize it on the first place. Otherwise you just end up curvefitting.
Do you mean optimize it on one market or stock and then backtest it for profitability on another?
I don't agree with this either. Every market performs differently in past and in the future.
Just because a strategy performs well in one commodity or stock, you can't expect the same pattern to exist in another.
And if it does exist in other markets, I don't think that increases the probablility of the pattern to hold in any or most of them
The point is you can look at any sample data, say last months data for a certain market, and come up with something that would have made 30 winning trades.
Now if you take this strategy and apply it to the month before, or the 2 months before, and still get the same ratio of wins, then this is more significant.
But if you take all the data you have, and use this to optimize your strategy, you just end up curvefitting it. You need to optimize it on one set of data, then backtest it on another. It can be the same market, just not from the same time period.
Now if you take this strategy and apply it to the month before, or the 2 months before, and still get the same ratio of wins, then this is more significant.
But if you take all the data you have, and use this to optimize your strategy, you just end up curvefitting it. You need to optimize it on one set of data, then backtest it on another. It can be the same market, just not from the same time period.
Quote from Aaron:
30 or more trades is enough to start to do statistics with. More trades = better statistics, though. But be sure these are out of sample trades -- not ones that have been optimized for profitability by adjusting your parameters or indicators.
Aaron Schindler
Schindler Trading
Aaron,
What about 91 trades in 2 months, or 467 trades in 12 months?
The system is based on indicators with no optimization.
Is it solid result statistically?
Cheers
Will
Quote from Willleung:
Aaron,
What about 91 trades in 2 months, or 467 trades in 12 months?
The system is based on indicators with no optimization.
Is it solid result statistically?
It depends on how profitable your system is on average relative to the variability in its profitability. If it was consistently quite profitable even after deducting transaction costs, you probably have a profitable trading strategy and you can proceed to begin trading it.
How profitable is "consistently quite" profitable? Teaching hypothesis testing is beyond the scope of this ET post, but that's what you should look up in an undergraduate statistics text. 91 or 467 out-of-sample trades is certainly enough data to calculate useful statistics from.
Here's a free online seminar I gave on quantitative methods for trading (registration required): http://www.cbot.com/cbot/pub/cont_detail/0,3206,1058+15785,00.html. Perhaps it will whet your appetite for learning more about hypothesis testing and how to apply it to your trading. The CBOT wanted me to apply my methods to their Dow Futures, but they can be applied to any market.
Aaron Schindler
Schindler Trading
Quote from Samspon:
But if you take all the data you have, and use this to optimize your strategy, you just end up curvefitting it. You need to optimize it on one set of data, then backtest it on another. It can be the same market, just not from the same time period.
Thanks, Samspon. Well said.
Only 5 posts, Sam... Although you are a newbie to ET, you sound like an experienced trader. What's your background? What do you trade? Welcome to ET!
Aaron Schindler
Schindler Trading