What do you look for when backtesting a system?

E

ej420

Hi everyone, I have been lurking on these forums for a while. Recently, I developed my own system for trading indeces ( I guess I could apply it to futures as well, I was just going with whatever data I have.)

What do you look for when backtesting a system? Are there some common pitfalls to look for before going online and risking real money with it?

I have backtested on data from the last 15-20 years, and my system consistently beats the market and has prartically no losing months. The Sharpe ratio computed using 10-day returns is consistently above 1 ( using 5% risk free return).

I have also accounted for reasonable slippage ( is 1/10th of a percent slippage reasonable for trading an index such as S&P 500)?

Are there any other lessons you have learned, when developing automated strategies, that you wish you knew when you were starting?

Thanks in advance, and good luck with your trading.
 
I look for a smooth upward account balance over time and I look at the win/loss ratio and the win %. I then model a monte-carlo style look at 100 different accounts with position sizing to see how the worst case account balance curve compares to the best case account balance curve to see if I might be in for any shocks.....

I don't model slippage currently due to a software problem, most orders are limit orders, but I look at the number of times the strat hits the stops and make a mental note of how many times I might experience slippage of a point or three...

It is a safe assumption that your system will never give as good results in realtime as in optimized backtesting, the real question becomes how much worse will it be, thus, if you have enough data, you want to backtest and optimize over some of the data and then test over some more of the data [out of sample testing] to see how they compare and then compare those sets of results to results from real time tests on a sim account...

I'm having the best coding success by collecting data and running the strategies live on a simulation account during market hours and changing the code and/or backtesting after hours. That way I can debug the code against a backtest and against a real market over a period of time and 1) wring out all the code problems 2) be ensured that the code that I finally backtest with will work and be the exact same code when I take it live...
 
My best piece of advice is to test your strategy on as many instruments as possible. You might find a few homeruns, but that doesn't mean it will hold up in the long run. Looking at 50 different charts should give a solid general picture.

The best thing I've found is to understand what went wrong with your strategy when it nosedives. You're most likely to save yourself from catastrophic losses if you know when NOT to trade.

As for measures I look at, I mainly care about Sharpe ratios and drawdowns. I obsess about creating the smoothest equity curve possible.

The only other thing is making sure to include commissions, spreads, slippage and all the other crap that hurts performance. I can't tell you how many glorious equity curves I've created only to realize that I forgot commissions. More often than not, the chart goes from lower left to upper right and changes to upper left to lower right.
 
You must log in or register to reply here.
Back
Top