Looking for opinions on this system.

W

walterjennings

Hey, I've recently stumbled upon a system while hacking around in TradeStation. To me it seems fairly good (great?), but I have no background to truly put the stats TradeStation gives me in context. I was hoping a veteran system designer could look over my numbers and give me their honest opinion on it.

Attached is a TS Performance Summary ran over 5 years, trading 1000 shares of SPY. The system seems to do a single trade every few days. Which is a departure from my usual high'ish frequency trading, but results in a very nice per trade average profit (16 cents per share), which would result in slippage being much less of a factor. Though it comes off a bit as 'too good to be true'.

I've also uploaded a picture of the equity curve since that is normally how I gauge system quality.

http://i.imgur.com/epdVF.png

Thanks!
 

Attachments

One common problem is, as you sure know, curve fitting. On testing a new strategy, my first approach to check curve fitting sensitivity is to optimize parameters for a first period, e.g. two years, and then see how performance is in the following year(s) using these parameters. Sometimes I allow adjusting the parameters after each year, but only going forward. In most cases I think, good that I did not trade the system live.
 
Quote from Loki45:

One common problem is, as you sure know, curve fitting. On testing a new strategy, my first approach to check curve fitting sensitivity is to optimize parameters for a first period, e.g. two years, and then see how performance is in the following year(s) using these parameters. Sometimes I allow adjusting the parameters after each year, but only going forward. In most cases I think, good that I did not trade the system live.
More...

Hi Loki, thanks for your reply. Due to the fact that my approach is fairly simple, and wasn't optimized or tweaked much, and shows relatively consistent performance over long periods of time over varying market conditions, leads me to believe the probability of curve fitting is minimal.
 
Why don't you follow what Mark Douglas says and trade it for the next 30 trades and see how it turns out? That will give you the best indication of how it will work as a trade.

Try to figure out what mechanical aspect of the markets that this setup takes advantage of. Remember, we've been in a straight up market for the last 2 years, so you need to differentiate between those effects had on your system. If 2011 is flat or mildly down, how would your trade turn out?
 
Quote from jedwards:

Why don't you follow what Mark Douglas says and trade it for the next 30 trades and see how it turns out? That will give you the best indication of how it will work as a trade.

Try to figure out what mechanical aspect of the markets that this setup takes advantage of. Remember, we've been in a straight up market for the last 2 years, so you need to differentiate between those effects had on your system. If 2011 is flat or mildly down, how would your trade turn out?
More...

Thanks for the reply Jedwards. I've been trading it live since the start of Jan. Its been tracking the simulated results minus about $0.009~ per share traded. Which is not too bad seeing the average profit per trade is about $0.11 over the past year and $0.16 over the past 5 years. I was wondering if there was anything that stands out in the performance report as a classic trip-up for system designers.
 
Quote from walterjennings:

Thanks for the reply Jedwards. I've been trading it live since the start of Jan. Its been tracking the simulated results minus about $0.009~ per share traded. Which is not too bad seeing the average profit per trade is about $0.11 over the past year and $0.16 over the past 5 years. I was wondering if there was anything that stands out in the performance report as a classic trip-up for system designers.
More...

That's awesome! If that's the case, then don't look a gift horse in the mouth and keep trading it. If you understand the particular mechanics of the setup, then watch your trades and see if it starts to break down. I would set certain limits beforehand to determine when to suspend or shut down your system, as well. The true gutcheck of any system designer is toughing it through drawdowns. This is something I'm horrible at.

One thing you could do is go back as far as you can in terms of historical data and find a point in time where your setup didn't work, and see if you can identify why.

I have a system on the DAX that since October has completely broken down, giving me a huge drawdown that month and would have wiped me out if I didn't stop trading it. Backtesting it, it looked reliable going back for several years, so I had never seen 4 huge losing months in a row. Feb is the first month since October where I'm seeing profitable results so I might start trading it back again in March.
 
Quote from jedwards:

That's awesome! If that's the case, then don't look a gift horse in the mouth and keep trading it. If you understand the particular mechanics of the setup, then watch your trades and see if it starts to break down. I would set certain limits beforehand to determine when to suspend or shut down your system, as well. The true gutcheck of any system designer is toughing it through drawdowns. This is something I'm horrible at.

One thing you could do is go back as far as you can in terms of historical data and find a point in time where your setup didn't work, and see if you can identify why.

I have a system on the DAX that since October has completely broken down, giving me a huge drawdown that month and would have wiped me out if I didn't stop trading it. Backtesting it, it looked reliable going back for several years, so I had never seen 4 huge losing months in a row. Feb is the first month since October where I'm seeing profitable results so I might start trading it back again in March.
More...

How much larger was your Oct-Jan drawdown compared to the maximum you saw over the past few years of back testing?
 
Quote from walterjennings:

How much larger was your Oct-Jan drawdown compared to the maximum you saw over the past few years of back testing?
More...

Here are the backtested results from 2008 to 2009, in terms of DAX pts, 25 euros per pt. Sorry for the crappy formatting. The 3 columns on the right are wins, full losses and moved stops (a small loss).

2008-01-01 2008-01-31 -51.000000 8 6 8
2008-02-01 2008-02-29 56.500000 10 3 8
2008-03-01 2008-03-31 33.000000 8 2 9
2008-04-01 2008-04-30 -16.000000 7 6 9
2008-05-01 2008-05-31 29.500000 8 2 11
2008-06-01 2008-06-30 -22.000000 8 5 8
2008-07-01 2008-07-31 84.000000 14 4 5
2008-08-01 2008-08-31 -20.000000 6 6 9
2008-09-01 2008-09-30 -75.500000 7 7 8
2008-10-01 2008-10-31 2.000000 11 7 5
2008-11-01 2008-11-30 27.500000 9 3 8
2008-12-01 2008-12-31 73.000000 11 2 6
2009-01-01 2009-01-31 -25.500000 6 4 9
2009-02-01 2009-02-28 -4.000000 9 6 5
2009-03-01 2009-03-31 5.500000 11 7 4
2009-04-01 2009-04-30 85.000000 13 1 6
2009-05-01 2009-05-31 34.500000 8 3 9
2009-06-01 2009-06-30 76.000000 13 1 8
2009-07-01 2009-07-31 85.000000 15 2 6
2009-08-01 2009-08-31 43.000000 9 2 9
2009-09-01 2009-09-30 34.000000 11 3 8
2009-10-01 2009-10-31 -14.000000 8 6 8
2009-11-01 2009-11-30 8.000000 9 5 6
2009-12-01 2009-12-31 28.000000 10 4 6

The only really bad months were 1/2008 and 9/2008. I discounted 9/2008 because that was the month of the financial crisis, so the volatility was extremely huge and I don't know if I would have even traded then. The absolutely stellar results in 2009 really fooled me.

These are the results from 2010 on a per month basis:

2010-01-01 2010-01-31 -38.500000 6 7 7
2010-02-01 2010-02-28 19.000000 9 4 7
2010-03-01 2010-03-31 15.500000 10 1 12
2010-04-01 2010-04-30 18.000000 11 3 6
2010-05-01 2010-05-31 -26.500000 6 5 10
2010-06-01 2010-06-30 12.000000 6 2 14
2010-07-01 2010-07-31 20.500000 10 3 9
2010-08-01 2010-08-31 49.500000 14 4 4
2010-09-01 2010-09-30 -5.500000 7 4 11
2010-10-01 2010-10-31 -59.500000 2 7 12
2010-11-01 2010-11-30 -33.500000 6 6 10
2010-12-01 2010-12-31 -15.000000 7 6 8
2011-01-01 2011-01-31 -20.000000 4 3 11

I started trading the DAX in Jan 2010 after backtesting it for a while, so you can tell how my luck has been for 2010 :)
 
Quote from walterjennings:

Attached is a TS Performance Summary ran over 5 years, trading 1000 shares of SPY. The system seems to do a single trade every few days.
More...

Is this system trading on 1-min data? If it is, and you get one trade every few days, then it is not only fitted, it is royally fitted and filtered. You should have thousands of trades on 1-min data for this period. What is the point in trading 1-min data if you are going to get a trade every few days? The only purpose I can see is for giving room for optimizing the system and filtering out bad trades using all sorts of indicators.
 
You must log in or register to reply here.
Back
Top