Two backtesting questions

I

Inittowin

I searched and could not find these exact answers anywhere so I was hoping one of you could help me out. I am backtesting an automated trading program of my own design that I hired a programmer to create. I am backtesting over the S&P 500 and have been calculating 5 bp slippage. I have been told that 5 bp is reasonable for highly liquid stocks but to use 15 bp for illiquid stocks. Do any of you have any advice on what number to use? The scary thing to me is that I can show a healthy profit at 5bp, but usually a loss at 15bp.

Also, I am backtesing on 1 minute data from Reuters, but only have access to one years worth of data. If the program trades about 400 times per stock and shows a profit across all 500 (I have not tested on all 500 yet I am trying to optimize over 40 before I test on the entire S&P) any advice on whether or not it is possible to still be so overfit, because it is just one year of data, that my real results will be drastically different?

Any advice would be really appreciated.
 
You can get free 1 min data for US stocks from OpenTick. Data up to about three years back is more or less OK. Earlier is a bit ropey with some missing bits.

I have no idea what your system is, so I have no idea if it would be successfull during other periods, but markets do change - especially volatility, so it would seem to be prudent to test over other periods.
 
Thanks for the tip on OpenTick. I guess I also should have gone a bit into my strategy, it is a long/short system purely based on average daily trading range varied by the direction of the market and the sector.
 
Just a quick bump to see if anyone has any thoughts on how many basis points to use for slippage depending on the # of shares traded daily.
 
If you are using limit orders, slippage is actually zero. But there is another problem with limit orders. Some trades that are filled in testing, could not be filled in real trading for different reasons. These missed trades can be winning or losing and it's very difficult to evaluate the influence of missed trades on the final results.
 
Yea, that is a good point. The other thing to consider is that since all of my entry and exit points are based on the stock hitting certain prices, if I have limit orders on the books in some instances I would be creating the signals that I am using to determine whether to enter or exit a position.

I have another question for backtesting if someone could be so kind to answer I would appreciate it. I have been backtesting this strategy and I am seeing very different results across the different stocks that I am testing on. My profit varies from about 35% a year to 170% (this number includes a $.005 commission and a 5 basis point slippage). The question I have is in regards to a large amount of profit coming from one trade. In some instances 20% of my profit is coming from one trade, while in another it is almost 40%. Now the system would still be profitable without that trade obviously, but it would be much less so.

My system trades between 100-200 a year per stock and because of this I had planned on maintaining maybe 10 open positions at a time, but monitoring maybe 50 stocks so that I am in the market as often as possible. Does anyone here have any thoughts on what percentage of overall profit coming from one trade is enough to really raise a red flag and where I should stop trading? The advise so far has been helpful, anything else would be much appreciated.
 
I would disregard the big gains entirely. They may happen in actual trading, and you should regard them as a gift from the gods. If you can achieve 35% in actual trading, with no liquidity constraints, you may have an excellent system. Kick butt!
 
Thanks, I think that is probably best approach. I was hoping someone could give me a number of if its over 10% take it out, but anything that is less than 10% if it is really over only about 100 trades that were profitable, could be left in. I think that is probably the number I will use, 10%.

Also, yea 35% in real time would be awesome. Like I said my backtesting results are looking pretty good, but the profitability percentage is insanely high, between around 85% and 100% so I am trying to figure out it I missed anything and even if I didn't I doubt that I will be hitting 100% profitable trades for a year in real time, it is just way, way too unlikely. I think my slippage will likely be what kills me, when I use a 15 basis point slippage my profit moves more toward 5-10% which is obviously nothing to write home about. My programmer is still working on the real-time version so it is still unknown whether or not the positive results will continue.

Any other advice would be appreciated from people who have backtested their own trading systems.
 
Well, the main problem right now is you only have 1 year of data. Is the system optimized for that one year? If it is, you may be sad to find out it might not work outside of that area. In my testing, I have found 2007 to be more profitable than any other year with my stratgies.

Opentick data is very shotty. Make sure you chart out and go through every single trade to make sure the data isn't messed up. It sometimes requires a lot of 'massaging" to get the data right.
 
Do a search here on ET for "in-sample" and "out-of-sample" testing. The search selector is in the upper right corner of the screen. Search with and without the hyphens. You could also try "optimize" and "optimization" but they will probably return lots of extraneous hits. "Over-optimize" should be good. If you haven't used the search feature be sure to read the "Advanced Search" info just under the search entry field.

Jack
 
You must log in or register to reply here.
Back
Top