Backtesting and Realistic Fills for Stocks

[maninjapan]

I am in the process of testing an intraday strategy trading Nasdaq 100 + SP500 stocks. The strategy uses limit orders for entries and profits, stop orders for Stop losses, all tested with tick data.
This is my first real foray into stocks and was after some advice on how to get my results as realistic as possible regarding fills and slippage on stop losses. It seems a lot harder to estimate a fill in stocks than it does in futures which I am more familiar with.

My first thought is to calculate an average bid/ask spread for each stock to estimate the slippage for stop losses, but not sure the best way to accurately identify executions.
With my limited experience with stocks to date ( trading with IB) I have seen numerous occasions where 'fills' occur past my own limit order without getting executed myself.
I'd like to try and have my backtesting reflect this difficultly in getting filled.
I have a couple of ideas that I plan to test, one being a minimum number of trades or shares traded past my stock price or a minimum distance based on the spread, or a combination of both but was hoping someone could offer some advice based on their own experiences.

 

[Deep Algorithms]

As you have seen firsthand, there is a very real issue of limit orders being filled. The standard way of handling this in backtesting is to set an amount by which price must be penetrated in order to consider the order filled. This may not be realistic though. While it helps ensure that limit orders that would actually be filled are, you may also miss out on filling a number of trades that would actually be filled.

One idea that I have thought about that may help to overcome the above problem is the following. Record all trades, and consider all limit orders to be filled as long as they are touched by price. Also record the amount by which the limit price is penetrated (if at all). You could then generate a number of possible trade lists, by using this amount to assign a probability to the order of being filled (e.g., a decaying function that goes to 0 at the limit price). You could also use this probability directly (e.g., in resampling, just consider it a trade with zero return). Of course, since you (well, at least I) don't know the (approximate) shape of this function, this may not be any more realistic than just setting a cutoff.

 

[ronblack]

With my limited experience with stocks to date ( trading with IB) I have seen numerous occasions where 'fills' occur past my own limit order without getting executed myself.

Can you provide an example of a limit order you placed with IB for which there were fills past the limit price but you did not get a fill?

 

[jharmon]

Generate yourself some signals. Be very precise with timing. Then get yourself a time & sales report with corresponding best bid/ask information. and see what you would have done. Some platforms allow for a feature called "Market Replay" whereby you can take a peek back in time and see what happened with regards to L2 data. I think CME also have some data in their "Data Mine" too.

 

[aqtrader]

I am in the process of testing an intraday strategy trading Nasdaq 100 + SP500 stocks. The strategy uses limit orders for entries and profits, stop orders for Stop losses, all tested with tick data.
This is my first real foray into stocks and was after some advice on how to get my
I'd like to try and have my backtesting reflect this difficultly in getting filled.
I have a couple of ideas that I plan to test, one being a minimum number of trades or shares traded past my stock price or a minimum distance based on the spread, or a combination of both but was hoping someone could offer some advice based on their own experiences.

I did backtest using intraday minutes data from google or yahoo instead of using tick data. The problem using any data in backtesting is in liquidity and out of band data that will eventually invalidate the backtesting results. So what I do is firstly to purify the intraday data by applying a conservative approach which basically removes all points that are too-good-to-be-true. Then, testing/trading only equity that have certain level of minimal volume to sure limit order get filled.

 

[maninjapan]

I did backtest using intraday minutes data from google or yahoo instead of using tick data. The problem using any data in backtesting is in liquidity and out of band data that will eventually invalidate the backtesting results. So what I do is firstly to purify the intraday data by applying a conservative approach which basically removes all points that are too-good-to-be-true. Then, testing/trading only equity that have certain level of minimal volume to sure limit order get filled.

Liquidty isn't an issue for me at the moment, my universe consists of SP500 stocks only at the moment with a minimum daily volume. I do agree that bad ticks can cause problems but hasn't caused me any major headaches to date ( I currently use data from Kinetick). Although I'm sure that if/when I branch out to less liquid stocks I will have to be a bit more careful with the data. My main issue is trying to replicate a realistic fill in my backtests.
My original logic was 1tick past the limit order = filled. This worked well enough with futures but not even close on stocks.
I often see executions a cent or so past my own limit order without getting filled not something I had to worry about on futures exchanges.
My next step is to start forward testing and see which of the execution methods are closest to my actual results.

 

[maninjapan]

Can you provide an example of a limit order you placed with IB for which there were fills past the limit price but you did not get a fill?

ron, I will put up a screen shot when I get a chance. Happens quite regularly though.
I currently use IB's smart routing.

Does anyone have experience with improving execution by rerouting their orders elsewhere?

 

[lindq]

My experience has been that if a retail system is so sensitive to fills as to cause you serious concern for overall profitability, then it may not be a system that is worth pursuing. If in doubt, you're always better off assuming a worst-case-scenario, and planning for it.

 

[promagma]

It's a hard question, and in real life it will also depend a lot on the liquidity of the stocks, the size of your order, and so on. You can take a deductive approach ... code and test two new versions of your system

A. Buy/sell at market when the stock hits your buy/sell point
B. Buy/sell at midpoint when the stock hits your buy/sell point

Whatever games you play with limit orders, live trading should not be worse than (A), and will probably never be better than (B) (because achieving midpoint or better, you will face stiff competition from HFT/MM/scalper algorithms). In my experience, you gotta be smart about order size and routing, and it's very hard to get closer to result (B) then to (A).