Quote from frostengine:
I generally trade with market orders in my automated system... This removes some of the missed trades you hit on.. but introduces more slippage... which sort of evens out....
However, there is more to missing trades than fills when comparing historical data to real time trading.... I can deal with missed fills... can be compensated much the way you have...
The problem is missed SIGNAL generation. This can occur from MANY sources. For instance... lets assume you buy 5 years worth of futures data from a high quality vendor.. So your data is very accurate.... very little bad ticks.. etc... You develop your strategy on this data.. Now you go live using say IB (IB is who I use)...
Well IB's data is not pristine like the data you purchased.. these VERY VERY subtle differences in the two data sets creates HUGE differences in what your automated strategy sees... If your trading using indicator values such as when indicator x > y... etc... you will run into problems.. These subtle data differences can cause indicator x to vary several percent. So with one set of data it took the entry or the exit there... but in the other it did not... You wouldn't believe how different the traders were when I compared data I captured live with IB to data I bought after the fact... It was nigh and day.. you would have thought I was trading 2 completely different strategies...
I noticed this even running 2 separate computers collecting data on 2 separate IB accounts. There was a very wide variation in results...
If I was ever to "backtest" again I would never do it on purchased data.... Backtesting on data I collected myself has "some" merit though. But purchased data is near USELESS in my opinion... So many things affect that data versus what you will collect on YOUR computer and YOUR network connection and YOUR platform...
Don't even start thinking about trading on a "time based bar" strategy using purchased data.. small delays.... time difference on your machine... the time of the day you start collecting data.. SO much stuff affects what that bar actually looks like and even the smallest changes affects what your strategy sees for indicator values...
One way around this is to use the same data feed for historical data (backtesting) and live data (execution). So far, I have found no one who offers better historical data than tradestation. At the same time, you can use a bridge to execute strategy code in IB. I use ninjatrader.
The work I'm doing right now is to migrate by strategy code from tradestation to ninjascript, and move to zen-fire live data feed. I heard Zen-fire is completely unfiltered, and tick-for-tick matches the OHLC bars that I backtest with in tradestation. This way I can isolate my execution platform to just ninjatrader, while remaining confident that the signals the TS model generates will be identical to the orders issued through ninjascript.
The worst possible thing you can do is use IB as a datafeed. At the least, it should only be used as a backup feed (nice option when you're using ninjatrader).
If the feed quality is the primary issue you were struggling with in your futures trading, I encourage you not to abandon the effort and consider the suggestions I've posted here.
rt