Backtesting with Interactive Brokers

christianhgross · Jul 16, 2011

Quote from LeeD:

However, it is totally misleading to claim that AmiBroker can get better (or more) data from Interactive Brokers than other charting tools.I guess the point of this rant is IB doesn't provide tick price history and the prices IB provides in real time are not suitable for reconstruction of 1-minute bars.

If you trade on, say, 5-minute time frame, all of this will have hardly any impact.

If you are relying on anybody's tick data to reconstruct bars then you are fooling yourself. I have used Bloomberg, IB, and several other data providers. They all have issues.

IB takes the best approach in that they provider 300 ms bars as that tends to filter the outlyer data. And if you can deal with 5 sec intervals you trade their 5 second interval bars, which reconstruct perfectly to the historical.

Rationalize · Jul 16, 2011

Quote from DeeDeeTwo:

...

It's ALL ABOUT EXECUTION anyway...
Your strategy is 10% of equation...
EXECUTION is 90% of equation.
...

Depends on the timeframe you're trading.

And that depends on the cost of lower latency.

And that determines whether it's worth capturing the spread at all.

If you're fast, your strategy can be simpler & more passive.

james2hunt · Jul 17, 2011

Well I have constructed 5min and 1min bars this weekend for 600+ stocks and found that I have a 97% match between IB TWS bars and those provided by my charting platform eSignal.

So for stocks NYSE/NASDAQ the data is good enough.

I wish people on this forum would give actual real statical data rather than just rant this is bad this is good e.t.c.

Give us some facts so that we know you have actually done back testing or do actually use these platforms or do have a programming background.

The HFT guys obviously cant use IB TWS as you need a platform that actually resides close to the mkt because of data latency.

And anyone who says back testing is a waste of time, how the hell do u test your code with real $$$ if so I know your talking BS.

Thanks
James

cloudy · Jul 17, 2011

You can demo Mirus /w Zenfire and sign up to the Mirus demo server with Ninjatrader. Then translate your algo to ninjascript and use Ninjatrader 7 to backtest for free.

Rationalize · Jul 17, 2011

Quote from james2hunt:

Well I have constructed 5min and 1min bars this weekend for 600+ stocks and found that I have a 97% match between IB TWS bars and those provided by my charting platform eSignal.

So for stocks NYSE/NASDAQ the data is good enough.

I wish people on this forum would give actual real statical data rather than just rant this is bad this is good e.t.c.

Give us some facts so that we know you have actually done back testing or do actually use these platforms or do have a programming background.

The HFT guys obviously cant use IB TWS as you need a platform that actually resides close to the mkt because of data latency.

And anyone who says back testing is a waste of time, how the hell do u test your code with real $$$ if so I know your talking BS.

Thanks
James

* I have coded against IB, as well as directly to an exchange or two.

Currently, HFT is not so much about data latency as network latency. Location is about physical network length & # devices. That's how perverse it has become.. Any retail trader who thinks they can compete by running similar strategies from home is .. wrong.

As for IB, I see about 500ms between sending an order, and receiving an ack. That's a lot. And, considering that delay, and adding whatever their data latency delay is, I think we're talking up to 1 second in latency.

So .. that moves IB based strategies away from HF, and toward something else.

LeeD · Jul 18, 2011

Quote from Engine99:

if you're looking for futures data go to this site
http://www.tradingsimulatorsoftware.com/dataindex.htm
I'm not sure how "current" their data is but it seems they offer decent tickdata for free

Given the history goes back to 2000 for many instruments, this is more history than most popular data providers offer. Do you know if there is an easy way to extract this data from files provided for use outside Trading Simulator platform?

LeeD · Jul 18, 2011

Quote from JackR:

Lee:

IB provides individual back contract futures data for a year, more or less. To see what's available without having to remember the symbol exactly, put a current futures contract in the TWS main window, then select "Previous Expiry" and repeat a few times. Then select Chart and if the data is available it will plot.

Most decent commercial charting packages can retrieve the data. It is not continuous contract data. You have to link it.

Jack

Thanks for the pointer! I will give it a try. Maybe there is a mess-up in my charting software with contract codes, such as using 1-digit year instead of 2 digits... still works fine for the current month.

Edit: I just checked it and my favourite charting package is indeed able to extract data for expired futures contracts from IB when contracts are specified individually. Perhaps, there is a problem with requesting data for expired constracts in the "continuous contract" algorithm.

Thanks JackR!

LeeD · Jul 18, 2011

Quote from christianhgross:

If you are relying on anybody's tick data to reconstruct bars then you are fooling yourself. I have used Bloomberg, IB, and several other data providers. They all have issues.

IB takes the best approach in that they provider 300 ms bars as that tends to filter the outlyer data. And if you can deal with 5 sec intervals you trade their 5 second interval bars, which reconstruct perfectly to the historical.

I agree there is no "ideal world". In particular, in an attempt to privide higher-quality historical data many providers "scrub" both historical "tick" and "bar" data in addition to filtering they do in real time.

The difference is just in how often there is a discrepancy... and with IB the problem is more severe than witha few other data providers:

1) IB real-time data is "sliced". The last trade price is sent 3 times a second or so as opposed to every tick. So, bars constructed from real time data have incorrect high and lows very often.

2) IB real-time data lack time stamp. As result, beginning/end of bar depends on how long it takes the data make way from IB to the trader's computer and on the trader's computer clock. So, open and close of a bar is off on most bars during periods of higher trading activity.

IB made a reasonable trade-off to make sure they can deliver data that doesn't lag during spikes of market activity without investing in infrastructure required to deliver reliably every tick. Given the price you can't really complain.

JackR · Jul 18, 2011

My belief is - If you are going to trade that data stream, test against that data stream (as best as can be done). Of course a lot depends on what type of trading you are doing.

Jack

LeeD · Jul 18, 2011

Quote from JackR:

My belief is - If you are going to trade that data stream, test against that data stream (as best as can be done). Of course a lot depends on what type of trading you are doing.

Jack

When forward-testing, yes absolutely. However, it's worth being aware that historical data from most data providers is different from their own real-time data.