IB data feed

[dcraig]

Quote from simki:

There is also significantly difference between real time and backfill volume for quotetracker too. In some cases backfill is showing only half the vol shown at real time.

Thus so far ibcharts, multicharts, sierracharts,quotetracker does not have a work around to capture realtime volume accurately.

sigh..i would like to use IB feed since its free and faster... but looks like i need to subscribe to a feed vendor

You've got my attention here. Does the problem exhibit itself only on short time frames such as 1 minute or is it equally evident on all time frames. Also, what instrument (s) are you looking at ? I have my own charting software. I will take some screen shots and compare 'real time' and backfilled charts and post the results if I can find anything interesting. Will take a few hours though to gather the data.

 

[dcraig]

Here are two charts of the DAX futures. The first, attached to this post was obtained from the IB real time streaming data commenced about 5 minutes after the DAX open ie about 9:05 CET.

The second, in the next post, was obtained from backfill from IB. I can't see any significant difference in volume bars. The IB backfill and streaming datafeed agree closely. There does not seem to be a problem with IB's datafeed.

 

[dcraig]

And here is the chart derived from IB backfill.

For 1 minute bars, it is important that your PC clock be accurate as the IB feed does not time stamp messages. PC clocks are notoriously poor at time keeping. I synchronize my pc clocks with a network time server every 30 minutes. Drift is generally no more than 200 millisecs max if you do this.

 

[just21]

What software are you using to chart on the attached charts? thanks

 

[dcraig]

Quote from just21:

What software are you using to chart on the attached charts? thanks

My own written in Java using the JFreeChart library (modified). http://www.jfree.org

 

[j_medved]

Quote from simki:

There is also significantly difference between real time and backfill volume for quotetracker too. In some cases backfill is showing only half the vol shown at real time.

Thats not possible. QT is going by cumulative volume, which is exactly the same whether you backfill or get it from Level I. The granularity is different, but the VOLUME is the same. Can't be any other way. I think you are looking at something other than volume and calling it volume. # of quote records will be drastically different with backfill or with Level I data. But that has nothing to do with volume

 

[simki]

Quote from j_medved:

Thats not possible. QT is going by cumulative volume, which is exactly the same whether you backfill or get it from Level I. The granularity is different, but the VOLUME is the same. Can't be any other way. I think you are looking at something other than volume and calling it volume. # of quote records will be drastically different with backfill or with Level I data. But that has nothing to do with volume

I did a comparison,
1. Collect real time and write down the vol displayed
2. Clear all chart data, do a backfill

I am using 1 min on HSI futures
here's my results attached.

Initially i was comparing QuoteTracker real time with IB Charts backfill as the base as its easier to activate backfill via IB Charts. I think the problem may be with IB Charts backfill.

Comparing QuoteTracker realtime with QuoteTracker backfill.. guess the difference is not as bad.

 

[j_medved]

IB L I quotes do not have time stamps. Therefore, QT uses system time (internally adjusted). Backfill does have time on the data. The two times may not line up exactly. Therefore, backfill data might have some volume shifted to the next or previous minute relative to the L I data. If you look at the data, any additional volume in one minute is canceled out by the lack of approximately same amount of volume in the next minute. The larger time intervals you look at the less error you will see.

 

[simki]

Quote from j_medved:

IB L I quotes do not have time stamps. Therefore, QT uses system time (internally adjusted). Backfill does have time on the data. The two times may not line up exactly. Therefore, backfill data might have some volume shifted to the next or previous minute relative to the L I data. If you look at the data, any additional volume in one minute is canceled out by the lack of approximately same amount of volume in the next minute. The larger time intervals you look at the less error you will see.

Ok got it.

Thanks everyone for your responses.

 

[oneway]

IB Data is not a real data feed as others have said here.

First off understand that the "Ticks" are reported in two seperate events - one for price and one for size.

Further, a report or event is only fired if one of the prior paramters change, either price, type or size of the tick.

That is to say that you could have 10 trades at the same price and size, and you will only get the first event firing on your client.

Imagine the havoc that such a model creates with two asynchronous events fireing only on data changes in this scenario :

10 trades go off for different sizes but the same price(10 Size Event)s
10 trades go off for the same size and price (2 Events Fired,1 size, 1 price)
10 trades fo off for the same size but a different price (10 Price Events)
2 trades go off for different size and prices (4 Events Fired, 2 size, 2 price)

In this case we have 32 trades, and yet we have recorded 26 events (13 size events and 13 price events.

Further still, the tick data is not time stamped.

Attempting to use IB Data for tick analysis or time & sales is a futile excercise at best, and is exacerbated by the asynchronous nature of the data reporting.

Real tick data is delivered to a single delegate or event handler and is comprised of a single object or structure that contains all of the information for the tick event, no matter how redundent it may be.

As to aggregated volume, typically real data feeds will provide a method or event to listen to that continually updates the details of the symbol or contract as they change. Once the symbol has been subscribed to, the feed sends packets to the event when any parameter changes for the symbol - volume for instance, or high of the day.

There are rough and sloppy workarounds for the IB Data, that some use to draw charts and track data.

An example would be as some stated to synch your clock up and create your own internal bar or candle closing event, at which point the IB servers could be repolled for the historcal chart data, and then your bar or candle 'zero' than gets it new data from the tick data flowing in over the next bar period.

In this way your historical bar volume will be up to date.

However, any aggregated volume data you collected over the bar period is flawed to the point of being useless. Your trade count data is also useless for it is not real.

The bottom line is that IB Data is fine for display in grid style displays, but as data feed it is at best a poor mans solution.

The best use of IB Data in charts is to use a real feed for historical and real time updates, and then use the IB Data to reconcile that data - seeing as its the IB Data that ultimatley determines the execution prices your getting through IB.

Thats my opinion, and speak from a bit of experience.


Scott
Owner - IB TWS API Yahoo Forum