IB - data generation process

[jimrockford]

kostia00,

I think that these changes are very good. But why not also include, in each real-time snapshot, the highest and lowest prices which have occured since the fraction of a second of time since the previous snapshot? This way, charts would immediately reflect high and low values, without any 5 second delay. Charts already reflect the most recent prices, at the time of each snapshot, without any delay, and with a few snapshots happening each second; but the highest and lowest values occurring in between these snapshots will be delayed up to 5 seconds in the current plan.

By the way, I don't want newbie traders to be misled by my suggestion. The IB data feed is already superb, aside from occasional breakdowns and problems which IB is addressing through various planned improvements. The changes just announced by kostia00, to the design of the datafeed, are going to make it better by tweaking it slightly, even though it was already superb. I am suggesting yet another way to tweak it even a little better. Many newbie traders misinterpret these types of debates to indicate that IB data are somehow inferior or slow. Nothing could be further from the truth, in my opinion, and I know very many traders agree with me on this.

 

[kostia00]

Quote from jimrockford:

kostia00,

I think that these changes are very good. But why not also include, in each real-time snapshot, the highest and lowest prices which have occured since the fraction of a second of time since the previous snapshot?

Hi Jim!

I believe our main consideration is the network bandwidth usage. We would like to make sure we can keep up with the demand for this data before we commit to providing any higher-frequency high/low data. Thanks!

 

[giles117]

If the issue is network bandwidth usage, why not JUST get more bandwidth??????

 

[rwk]

Quote from giles117:
If the issue is network bandwidth usage, why not JUST get more bandwidth??????

It's customer bandwidth as well as IB's bandwidth.

 

[walterjennings]

Quote from kostia00:

Hi Jim!

I believe our main consideration is the network bandwidth usage. We would like to make sure we can keep up with the demand for this data before we commit to providing any higher-frequency high/low data. Thanks!

would IB ever consider offering a 'level 2' data connection which could be paid for. on its own fiber. where users who want/need every tick realtime as if we were directly connected to the exchanges? and are willing to pay for the improvement?

maybe also dealing with the msgs / second limit. im surprised that IB would limit an API using client to 50 (or 150) messages. Id imagine any money saved in bandwidth costs (if that is the reason for limiting messages) wouldn't offset the lost commissions from rejected / delayed orders. a few extra bytes of bandwidth for a possible 2.50+ commission seems like a good deal.

 

[giles117]

Quote from rwk:

It's customer bandwidth as well as IB's bandwidth.

He said nothing about customer bandwidth.

He stated it's about THEM keeping up with bandwidth. I have a 19.2mb/s (1.9 MB/s) down pipe, so customer badnwidth for this little packet data shouldnt be an issue.

I have a DTN.IQ feed and at most have 13-15 T&S windows open. (average 6-8) Never had a bandwidth issue there.

SO I am gonna lean on HIS words. So again I ask, if it is a bandwidth issue, why not get more bandwidth????

BTW, Personally it isnt an issue with me, My DTN data feed is my mainstay. As horrible as the TWS charts are...(IMO)

My question is one of understanding the mindset of with whom I am in business with.

 

[kostia00]

Quote from giles117:
My question is one of understanding the mindset of with whom I am in business with. [/B]

It is both internal network bandwidth required to deliver data from N exchanges to M distribution servers, and external network bandwidth to deliver from M distribution servers to customers.

The question of whether to get more of either or both is more than a technical question and will be examined by IB management: note that as we are not charging for the backfill and for 5-second bars (besides regular market data fees), we would like to keep the costs of these additional services under control.

 

[giles117]

Quote from kostia00:

as we are not charging for the backfill and for 5-second bars (besides regular market data fees), we would like to keep the costs of these additional services under control.

Sounds fair to me...

 

[originalskunk]

Quote from ddunbar:

It's not going to get much worse than that - 3 to 4 ticks off. And it's not as if it happens everyday. Will this affect trading? Doubtful, unless one is looking to scalp day's range extremes. Or if they're scalping off 1 min or tick charts towards the high and/or low of the day.

It depends on what instruments you trade. For "thick" futures like ES or ZN, it will be less of an issue than with YM, ZG or ER2.

 

[Cdntrader]

Quote from kostia00:

Starting with TWS version 866, the charts are receiving the 5-second bar data from the IB servers, and starting yesterday the calculation of the 5-second bars for US SMART stocks and GLOBEX futures have been changed to include every tick, so the TWS charts should never miss high or low of the day and the bar volume displayed should be 100% accurate. We are in the process of rolling this out to all exchanges and products.

We will be upgrading the API in the coming weeks and giving third-party applications access to the same real-time and accurate 5-second bar feed.

excellent!