IB; Biggest Possible Improvement?

Should IB include all highs and lows in the data feed?

  • I'm an IB user and I don't care

    Votes: 31 25.2%

  • I'm an IB user and I really would like improved data

    Votes: 85 69.1%

  • I'm not an IB user and I wouldn't care even if I was

    Votes: 1 0.8%

  • I'm not an IB user but I would want it if I was going to use IB

    Votes: 6 4.9%
  • Total voters
    123
I'm with you Kiwi, IB should make it a priority to make sure their data is accurate.

There's no need to prove their data is incomplete, you can just put up pretty much any other data feed and you'll notice that some data doesn't come in through TWS.

For example, I also use Cybertrader, which I think delivers every tick (or at least it seems that way) and does it frequently faster. AND their platform takes up much less resources even with 15 charts open at once. CyberTrader has it's limitations though, which is no API and no universal accts (come on guys get with it, you'd rule the trading world if you did!).

So if Cyber can do it faster and cheaper (less resources) why can't IB?

Even if it takes more resources, I think the argument over resource hogging is lame, in this day and age with cheap computing power I'd take accurate data over resources anyday.

I'd hate to have to pay $150/month to Realtick so I can access accurate data through an API. Maybe if IB offered a accurate data subscription service for a reasonable amount it would be good for the "hard core" users. Although I would think everyone would want the most accurate data available, call me crazy.

While we wait from someone from IB to respond.....

Anyone have experience with MB Trading and their data feed?
 
though conceptually a good idea, in practice, this would be too problematic for IB to implement. Looking at this from the programming point of view:

1) High/low for charts is dependent on the charting package as well as user selected settings on the charts. You may be viewing 5 min charts. Joe Shmoe may be viewing 1 minute chart, and John Doe may have his set to 30 second candles. The datafeed has no way of knowing what chart frequency to send "all High/Low" values for. Some of this may be taken care of by assuming 1 minute frequency. That would take care of 1 min and higher charting. But then everyone using tick charts will complain about being left behind :)

2) As already mentioned, since there are no timestamps on the quotes, aligning the candles on the client to the time period that the server uses for determining high/low is impossible. Yes you can sync the time on the client every minute if you want, but that is not sufficient. Server clock may not be exactly the same, there may be communication delays both between server/TWS and TWS/client app. The only way to align minute periods between the client and server is if you add timestamps

3) if something is implemented for this, definitely do not add new fields. Just modifying the aggregation routine to always send the tick if its a "New High/Low" for the 1 minute period would do the trick without requiring any modifications on any of the client applications.

4) If anything is done, it cannot possibly be done without increasing CPU and bandwidth on both client and server. Think about it - you just closed one candle. Just started another one so the range has not been established yet. Almost every other tick that is coming in will be establishing a new low/high, so will have to be sent, where as before it may have been aggregated. The increase in amount of data sent will not be as much as sending every tick, but it will be an increase, and probably a significant one particularly at the start of every minute.

Hoi - comments?
 
Quote from j_medved:

Hoi - comments?
More...

Valid points, but it could be implemented without much overhead (that should be the intention).
Forget about bar-charts and their time-periods..that's not what we need (and the responsibility of the client-software)..but instead keep using the snapshot-feed.

"Simply" add 2 fields to the snapshot (only present in the data-stream when needed). One for "snapshot-high" and one for "snapshot-low", a snapshot is only 0.3 secs, and in 90% of all cases this very small period has no "shapshot-high" nor "shapshot-low" to fill (so no overhead to transport)...in 10% of all cases the High and/or Low is present, and can be send to TWS, which can use it for updating its own TWS-realtime-charts, and can be transported to the API.

This will not take any significant bandwidth, because the high/low will be send in compressed bits (not the value-as-a-double, but a tick difference with the LastPrice)...I know IB can do this, using only a few bits (not even a byte).

In this case the IB-server needs some CPU to keep track of the high-lows during the snapshot-cycle of 0.3 secs...I don't know how IB have build this snapshot-cycle, so I can not tell if this can be implemented without much resources. But it is worth to give it a try.
 
I guess they could create a new record for sending LAST+HIGH+LOW. Would have to always send it if the criteria for HIGH/LOW sending is met, even if LAST did not change from the previous quote
 
Quote from j_medved:

though conceptually a good idea, in practice, this would be too problematic for IB to implement. Looking at this from the programming point of view:

1) High/low for charts is dependent on the charting package as well as user selected settings on the charts. You may be viewing 5 min charts. Joe Shmoe may be viewing 1 minute chart, and John Doe may have his set to 30 second candles. The datafeed has no way of knowing what chart frequency to send "all High/Low" values for. Some of this may be taken care of by assuming 1 minute frequency. That would take care of 1 min and higher charting. But then everyone using tick charts will complain about being left behind :)

2) As already mentioned, since there are no timestamps on the quotes, aligning the candles on the client to the time period that the server uses for determining high/low is impossible. Yes you can sync the time on the client every minute if you want, but that is not sufficient. Server clock may not be exactly the same, there may be communication delays both between server/TWS and TWS/client app. The only way to align minute periods between the client and server is if you add timestamps

3) if something is implemented for this, definitely do not add new fields. Just modifying the aggregation routine to always send the tick if its a "New High/Low" for the 1 minute period would do the trick without requiring any modifications on any of the client applications.

4) If anything is done, it cannot possibly be done without increasing CPU and bandwidth on both client and server. Think about it - you just closed one candle. Just started another one so the range has not been established yet. Almost every other tick that is coming in will be establishing a new low/high, so will have to be sent, where as before it may have been aggregated. The increase in amount of data sent will not be as much as sending every tick, but it will be an increase, and probably a significant one particularly at the start of every minute.

Hoi - comments?
More...

Mr. Medved,

you don't understand the proposal. The proposal you are discussing is not the proposal which kiwi made.

Timestamps have nothing to do with the proposal. Aligning bar charts has nothing do with the proposal.

The most important thing is that the proposal will not consume more computing or bandwidth resources. The proposed method can easily be adjusted to consume exactly the same amount of resources as the current method, or a little bit more, or a little bit less, or a lot more or a lot less. The proposed method can easily be calibrated and adjusted to consume exactly the amount of resources available at any particular point in time.

The proposal is to include high and low ticks in each snapshot. Each snapshot covers a fraction of a second. The exact size of this fraction, the exact size of the snapshot interval, determines the amount of resources used. It will be inversely proportional to the length of the snapshot interval. The exact amount of resources needed can be adjusted simply by adjusting the exact length of the snapshot interval. If the low equals the high (and most of the time it will), then only one tick need be sent for that particular snapshot interval.

No new fields need to be added. No 3rd party software needs to be modified. No preferred bar chart time interval need be selected (like 1 minute, 5 minutes, etc.) Just send the high and low from each snapshot, let's say, for each 0.3 seconds, if that is the interval which fits the available resources.

This proposal will, for example, ensure that a chart built from the data will contain the day's hi and lo. IB data currently produce charts which often do not reach the day's hi or lo. This is only one example of the proposal's benefits. If we look at shorter intervals, let's say one hour, we will get the same benefit. Some charts might show a particular extreme tick in one bar, while other charts might show it the next bar, because we are not talking about aligning bar chart intervals. The important benefit is that no matter which one of the two bar chart intervals captures this extreme tick, this extreme tick WILL be captured and included on the chart, at least somewhere. The point is not to align bar intervals, but to be able to see other things, for example, support and resistance levels. Throwing away extreme ticks discards what many traders see as valuable info about S/R.
 
Actually let me rephrase that. I dont want a leaner TWS. I dont want TWS at all.


I put my details in another post. What IB could do to have the best of both worlds.


Quote from MajorUrsa:

I agree. All the bells and whistles should be extra front-end tools, just like the ones we use now from 3rd party supply. Even the order grid could be skipped because many use an external trader anyway.

Ursa..
More...
 
IB needs to extract the small part of their "execution api" from TWS.

Give this as a small jar/lib to all their traders who dont use TWS.

TWS can stand as a seperate application that also uses this same jar/lib.

In other words, TWS would just be one of many front end clients to the execution library.

I dont think they will do it, because from what I glean from their architecture, TWS is used as a hub for multiple connections to a single account.

They would have to change everything to allow multiple clients from the same user to connect directly to their servers.

Or come up with a master/slave configuration, where the first client which uses the execution jar file, allows connections from all the other front end clients.
 
Quote from j_medved:

jimrockford - disregard the stuff about minute timeframes - misread that.

Fields however would be needed to identify high/low within the snapshot. Refer to Hoi's post:
http://www.elitetrader.com/vb/showthread.php?s=&postid=1172670#post1172670
More...

No fields are needed. If a snapshot contains only one value, this represents the fact that the hi equals the lo for the time interval stretching between the previous snapshot and the current snapshot. If a snapshot contains two values, in either order, then this represents the fact that the greater of the two values is the hi, and the lesser of the two is the lo, for the time interval stretching between the previous snapshot and the current snapshot. If the hi occurs first, then it is transmitted before the lo for that snapshot. If the lo occurs first, then it is transmitted before the hi for that snapshot. No fields.

I disagree with Hoi's idea of representing price ticks as differences from previous price ticks. If only one value in the stream is lost, then all subsequent values will be screwed. It will require more resources to send prices instead of price changes, but it will be more robust and better able to cover from the inevitable disruptions and interruptions.
 
What you are talking about is sending the high/low as LAST if they exist. The problem with that is that it will break the way ticks are interpreted by many programs. IB sends each field separately. If LAST is in affect overloaded, it would change how various fields would have to be grouped together into a single quote. For example, does the volume that is sent get assigned to the quote with the LAST=HIGH, LAST=LOW or just LAST?

As for sending the high/low as a delta, well, that may be over complicating things a bit :)
 
You must log in or register to reply here.
Back
Top