Evaluating data feeds - feedback please

W

WinstonTJ

I'm doing some due diligence on three equity feeds for a client:

Bloomberg
Thomson/Reuters
NxCore

Looking for feedback during high-volitality and high-trading-volume times. During times such as the flash crash did anyone experience system crashes or freezes due to any of these three data feeds? We will most probably choose two of the three names, the most robust for the primary and the second most robust for the secondary.

All sorts of funky stuff can happen during crazy times like bad symbols, dropped packets, etc. This is what I'm interested in. Did anyone experience system outages directly due to their data feeds??

Thx.

Feel free to PM me if you don't want to publicly state.
 
Quote from WinstonTJ:

I'm doing some due diligence on three equity feeds for a client:

Bloomberg
Thomson/Reuters
NxCore

Looking for feedback during high-volitality and high-trading-volume times. During times such as the flash crash did anyone experience system crashes or freezes due to any of these three data feeds? We will most probably choose two of the three names, the most robust for the primary and the second most robust for the secondary.

All sorts of funky stuff can happen during crazy times like bad symbols, dropped packets, etc. This is what I'm interested in. Did anyone experience system outages directly due to their data feeds??

Thx.

Feel free to PM me if you don't want to publicly state.
More...


By far, reuters is the best of all.
It all depends on which Reuters API you will use and if you will get the feed from a line handler or a P2Ps but regardless of the API, Reuters is your best choice.

UPA is the API they market to be at par with direct feed performance and in reality, besides direct feeds, it may be the best choice for low latency high throughput data when coupled with a line handler.
RFA 6+ OMM is second best.
I do not think they still support RFA 5 "marketfeed" but during the flash crash we did not have any issues with that product, but the number of subscriptions makes a big difference.

Unlike NxCore, both Bloomberg and Reuters are subscription based so a lot depends on the number of subscriptions and the number of dispatcher threads you have.

I have worked with Bloomberg server-API and Bloomberg B-Pipe.... I will not touch either again if I have a choice.

All 3 of the feeds you consider are on different leagues.... Didn't say if you are considering the high end Bloomberg or Reuters feeds. In that case you may also want to consider "NYSE superfeed" as it allows you to bypass the middle man and not require another vendor as backup solution, which has its own issues by itself.
 
Would be high end (B-Pipe, etc.)

The nice thing about a middle man for non-latency sensitive stuff is that someone else consolidates. NYSE Superfeed and other direct feeds are nice however you need to then subscribe on an exchange by exchange basis which this client does not need.

I deal with a lot of NYSE market making firms and I'm well aware of the pros and cons of direct exchange feeds.

In this case I'm curious about NxCore a bit. I have a few clients pulling the feed but most of them are recent subscribers (post flash crash) so I've never seen the compression and decompression react to major flashes like that day.
 
Quote from WinstonTJ:

Would be high end (B-Pipe, etc.)

The nice thing about a middle man for non-latency sensitive stuff is that someone else consolidates. NYSE Superfeed and other direct feeds are nice however you need to then subscribe on an exchange by exchange basis which this client does not need.

I deal with a lot of NYSE market making firms and I'm well aware of the pros and cons of direct exchange feeds.

In this case I'm curious about NxCore a bit. I have a few clients pulling the feed but most of them are recent subscribers (post flash crash) so I've never seen the compression and decompression react to major flashes like that day.
More...

Don't want to seem I am 'selling' NYSE products, but if it is affordable I would give it a second look. Just in case you do not have all the info, I am going over some high level stuff.

The feed is consolidated. That is what NYSE is trying to do with superfeed, a single solution, normalized, consolidated data. For level 1 you subscribe for CTA (Consolidated Tape Association) and UTP (Unlisted Trading Privilege). Those contain the consolidated pricing feeds of all us equity exchanges. Based on the symbology you can subscribe for the consolidated symbol or for exchange specific. For example IBM is the consolidated NBBO symbol and IBM.N the nyse symbol, IBM.P arca etc...
The exact same symbology as Reuters RICs.

As a matter of fact, reuters and bloomberg use the exact same feeds CTA & UTP that they normalize to their internal format before sending it to you, so when there is an outage on UTP feed for example, your reuters or bloomberg feed does not have data either. So by eliminating the vendor you are not loosing anything other than an extra point of failure.

Regarding consolidation on level 2, book data, there is a product from NYSE that does consolidation for you. It is called "superbook" and it has as input individual exchange feeds and output a single consolidated feed. To best of my knowledge neither bloomberg or reuters have something similar. You need to do book consolidation on your own. Reuters licenses a third party product for that though. Not sure if bloomberg ever caught up with book consolidation.

The only reason, in my opinion to go with bloomberg or reuters is the value added data and reference content they provide to enhance their feed.

I have worked with b-pipe, I wouldn't touch this product. PM me if you need more details on b-pipe, but some known history of the product is the following:
Bloomberg wanted to retire the legacy server-API so the new product was started as BMDS (Bloomberg Market Data Service). After not delivering, it was rebranded to PhatPipe. After PhatPipe could not deliver, it was rebranded to B-Pipe. B-Pipe did not work out either, so the product now has been rebranded to "Managed B-Pipe" which is a server-API upgrade.

Very interested to learn about NxCore as well. Please share any feedback if you evaluate it.
 
Quote from WinstonTJ:

In this case I'm curious about NxCore a bit. I have a few clients pulling the feed but most of them are recent subscribers (post flash crash) so I've never seen the compression and decompression react to major flashes like that day.
More...

Win- Can I ask what kind of latency is typical with NxCore? Am I correct that the closest server to NYC is in VA?
 
NxCore sales guys can be more specific however I believe that they currently have servers in a bunch of locations - including NYC and Chicago. They deliver every message but timestamps are every 25ms at the moment but they are working on 1ms resolution - but I'm not exactly sure on that deliverable.

I have two clients that pull B-Pipe so I'm well aware of that product. I'm also pulling about 5 NxCore feeds for clients.

I think I was confused on NYSE Superfeed so thanks Makis for clearing that up. The NYSE MM firms I deal with only pull NYSE listed quotes and then each firm pulls the rest of their "outside NYSE" data from B-Pipe.

In a shop with 15-20 bloomberg terminal users the B-Pipe is an excellent solution (for their needs) and I know how that behaved during high volatility and volume times but I don't know about NxCore. Very interested to hear real-world experiences.

#1 did anyone crash or go down during flash crash (or on November 7th 2008 when S&P opened limit-down)

#2 has the feed during crazy times always been solid and reliable or have there ever been strange anomalies in the feed?
 
I like NxCore as it provides a whole market feed in an efficient compressed transmission format. All messages are collected and databased for consistent post analysis... API is well documented with sample code available here: http://nanex.net/apidocs.html

I don't recall the flash crash days in particular but reading through their message boards indicate some type of outage. I suspect the market disconnects would affect them no differently than all other data providers as they all use the same sources CTA / Exchanges etc.

From what I understand NxCore is their institutional product and IQFeed is the retail subscription type product put out by Telvent DTN.

NxCore is especially efficient delivering voluminous options data. I believe the NxCore technology powers IQFeed, Kinetix and other 3rd party data remarketers including many broker platforms.

I think their whole market feed and API are the right approach depending on the sophistication of your client. Its not a platform per se but an API technology allowing programmed access to their market data feed servers.

A great combination might be using both NYSE superfeed and NxCore so you have redundant data sources.
 
NxCore fell behind during the flash crash, although it never crashed; it was a few minutes behind for about an hour, at least in my case, after which it caught up again and operated normally. I think this was due to a bug that they knew about at the time but hadn't fixed, as the data volume was unexpected, but should now be fixed.

Overall I've had very little downtime -- perhaps two other instances in the several years I've been a subscriber. The "receiver" application, which is Windows-based, seems ultra-robust and fault-tolerant, and I think it runs flawlessly under both wine and VMWare, for those who are so inclined.

I would not call NxCore an ultra-low-latency solution, as the data are collected by parent company DTN first, and are then sent out. But latency seems pretty decent, even with that extra transit.

All in all, I think NxCore is a fantastic product; it easily makes my top 5 of best engineered software products of any kind that I've used. Its automatic logging of all the data as a "tape", and the subsequent ability to play it back exactly as it played during the trading day, is great; and it achieves extraordinary compression, which is nice for both bandwidth use and archiving the "tape" files.

For what it's worth, it is also very easy to implement, in my experience.

OP, have you also considered Activ and QuantHouse? Those both looked pretty good to me, too, when I was investigating feeds.
 
Quote from Occam:

NxCore fell behind during the flash crash, although it never crashed; it was a few minutes behind for about an hour, at least in my case, after which it caught up again and operated normally. I think this was due to a bug that they knew about at the time but hadn't fixed, as the data volume was unexpected, but should now be fixed.

Overall I've had very little downtime -- perhaps two other instances in the several years I've been a subscriber. The "receiver" application, which is Windows-based, seems ultra-robust and fault-tolerant, and I think it runs flawlessly under both wine and VMWare, for those who are so inclined.

I would not call NxCore an ultra-low-latency solution, as the data are collected by parent company DTN first, and are then sent out. But latency seems pretty decent, even with that extra transit.

All in all, I think NxCore is a fantastic product; it easily makes my top 5 of best engineered software products of any kind that I've used. Its automatic logging of all the data as a "tape", and the subsequent ability to play it back exactly as it played during the trading day, is great; and it achieves extraordinary compression, which is nice for both bandwidth use and archiving the "tape" files.

For what it's worth, it is also very easy to implement, in my experience.

OP, have you also considered Activ and QuantHouse? Those both looked pretty good to me, too, when I was investigating feeds.
More...

Occam,
Do you know if NxCore ever fixed the exchange timestamp precision?
To my knowledge they only provide exchange timestamps with seconds precision.

It is kinda turnoff for a product in that league not to support milliseconds at the time exchanges are getting into microsecond time stamps. Their lower end sister product iqfeed already has millisecond exchange time stamps for few months now.

Have you heard of any plans fixing it?
 
Quote from Makis:


Do you know if NxCore ever fixed the exchange timestamp precision?
To my knowledge they only provide exchange timestamps with seconds precision.
More...

There is no ms resolution for exchange timestamps yet. They were saying towards the end of last year that it would be in place on Jan 1, but apparently higher-priority issues came up. I remain hopeful, though, as it does seem to be on their to-do list. But in practice, it's not a huge issue for me as their own servers put out timestamps at 25ms resolution, which more or less suffices for my purposes.
 
You must log in or register to reply here.
Back
Top