Whole-market snapshot feeds

[_wd]

Nanex? He said the quote is out of his price range. He is trying to subscribe to ALL equities trades+quotes. For scanner, not market making or even trading. Floating point up to SIX decimal points should be the least of his concerns!

I'm going to play with Polygon's 1 second aggregate feed next week, it still looks like it could fit what I want. The complaints above are for the benefit of Polygon, for every user like me getting a bad first impression and wanting to run away, but is willing to report why that is, 100 more users simply ran away. So complaints like that ideally should be good for business

Regarding peak message rates, there is no way whole-market view of trades is going to be possible. For AAPL alone it peaks at 5000 trades/sec on Friday, and Friday was not even a busy day. That is over 512KB/s bandwidth peak just for one symbol, on a relatively quiet day. Even only S&P 500 would require at least a gigabit connection given Polygon's message format, so really the only option left is their 1 second aggs

 

[qlai]

Regarding peak message rates, there is no way whole-market view of trades is going to be possible.

They have customers who collocate and can handle the load.

 

[algoseek]

I

Then you can either try to scrape/collect the data export from Finviz Scanner (with paid subscription) - just export all the data without filtering, or filter the way you want.
Not sure if they won’t block you after too many exports, but once a minute should be ok.

Or try Polygon.io that streams all data like Nanex.

I agree. It's better to opt for providers such as Polygon, algoseek, Tickdata, as they offer unbundled services which is good for price-conscious individual.

 

[931]

I'm going to play with Polygon's 1 second aggregate feed next week, it still looks like it could fit what I want. The complaints above are for the benefit of Polygon, for every user like me getting a bad first impression and wanting to run away, but is willing to report why that is, 100 more users simply ran away. So complaints like that ideally should be good for business

Regarding peak message rates, there is no way whole-market view of trades is going to be possible. For AAPL alone it peaks at 5000 trades/sec on Friday, and Friday was not even a busy day. That is over 512KB/s bandwidth peak just for one symbol, on a relatively quiet day. Even only S&P 500 would require at least a gigabit connection given Polygon's message format, so really the only option left is their 1 second aggs

Hi, While testing did you find out if polygon.io-s socket based stock aggregate data feed has instrument count limit?
I cant find much info on that on the website.
For example how many instruments at sec or minute intervals streams possible?
@Polygon.io

 

[guru]

Hi, While testing did you find out if polygon.io-s socket based stock aggregate data feed has instrument count limit?
I cant find much info on that on the website.
For example how many instruments at sec or minute intervals streams possible?
@Polygon.io

Their website and API docs are pretty clear that they don’t have any limits. It’s unlimited.

 

[qlai]

It’s unlimited

I think I remember someone complaining that they were closing connection if you don’t keep up reading the data. So you may have to create multiple connections. I don’t think they have any limits on that either (many do). Strangely enough, I don’t hear many people recommending them.

 

[jabowery]

Last night, in response to:

curl "https://api.polygon.io/v2/snapshot/locale/us/markets/stocks/tickers?apiKey=yada yada"

I keep getting:

{"count":0,"status":"OK","tickers":[]}

This, after the initial invocation of that same curl command returned the expected snapshot JSON.

Is this expected behavior for some reason?

 

[Polygon.io]

Last night, in response to:

curl "https://api.polygon.io/v2/snapshot/locale/us/markets/stocks/tickers?apiKey=yada yada"

I keep getting:

{"count":0,"status":"OK","tickers":[]}

This, after the initial invocation of that same curl command returned the expected snapshot JSON.

Is this expected behavior for some reason?

This API resets every night at midnight. So from 12am-4am it will be empty.

 

[Polygon.io]

I think I remember someone complaining that they were closing connection if you don’t keep up reading the data. So you may have to create multiple connections. I don’t think they have any limits on that either (many do). Strangely enough, I don’t hear many people recommending them.

We have fixed the issue with WebSocket disconnects. It was an issue in our load balancers.

 

[qlai]

So what happens if I don’t read fast enough or at all? Also, as I just started experimenting with the api, it would be really nice to have some sort of keepalives going on web sockets.