Reliable data providers to download historical data five-minute data for S&P 500 stocks

@MetaG thank you for the response. I looked at the company. Looks like it has been launched recently. And as you mentioned, they have also recently reduced some free data. What is the reliability of the historical data in this?
Thank you for your in inputs
I think IEXcloud is a new platform that was once part of IEXtrading (Group founded 2012?). I've only used them for a few months for mostly intraday and some historical.

Autoit has good web automation but can break easily with page revisions. There are other web scrapers too but they'll have the same issue.
 
At this stage, I'm just looking for SP500 tickers 5 minute bars. Regular trading hours is good enough for now.
I'll PM you. I have the data you want and will trade for any of a couple of simple programming projects that I haven't had time to do myself. I have 10+ years fairly high quality current names only, 5 years lower quality but including delisted symbols. I have current but not historical SP500 constituents, so you'd have to come up with those yourself. First set includes all pre and post market, second runs 9:00 to 17:00 NYC time.


I know Python but I can pick up selenium and js also. I had used selenium a few years ago.
The JS is simple, you just have to hook all XHR calls by overwriting the prototypes for open and send. I have code for that, you just have to inject it into the target page with Selenium executeScript.

Python Selenium doesn't really work off the shelf for this type of project as the driver and browser are too tightly bound. Perhaps you can find a way around this, or open e.g. chomedriver.exe directly and send commands via its socket interface, or just use another language that supports multiple lightweight browsers per driver instance.
 
@Kevin Schmit
At this stage, I'm just looking for SP500 tickers 5 minute bars. Regular trading hours is good enough for now.
I know Python but I can pick up selenium and js also. I had used selenium a few years ago. Eventually point in time tickers will also be needed because of survivorship bias but for now just current SP500 tickers are sufficient.
Thanks for your response.

Hey rvsw,

Im currently facing the same problem of trying to find a database of intraday prices to feed them into a ML master thesis. Apparently this first step is way more complicated than originally expected ;).

Could you tell me which data vendor you chose in the end?

Thanks

Thomas
 
Wow they just added Websocket to get realtime trades in free plan as well. Hopefully they don't slap us with a ridiculous bill when they start to monetize lol
Thanks for the heads up. They'll certainly monetize at some point. They seem to be associated with Polygon.io (along with at least one other nominally unaffiliated site) so they'll probably follow the same monetization strategy that Polygon followed with its formerly free FX feed.

JSON is a spectacularly bad choice for a streaming feed format, but you can't argue with free.

Websocket API documentation is a little sketchy, maybe they'll flesh it out over time. Timestamp appears to be 10 digit (seconds resolution) vs 19 digit at parent site Polygon (nanos).
 
I am trying to get their 5 minute data and so far I have found issues with some SPY tickers where the data is only available from 2018 onwards. However for the most of the tickers the data seems to be there from 2003 onwards which is huge. Still need to validate and weed out inconsistencies though ! They do have after market data which causes some or these.
 
Last edited by a moderator:
...for most of the tickers the data seems to be there from 2003 onwards...
Check your code on that, you may be doing something wrong. Data appears to go back to 2000. Here is a a URL, using the API key that AndyM so graciously provided earlier in the thread, that returns 3 weeks of AAPL 5min bars from Sept 2000.

https://finnhub.io/api/v1/stock/can...800&from=968126400&token=bma06t7rh5rfd8vpvcpg

Of course the other possibility is that AndyM's API key is better than yours.

Also, a large number of SP500 tickers going back only to 2018 seems wrong too. Post a comma separated list of the missing tickers and I'll check them for you.
 
I think the API key is free for use. It allows for 60 requests/second which is more than plenty in my mind. But try fetching the data for there tickers prior to 2018/2017

PGR
IQV
CXO
USB
COST
ALGN
DLTR
EIX
ROP
ITW
DG
 
Back
Top