scrape
Historical options data
- Thread starter Aquarians
- Start date
scrape
Matt_ORATS
Sponsor
Hi
We have special EliteTrader pricing. https://info.orats.com/elitetrader
Historical near end-of-day quotes back to 2007 via FTP are available for a special price of $399 per yearfor updating files daily plus only $399 one time fee for initial bulk download.
We offer a one-time and recurring download of our historical options quotes gathered 14 minutes before the close since 2007. These are available on all standard options (weekly, monthly, quarterly) for all US equity optional symbols.
Here's a sample day from 2017
We apply our smoothed market values (SMV) process to produce good greeks, theoretical values, and IVs.
Then I will continue working on my open source data-curation project: https://github.com/aquarians/Public/tree/main/Aquarians
So far the only goal I expose in the open source code is to ensure the accuracy of data. Horribly hard in it's own if you ask me, and that's only the beginning into a journey that's supposed to make money instead of taking them.
I had Orats when they were free with my broker. Getting Orats and LiveVol paid were glory days. ORats is worth it’s weight in gold!
fuck that tbh, I heard from several people they tried to test a hyp on cheap option data, after using cheap OD they used ORATS and other data providers and got completely different results to the cheap option.
Resurrecting this thread to ask @Matt_ORATS some detail about the method of collecting historical market data.
I'm interested in daily data, so one sample per day. The main problem I saw with my current historical market data provider is that there's no telling WHEN the sample was taken, specifically end of day. It's far more important for the data to be CONSISTENT rather than the specific moment of day when it was taken.
And for that, along with the sampled option quotes, I also need THE SPOT PRICE OF THE UNDERLIER at the moment of sampling. Checking "ORATS_SMV_Strikes_20170824.csv" (provided sample) I noticed that spot price is missing. There is a column in the CSV file named "spot_px" (which I assume it's the spot price of the underlier at the moment of sampling), but it's empty.
Now, that's not necessarily an insurmountable problem IF THE DATA IS CONSISTENT since one can infer the underlier price from the option quotes using put-call parity.
What do I mean by "consistent data"? Well, option prices are highly dependent on the spot underlier price, right? So if the spot jumps a lot, the options will also jump a lot. Worst case is when option prices on the same underlier are sampled AT DIFFERENT TIMES, ON WILDLY DIFFERENT UNDERLIER SPOT PRICES. I seen this countless times in my data (and also when working in professional finance). With the data I have, I seen plenty of cases where it's obvious some options of the same maturity and underlier were sampled at a certain underlier price and others at a different one. I can manually eliminate one or the other but it's a nightmare do process automatically - or trust any sort of backtest.
So when sampling options market data, one should make sure that options on a specific underlier are sampled AT THE SAME TIME with the underlier price, and the underlier doesn't move in the meantime. Because quotes arrive asynchronously, if the spot moves a lot, options can be all over the place. Therefore easiest is just WAIT TILL THE UNDERLIER PAUSES FOR SOME TIME, so option quotes stabilize. It's even easier for daily data, in a whole freaking day there must be a couple seconds when the darn stock doesn't move no more. This gives the options the chance to catch up and eventually align with the underlier price so there are no fake opportunities and violations of put-call parity.
But I've no idea how such as ORATS collect their data and strongly doubt they use the simple "wait till consistent" algo above.
I'm interested in daily data, so one sample per day. The main problem I saw with my current historical market data provider is that there's no telling WHEN the sample was taken, specifically end of day. It's far more important for the data to be CONSISTENT rather than the specific moment of day when it was taken.
And for that, along with the sampled option quotes, I also need THE SPOT PRICE OF THE UNDERLIER at the moment of sampling. Checking "ORATS_SMV_Strikes_20170824.csv" (provided sample) I noticed that spot price is missing. There is a column in the CSV file named "spot_px" (which I assume it's the spot price of the underlier at the moment of sampling), but it's empty.
Now, that's not necessarily an insurmountable problem IF THE DATA IS CONSISTENT since one can infer the underlier price from the option quotes using put-call parity.
What do I mean by "consistent data"? Well, option prices are highly dependent on the spot underlier price, right? So if the spot jumps a lot, the options will also jump a lot. Worst case is when option prices on the same underlier are sampled AT DIFFERENT TIMES, ON WILDLY DIFFERENT UNDERLIER SPOT PRICES. I seen this countless times in my data (and also when working in professional finance). With the data I have, I seen plenty of cases where it's obvious some options of the same maturity and underlier were sampled at a certain underlier price and others at a different one. I can manually eliminate one or the other but it's a nightmare do process automatically - or trust any sort of backtest.
So when sampling options market data, one should make sure that options on a specific underlier are sampled AT THE SAME TIME with the underlier price, and the underlier doesn't move in the meantime. Because quotes arrive asynchronously, if the spot moves a lot, options can be all over the place. Therefore easiest is just WAIT TILL THE UNDERLIER PAUSES FOR SOME TIME, so option quotes stabilize. It's even easier for daily data, in a whole freaking day there must be a couple seconds when the darn stock doesn't move no more. This gives the options the chance to catch up and eventually align with the underlier price so there are no fake opportunities and violations of put-call parity.
But I've no idea how such as ORATS collect their data and strongly doubt they use the simple "wait till consistent" algo above.
Matt_ORATS
Sponsor
Hi Aqua
We take a snapshot of all options prices and stock prices 14-minutes before the close. The stock price and options prices are taken at the same time. Moreover, we utilize a residual yield solving mechanism if for some reason the stock price is off from the options prices. The residual yield is mostly for estimating the hard to borrow and lining up the call and put IVs but can help if the stock price happens to be off at the moment the snapshot is obtained.
The spot price column is used for indexes. For all others use the stock price (stkPx). The stkPx for indexes is the implied futures price for that expiration using options prices to solve.
https://gyazo.com/3d0cf12f69007670c92d9fe40f7299da
-Matt
We take a snapshot of all options prices and stock prices 14-minutes before the close. The stock price and options prices are taken at the same time. Moreover, we utilize a residual yield solving mechanism if for some reason the stock price is off from the options prices. The residual yield is mostly for estimating the hard to borrow and lining up the call and put IVs but can help if the stock price happens to be off at the moment the snapshot is obtained.
The spot price column is used for indexes. For all others use the stock price (stkPx). The stkPx for indexes is the implied futures price for that expiration using options prices to solve.
https://gyazo.com/3d0cf12f69007670c92d9fe40f7299da
-Matt
Hi @Matt_ORATS
I've imported the data from your provided sample "ORATS_SMV_Strikes_20170824.csv" and ran sanity checks on it. I found quite a few cases of garbage data.
For comparison, I imported data from historicaloptiondata (my current historical market data provider) for the same day.
I'm attaching views for two cases:
- Arcelor Mittal (MT)
- MARKET VECTORS GOLD MINERS ETF (^GMQ)
In the pictures you can see the contents of the CSV file along with viewing the data in my software. In red it's garbage data.
In GMQ case both ORATS and HistoricalMarketData seem to be equally missing the point. Call prices must increase as they get more and more in the money with decreasing strike price. Both Orats and HData show blatant violations of call put parity in this case.
In MT case, HistoricalMarketData is more accurate! ORATS starts to show some strange strikes below 15, with put asks suddenly jumping up as if they were in the money, although they are far out of the money.
I've imported the data from your provided sample "ORATS_SMV_Strikes_20170824.csv" and ran sanity checks on it. I found quite a few cases of garbage data.
For comparison, I imported data from historicaloptiondata (my current historical market data provider) for the same day.
I'm attaching views for two cases:
- Arcelor Mittal (MT)
- MARKET VECTORS GOLD MINERS ETF (^GMQ)
In the pictures you can see the contents of the CSV file along with viewing the data in my software. In red it's garbage data.
In GMQ case both ORATS and HistoricalMarketData seem to be equally missing the point. Call prices must increase as they get more and more in the money with decreasing strike price. Both Orats and HData show blatant violations of call put parity in this case.
In MT case, HistoricalMarketData is more accurate! ORATS starts to show some strange strikes below 15, with put asks suddenly jumping up as if they were in the money, although they are far out of the money.
Last edited:
