Whole-market options data snapshots intraday every 30 minutes

thecoder · Oct 29, 2021

persistence said:
They don't charge you for bandwidth, they charge you for the data itself.

So, do you mean the data is expensive b/c it's data, and not because of its size? I would say size matters very well... :-)

traider · Oct 29, 2021

You can't get it for less than $100. The option exchanges all want to charge for realtime. You should just open a brokerage account and use that data. That is the cheapest but they will throttle you.

rb7 · Oct 29, 2021

thecoder said:
Anybody here who has experience with https://www.activetick.com/activetick/contents/MarketDataServicesPricing.aspx ?
I wonder whether the "Basic Package" there for $49.95/month can be used for my said use case.
They do not mention snapshot data, but say "Maximum streaming subscriptions 500" for the Basic Package.

.
They DO mention snapshot, but maybe not the way you would like it to work. Usually, real-time subscriptions are sending snapshots when you initiate your subscription for real-time quotes. So, you would need to subscribe to real-time quote each 30 minutes, and then unsubscribe, then re-subscribe in 30 minutes, and so on.
If you want to receive data on all listed option strikes (which comes out to approx. 1,000,000 strikes), you would need, each 30 minutes, to send 1,000,000 real-time subscriptions, but without having more than 500 real-time subscriptions at a time.
It would go like (each 30 minutes):
Send 500 rt subscriptions,
Receive 500 snapshots (usually cumulative from the beginning of the trading day)
Send 500 rt un-subscriptions,
And repeat 2000 times...

TANSTAAFL.

Handle123 · Oct 29, 2021

Yes, the options data is unbelievable expensive, I bought historical data some years ago from cboe, cost more than my house.

Every consider Tradestation, you can program what you want on it.
Market Data | TradeStation

traider · Oct 29, 2021

rb7 said:
.
They DO mention snapshot, but maybe not the way you would like it to work. Usually, real-time subscriptions are sending snapshots when you initiate your subscription for real-time quotes. So, you would need to subscribe to real-time quote each 30 minutes, and then unsubscribe, then re-subscribe in 30 minutes, and so on.
If you want to receive data on all listed option strikes (which comes out to approx. 1,000,000 strikes), you would need, each 30 minutes, to send 1,000,000 real-time subscriptions, but without having more than 500 real-time subscriptions at a time.
It would go like (each 30 minutes):
Send 500 rt subscriptions,
Receive 500 snapshots (usually cumulative from the beginning of the trading day)
Send 500 rt un-subscriptions,
And repeat 2000 times...

TANSTAAFL.

Is 1 option chain counted as a whole or do each individual option count? I'm assuming you are talking about IB?

thecoder · Oct 29, 2021

Handle123 said:
Yes, the options data is unbelievable expensive, I bought historical data some years ago from cboe, cost more than my house.

Really insane prices for such IMO public data.

Every consider Tradestation, you can program what you want on it.
Market Data | TradeStation

As I understand it, the data is kept remote in their datacenter, but I need to have the data locally, and also store it in a local database for later use by backtester etc.

thecoder · Oct 29, 2021

I'm going to try the YahooFinance API for $9.99/month for 20,000 requests/month as offered here:
https://www.yahoofinanceapi.com/pricing
It looks like that a single request brings ALL options data of an expiration date for the underlying instrument, as one can try the API out here (the 2nd GET API function):
https://www.yahoofinanceapi.com/

If the above said is true, then it should be possible to get the whole data in about 4000 requests (but this way just the data for the current expiration date), but then the 20,000 requests/month is of course by far not enough; one rather needs the unlimited package ("Mega") for $259/month... :-(

Here's what it sent for ticker PROG w/o specifiying an expiration time (this has to be unix time format or so as its a long number; haven't studied the API doc yet...):

Code:

{
  "optionChain": {
    "result": [
      {
        "underlyingSymbol": "PROG",
        "expirationDates": [
          1637280000,
          1639699200,
          1642723200,
          1645142400,
          1649894400,
          1657843200,
          1663286400,
          1674172800,
          1705622400
        ],
        "strikes": [
          0.5,
          1,
          1.5,
          2,
          2.5,
          3,
          3.5,
          4,
          4.5,
          5,
          5.5
        ],
        "hasMiniOptions": false,
        "quote": {
          "language": "en-US",
          "region": "US",
          "quoteType": "EQUITY",
          "quoteSourceName": "Nasdaq Real Time Price",
          "triggerable": true,
          "tradeable": false,
          "currency": "USD",
          "marketState": "POST",
          "shortName": "Progenity, Inc.",
          "exchange": "NGM",
          "longName": "Progenity, Inc.",
          "messageBoardId": "finmb_256359845",
          "exchangeTimezoneName": "America/New_York",
          "exchangeTimezoneShortName": "EDT",
          "gmtOffSetMilliseconds": -14400000,
          "market": "us_market",
          "esgPopulated": false,
          "firstTradeDateMilliseconds": 1592573400000,
          "priceHint": 4,
          "postMarketChangePercent": 1.1111166,
          "postMarketTime": 1635546350,
          "postMarketPrice": 3.64,
          "postMarketChange": 0.0400002,
          "regularMarketChange": -0.26999998,
          "regularMarketChangePercent": -6.9767437,
          "regularMarketTime": 1635537603,
          "regularMarketPrice": 3.6,
          "regularMarketDayHigh": 3.92,
          "regularMarketDayRange": "3.465 - 3.92",
          "regularMarketDayLow": 3.465,
          "regularMarketVolume": 33324635,
          "regularMarketPreviousClose": 3.87,
          "bid": 3.56,
          "ask": 3.63,
          "bidSize": 11,
          "askSize": 29,
          "fullExchangeName": "NasdaqGM",
          "financialCurrency": "USD",
          "regularMarketOpen": 3.77,
          "averageDailyVolume3Month": 56888827,
          "averageDailyVolume10Day": 146744462,
          "fiftyTwoWeekLowChange": 2.9429998,
          "fiftyTwoWeekLowChangePercent": 4.4794517,
          "fiftyTwoWeekRange": "0.657 - 7.86",
          "fiftyTwoWeekHighChange": -4.26,
          "fiftyTwoWeekHighChangePercent": -0.54198474,
          "fiftyTwoWeekLow": 0.657,
          "fiftyTwoWeekHigh": 7.86,
          "earningsTimestamp": 1628798400,
          "earningsTimestampStart": 1636405200,
          "earningsTimestampEnd": 1636750800,
          "epsTrailingTwelveMonths": -4.284,
          "epsForward": -1.01,
          "epsCurrentYear": -2.36,
          "priceEpsCurrentYear": -1.5254238,
          "sharesOutstanding": 132740000,
          "bookValue": -1.782,
          "fiftyDayAverage": 1.7938889,
          "fiftyDayAverageChange": 1.806111,
          "fiftyDayAverageChangePercent": 1.0068132,
          "twoHundredDayAverage": 2.300702,
          "twoHundredDayAverageChange": 1.2992978,
          "twoHundredDayAverageChangePercent": 0.5647397,
          "marketCap": 477864000,
          "forwardPE": -3.5643563,
          "priceToBook": -2.020202,
          "sourceInterval": 15,
          "exchangeDataDelayedBy": 0,
          "ipoExpectedDate": "2020-06-19",
          "averageAnalystRating": "2.7 - Hold",
          "displayName": "Progenity",
          "symbol": "PROG"
        },
        "options": [
          {
            "expirationDate": 1637280000,
            "hasMiniOptions": false,
            "calls": [
              {
                "contractSymbol": "PROG211119C00000500",
                "strike": 0.5,
                "currency": "USD",
                "lastPrice": 3.1,
                "change": -0.20000005,
                "percentChange": -6.060608,
                "volume": 274,
                "openInterest": 397,
                "bid": 3,
                "ask": 3.2,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635528336,
                "impliedVolatility": 0.500005,
                "inTheMoney": true
              },
              {
                "contractSymbol": "PROG211119C00001000",
                "strike": 1,
                "currency": "USD",
                "lastPrice": 2.6,
                "change": -0.25,
                "percentChange": -8.77193,
                "volume": 74,
                "openInterest": 745,
                "bid": 2.55,
                "ask": 2.65,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635530886,
                "impliedVolatility": 0.500005,
                "inTheMoney": true
              },
              {
                "contractSymbol": "PROG211119C00001500",
                "strike": 1.5,
                "currency": "USD",
                "lastPrice": 2.05,
                "change": -0.29999995,
                "percentChange": -12.765956,
                "volume": 552,
                "openInterest": 4180,
                "bid": 2.05,
                "ask": 2.15,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635537384,
                "impliedVolatility": 0.500005,
                "inTheMoney": true
              },
              {
                "contractSymbol": "PROG211119C00002000",
                "strike": 2,
                "currency": "USD",
                "lastPrice": 1.65,
                "change": -0.25,
                "percentChange": -13.157895,
                "volume": 690,
                "openInterest": 13203,
                "bid": 1.55,
                "ask": 1.7,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635537584,
                "impliedVolatility": 1.5312523437499999,
                "inTheMoney": true
              },
              {
                "contractSymbol": "PROG211119C00002500",
                "strike": 2.5,
                "currency": "USD",
                "lastPrice": 1.33,
                "change": -0.27999997,
                "percentChange": -17.391302,
                "volume": 1020,
                "openInterest": 17236,
                "bid": 1.25,
                "ask": 1.35,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635537541,
                "impliedVolatility": 2.031254921875,
                "inTheMoney": true
              },
              {
                "contractSymbol": "PROG211119C00003000",
                "strike": 3,
                "currency": "USD",
                "lastPrice": 1.08,
                "change": -0.25,
                "percentChange": -18.796993,
                "volume": 5356,
                "openInterest": 39511,
                "bid": 1.05,
                "ask": 1.1,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635537554,
                "impliedVolatility": 2.3125042187499996,
                "inTheMoney": true
              },
              {
                "contractSymbol": "PROG211119C00003500",
                "strike": 3.5,
                "currency": "USD",
                "lastPrice": 0.9,
                "change": -0.20000005,
                "percentChange": -18.181822,
                "volume": 3392,
                "openInterest": 25643,
                "bid": 0.85,
                "ask": 0.9,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635537558,
                "impliedVolatility": 2.4062539843749997,
                "inTheMoney": true
              },
              {
                "contractSymbol": "PROG211119C00004000",
                "strike": 4,
                "currency": "USD",
                "lastPrice": 0.8,
                "change": -0.19999999,
                "percentChange": -19.999998,
                "volume": 10495,
                "openInterest": 27628,
                "bid": 0.75,
                "ask": 0.8,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635537587,
                "impliedVolatility": 2.6328159179687503,
                "inTheMoney": false
              },
              {
                "contractSymbol": "PROG211119C00004500",
                "strike": 4.5,
                "currency": "USD",
                "lastPrice": 0.65,
                "change": -0.20000005,
                "percentChange": -23.529417,
                "volume": 3095,
                "openInterest": 17618,
                "bid": 0.65,
                "ask": 0.7,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635537593,
                "impliedVolatility": 2.750003125,
                "inTheMoney": false
              },
              {
                "contractSymbol": "PROG211119C00005000",
                "strike": 5,
                "currency": "USD",
                "lastPrice": 0.6,
                "change": -0.19999999,
                "percentChange": -24.999998,
                "volume": 7981,
                "openInterest": 35026,
                "bid": 0.55,
                "ask": 0.6,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635537575,
                "impliedVolatility": 2.7890655273437504,
                "inTheMoney": false
              },
              {
                "contractSymbol": "PROG211119C00005500",
                "strike": 5.5,
                "currency": "USD",
                "lastPrice": 0.55,
                "change": -0.14999998,
                "percentChange": -21.428568,
                "volume": 4092,
                "openInterest": 18844,
                "bid": 0.5,
                "ask": 0.55,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635537585,
                "impliedVolatility": 2.9140652148437503,
                "inTheMoney": false
              }
            ],
            "puts": [
              {
                "contractSymbol": "PROG211119P00000500",
                "strike": 0.5,
                "currency": "USD",
                "lastPrice": 0.03,
                "change": -0.020000001,
                "percentChange": -40.000004,
                "volume": 4,
                "openInterest": 1765,
                "bid": 0,
                "ask": 0.05,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635532954,
                "impliedVolatility": 4.7500040624999995,
                "inTheMoney": false
              },
              {
                "contractSymbol": "PROG211119P00001000",
                "strike": 1,
                "currency": "USD",
                "lastPrice": 0.02,
                "change": -0.01,
                "percentChange": -33.333336,
                "volume": 19,
                "openInterest": 3308,
                "bid": 0,
                "ask": 0.05,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635520228,
                "impliedVolatility": 3.06250234375,
                "inTheMoney": false
              },
              {
                "contractSymbol": "PROG211119P00001500",
                "strike": 1.5,
                "currency": "USD",
                "lastPrice": 0.05,
                "change": 0,
                "percentChange": 0,
                "volume": 152,
                "openInterest": 3813,
                "bid": 0.05,
                "ask": 0.1,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635536826,
                "impliedVolatility": 2.81250296875,
                "inTheMoney": false
              },
              {
                "contractSymbol": "PROG211119P00002000",
                "strike": 2,
                "currency": "USD",
                "lastPrice": 0.14,
                "change": -0.010000005,
                "percentChange": -6.66667,
                "volume": 314,
                "openInterest": 9444,
                "bid": 0.1,
                "ask": 0.15,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635537359,
                "impliedVolatility": 2.3750040624999995,
                "inTheMoney": false
              },
              {
                "contractSymbol": "PROG211119P00002500",
                "strike": 2.5,
                "currency": "USD",
                "lastPrice": 0.3,
                "change": -0.049999982,
                "percentChange": -14.285709,
                "volume": 667,
                "openInterest": 7388,
                "bid": 0.3,
                "ask": 0.35,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635537558,
                "impliedVolatility": 2.5781285546875,
                "inTheMoney": false
              },
              {
                "contractSymbol": "PROG211119P00003000",
                "strike": 3,
                "currency": "USD",
                "lastPrice": 0.58,
                "change": -0.030000031,
                "percentChange": -4.918038,
                "volume": 1307,
                "openInterest": 7639,
                "bid": 0.55,
                "ask": 0.6,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635537494,
                "impliedVolatility": 2.6562533593750004,
                "inTheMoney": false
              },
              {
                "contractSymbol": "PROG211119P00003500",
                "strike": 3.5,
                "currency": "USD",
                "lastPrice": 0.9,
                "change": 0,
                "percentChange": 0,
                "volume": 2640,
                "openInterest": 2037,
                "bid": 0.9,
                "ask": 0.95,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635537559,
                "impliedVolatility": 2.8593778515625,
                "inTheMoney": false
              },
              {
                "contractSymbol": "PROG211119P00004000",
                "strike": 4,
                "currency": "USD",
                "lastPrice": 1.3,
                "change": 0,
                "percentChange": 0,
                "volume": 298,
                "openInterest": 1589,
                "bid": 1.25,
                "ask": 1.35,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635537445,
                "impliedVolatility": 2.99219001953125,
                "inTheMoney": true
              },
              {
                "contractSymbol": "PROG211119P00004500",
                "strike": 4.5,
                "currency": "USD",
                "lastPrice": 1.75,
                "change": 0.07000005,
                "percentChange": 4.16667,
                "volume": 12,
                "openInterest": 690,
                "bid": 1.65,
                "ask": 1.7,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635521525,
                "impliedVolatility": 3.031252421875,
                "inTheMoney": true
              },
              {
                "contractSymbol": "PROG211119P00005000",
                "strike": 5,
                "currency": "USD",
                "lastPrice": 2.1,
                "change": 0,
                "percentChange": 0,
                "volume": 12,
                "openInterest": 1696,
                "bid": 2.05,
                "ask": 2.2,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635537062,
                "impliedVolatility": 3.2148457128906243,
                "inTheMoney": true
              },
              {
                "contractSymbol": "PROG211119P00005500",
                "strike": 5.5,
                "currency": "USD",
                "lastPrice": 2.55,
                "change": -0.049999952,
                "percentChange": -1.9230751,
                "volume": 7,
                "openInterest": 978,
                "bid": 2.5,
                "ask": 2.65,
                "contractSize": "REGULAR",
                "expiration": 1637280000,
                "lastTradeDate": 1635535595,
                "impliedVolatility": 3.3437516406249994,
                "inTheMoney": true
              }
            ]
          }
        ]
      }
    ],
    "error": null
  }
}

It's practically the same data as on this web page as the web page surely uses the same API and database...:
https://finance.yahoo.com/quote/PROG/options?date=1637280000&p=PROG

Btw, in the JSON data above, the "impliedVolatility" value has to be multiplied by 100.

sandy_s · Oct 29, 2021

You get for what you pay...

Clean data is one of the most difficult things to maintain and get - when you can generate billions of tick simultaneously.

There is a reason for the cost that is charged.

Try getting cheap data and the issues you face with cleansing it.

Another alternative is find out a way yourself to get real time tick data for the entire market and start building your own archive - you will soon see the challenges and why there is such cost involved in it..

thecoder · Oct 29, 2021

I haven't tried out yet, but it would be interesting to know whether one can request also past (ie. historic) options data via the yfapi.

rb7 · Oct 30, 2021

traider said:
Is 1 option chain counted as a whole or do each individual option count? I'm assuming you are talking about IB?

It's usually one subscription per strike.

No, I'm not referring to IB specifically.