Complete 24 hour bid/ask data?

P

prophet

I’m looking for 24 hour historical data for ES and NQ that contains:

* time, last, bid, ask, volume (plus best bid/ask sizes if possible)
* must show every change in the best bid/ask price, regardless of trades

The historical tick data from Qcharts and RealTick consists of trade-only ticks, where the last/bid/ask/volume is printed only when trades occur. I need data that shows every change in the best bid/ask regardless of trades. I have been logging this data myself using Medved Quotetracker and a custom app that connects to IB TWS. I have only collected two months so far, for ESM3 and NQM3.

I need several months of this data historically, and for other futures contracts. This must be 24 hour data. Gaps in price and time should be minimal. I have an excellent system which makes use of the specific features in this data (bid/ask changes and continuity), although I can still use ordinary 24-hour tick data with some performance penalty. I’m not sure if I should continue development with this special data or switch over to more commonly available data and deal with the performance penalty.

Can anyone suggest a source?
 
Quote from ArchAngel:

You must be looking at the wrong thing - the Qcharts ES/NQ data contains bid/ask changes
More...

Qcharts only prints the last/bid/ask/volume when a trade occurs. In between trades the bid/ask is still changing, which represents important information.

Look at this, but don't pay attention to the times. The times are different by a second. Look at the trade occurring at 1055. Qcharts reports a single tick at 1055 with a bid/ask of 1054/1054.5. In contrast, IB reports that the 1055 trade pushed the ask from 1054.5 to 1055. Then the bid increased to 1054.5, dropped back to 1054 and then back to 1054.5 again where a new trade took place. All this time, the ask stayed at 1055. Qcharts never shows the detail in these bid/ask changes, and the new ask of 1055 is not reported until the next trade.

Qcharts NQM3 on 3/28

(time last bid ask bidsize asksize volume)

09:30:11 1054.50 1054.00 1054.50 91 34 159
09:30:12 1054.50 1054.00 1054.50 91 14 162
09:30:12 1054.50 1054.00 1054.50 91 3 165
09:30:12 1055.00 1054.00 1054.50 91 3 212
09:30:14 1054.50 1054.50 1055.00 11 47 213
09:30:14 1054.50 1054.50 1055.00 9 45 214
09:30:14 1054.50 1054.50 1055.00 33 41 215

versus IB's data:

(time last bid ask volume)

09:30:11, 1054.50, 1054.00, 1054.50, 4601
09:30:13, 1054.50, 1054.00, 1054.50, 4750
09:30:13, 1054.50, 1054.00, 1054.50, 4756
09:30:13, 1054.50, 1054.00, 1054.50, 4759
09:30:13, 1055.00, 1054.00, 1055.00, 4761
09:30:14, 1055.00, 1054.50, 1055.00, 4761
09:30:14, 1055.00, 1054.00, 1055.00, 4761
09:30:15, 1055.00, 1054.50, 1055.00, 4761
09:30:16, 1054.50, 1054.50, 1055.00, 4762
09:30:16, 1054.50, 1054.50, 1055.00, 4763
09:30:17, 1054.50, 1054.50, 1055.00, 4764
09:30:17, 1054.50, 1054.50, 1055.00, 4782

Obviously either IB or Qcharts is reporting the wrong best bid/ask just following the trade at 1055. I'm leaning towards IB's data being correct, since a great many traders would be seeing bad executions if it were wrong.
 
Again, I think you're looking at the wrong thing, here's a quick snapshot off the current T&S for ES03M.

Looks like bid/ask changes are reported independent of trades to me:

05/07/2003, 16:08:38, 929.75, 753, Best bid
05/07/2003, 16:08:38, 930, 133, Best ask
05/07/2003, 16:08:38, 930, 183, Trade
05/07/2003, 16:08:39, 930, 753, Best bid
05/07/2003, 16:08:39, 930.25, 133, Best ask
05/07/2003, 16:08:39, 930, 67, Best bid
05/07/2003, 16:08:39, 930.25, 1, Trade
05/07/2003, 16:08:39, 930.25, 2, Trade
05/07/2003, 16:08:39, 930.25, 1, Trade
05/07/2003, 16:08:39, 930.25, 593, Best ask
05/07/2003, 16:08:39, 930, 2, Trade
05/07/2003, 16:08:39, 930, 73, Best bid
05/07/2003, 16:08:39, 930.25, 563, Best ask
05/07/2003, 16:08:39, 930.25, 570, Best ask
05/07/2003, 16:08:39, 930, 114, Best bid
05/07/2003, 16:08:39, 930, 121, Best bid
05/07/2003, 16:08:39, 930, 137, Best bid
05/07/2003, 16:08:39, 930, 4, Trade
05/07/2003, 16:08:39, 930, 133, Best bid
05/07/2003, 16:08:40, 930, 4, Trade
05/07/2003, 16:08:40, 930, 153, Best bid
05/07/2003, 16:08:40, 930.25, 1, Trade
05/07/2003, 16:08:40, 930, 4, Trade
05/07/2003, 16:08:40, 930.25, 2, Trade
05/07/2003, 16:08:40, 930, 149, Best bid
05/07/2003, 16:08:40, 930.25, 527, Best ask
05/07/2003, 16:08:40, 930, 100, Trade
05/07/2003, 16:08:40, 930, 15, Trade
 
Quote from ElectricSavant:

Interesting.
More...

The volume is different too. Qcharts reports a volume of 47 contracts for that trade at 1055.0, while IB reports only 2 contracts.

Here are some more differences, this time for the period of 9:30:00 to 15:31:15 on 3/28/2003:

Qcharts reports 18907 ticks and volume of 132298 contracts.

IB reports 16198 ticks and volume of 137533 contracts

So there are differences in the bid/ask, volume and tick reporting. What is wrong with this data? Do either of these data sources reflect the true last/bid/ask/volume at every point in time?
 
I needed to get some historical data....so I called up a friend at Refco...........He set be some OHLC for the ES going back to 1997. I assumed, that was good data.....I found out by accident that it, in fact, was 1 point off usually with the open and the high price...and then the low and close were also off by usually 3 ticks. This was just daily data!!!!!!


I am learning how to scalp....and I do not know if I will every figure it out....without good data to backtest with..

Michael B.
 
Quote from ArchAngel:

Again, I think you're looking at the wrong thing, here's a quick snapshot off the current T&S for ES03M.

Looks like bid/ask changes are reported independent of trades to me:

05/07/2003, 16:08:38, 929.75, 753, Best bid
05/07/2003, 16:08:38, 930, 133, Best ask
05/07/2003, 16:08:38, 930, 183, Trade
05/07/2003, 16:08:39, 930, 753, Best bid
...
05/07/2003, 16:08:40, 930, 15, Trade
More...

This is real-time collected data from Qcharts right? Indeed your data shows bid/ask changes independent of trades.

Seems like the historical Qcharts data is merging the Best bid, Best ask and Trade lines into single line and throwing away changes in the bid/ask between trades. My main concern is getting accurate historical data. What am I missing here?

Thanks!
 
With the E-minis I trust E-sig. I know from first hand trading....because my target hits right when e-sig trades through my price.....also my stops hit too!!!!!!

As far as open outcry....just for get about bid and ask

Michael B.
 
Would the CME have the most reliable historical quotes? Would it be complete with the excursions that the author of this thread is eluding to?

Michael B.
 
You must log in or register to reply here.
Back
Top