Intraday historical tick data

[rolando87]

Quote from WinstonTJ:

You can't even fit 6 weeks of an average stock on an XL spreadsheet


For $50 I'd rather see a trade go away than give away my tick data.

Thanks for your opinion, however, I'm not sure I understand what you mean by not being able to fit 6 weeks of an average stock, are you saying that the amount of data cells will be too huge for excel?

Also I didn't understand what you meant by ratehr seeing a trade go away.


Btw, for those searching for intraday data, I did some more research and came across this.

By far the best deal I've seen yet, for 159 bucks not including shipping for the dvd, you get about 1100 most popular stocks for the last roughly 8 years or so depending on the stock for any time frame 1 min or greater.

http://pitrading.com/intraday_ascii_data_stocks_edition.htm

 

[promagma]

As Bob mentioned your cheapest option might be to sign up for IQfeed for a month and download what you need. Although I have found that there are a lot of bad high/low values on the 1-minute data that need to be cleaned up (or ignored).

From the website -

30 calendar days of tick (includes pre-post market) and several years of 1-Minute history (Forex back to Feb 2005, Eminis back to Sept. 2005, Stock/Futures/Indexes back to May 2007) retrieval for charting and time & sales data

Or, open a Tradestation account, and scrape some data

 

[WinstonTJ]

Quote from rolando87:

Thanks for your opinion, however, I'm not sure I understand what you mean by not being able to fit 6 weeks of an average stock, are you saying that the amount of data cells will be too huge for excel?

Also I didn't understand what you meant by ratehr seeing a trade go away.


Btw, for those searching for intraday data, I did some more research and came across this.

By far the best deal I've seen yet, for 159 bucks not including shipping for the dvd, you get about 1100 most popular stocks for the last roughly 8 years or so depending on the stock for any time frame 1 min or greater.

http://pitrading.com/intraday_ascii_data_stocks_edition.htm

Yes, true, pure tick data will be too large even for Excel 2007 (which has a much higher row count per worksheet than 2003).

What I meant by "seeing a trade go away" is that my tick data is very valuable and I don't just give it out to anyone. I don't sell it to anyone because it’s against the terms of my agreement from my data provider - and - if someone does not want to work with me it’s easier/better IMO to walk away from a trade than to start giving out free tick data. By not giving it out it keeps the barrier to entry higher, reducing competition and increasing my edges in the market.

Not trying to be a jerk but if you want real tick data you need to pay to play. I have 3 years of S&P and just about every ETF on the market. It’s about 700 names total and it’s about 7TB worth of data. To put that in perspective with 1-min bars, you could get probably 20 years of 1-min bars for the entire NYSE and fit it into 3TB or less. 1-min bars don't give you much more than you can already get for free on Yahoo/Google Finance. If you can fit it into an excel spreadsheet or onto a DVD its probably not worth much.

For example, SPY is roughly 80mb per day in an Access database. Yes Access is a large format to keep tick data in but its easy to work with and smaller sections can be pulled into excel quickly. A simpler way to state it is that SPY has roughly 2.5 million ticks per day (in these recent VERY low volume days). An Excel 2007 spreadsheet only has 1,048,576 rows whereas 2003 only has 65,536 rows. You can't fit even one low volume day of SPY into an excel spreadsheet.

Why don't you just record from Google/Yahoo/Finviz (or your broker) for a few days?

 

[Rodney King]

Quote from WinstonTJ:

not giving it out it keeps the barrier to entry higher, reducing competition and increasing my edges

LOL. Yup, you're the only guy in the markets with a tick database.

 

[rolando87]

Quote from WinstonTJ:

Yes, true, pure tick data will be too large even for Excel 2007 (which has a much higher row count per worksheet than 2003).

What I meant by "seeing a trade go away" is that my tick data is very valuable and I don't just give it out to anyone. I don't sell it to anyone because it’s against the terms of my agreement from my data provider - and - if someone does not want to work with me it’s easier/better IMO to walk away from a trade than to start giving out free tick data. By not giving it out it keeps the barrier to entry higher, reducing competition and increasing my edges in the market.

Not trying to be a jerk but if you want real tick data you need to pay to play. I have 3 years of S&P and just about every ETF on the market. It’s about 700 names total and it’s about 7TB worth of data. To put that in perspective with 1-min bars, you could get probably 20 years of 1-min bars for the entire NYSE and fit it into 3TB or less. 1-min bars don't give you much more than you can already get for free on Yahoo/Google Finance. If you can fit it into an excel spreadsheet or onto a DVD its probably not worth much.

For example, SPY is roughly 80mb per day in an Access database. Yes Access is a large format to keep tick data in but its easy to work with and smaller sections can be pulled into excel quickly. A simpler way to state it is that SPY has roughly 2.5 million ticks per day (in these recent VERY low volume days). An Excel 2007 spreadsheet only has 1,048,576 rows whereas 2003 only has 65,536 rows. You can't fit even one low volume day of SPY into an excel spreadsheet.

Why don't you just record from Google/Yahoo/Finviz (or your broker) for a few days?

Thanks for that detailed explanation. Very good info.

In answer to your question, I need to know whether my gap theory really works and to find out i need to test over the last year or two not just the last few weeks. With google, i can get the data I need for the last month or so, recording the data in excel as it happens is another solution, however that would only be beneficial for backtesting purposes in the future from this date on. So for this task my only solution is to get historical 5 min bars.

 

[WinstonTJ]

Quote from Rodney King:

LOL. Yup, you're the only guy in the markets with a tick database.

I never said or even implied that. I do however know that I am one of very few people with clean 'scrubbed' data - none of those bad ticks, outside prints, etc. and I have EVERY sub-second tick none of the usual 100-300ms 'snapshot' stuff that many people record.

Quote from rolando87:

So for this task my only solution is to get historical 5 min bars.

That sounds like exactly what you need. Why over-pay for something you don't need? I was under the assumption you needed sub-second tick data. You can fit several years worth of 5-min bars in an excel 2007 spreadsheet. I sent you a PM about a few things, let me know how you make out.

Good luck.