Unreliable historical intraday data

Quote from I$land:


About limebrokerage and their API, how much software development was required ? The problem it seems you cannot get historical data but just current day data ? So getting 6 months of 5 min. bars from 20 companies would not be possible. Unless you are willing to wait 6 months to actually collect the data.
I$land
More...

It depends on what you want to do with the data. The API just gives you every quote/trade at every level of the book for the sources you subscribe to. It's completely up to you what you want to do from there.

For me, I've done quite a bit of development but if you just want the latest best price then it would be trivial to code in about 3 or 4 lines..

The API doesn't offer any historical info, but I don't believe that is something an API should offer anyway.. that is best handled by some more batch oriented process.

Also, you have to construct the bars, etc yourself if thats what you want to do.

The api is fairly elegant in its simplicity.. saves you the headache and overhead of developing to complicated APIs.
 
Quote from I$land:

I downloaded historical intraday (1 min., 5 min.) data from both ESignal and Prophet.net. To my big surprise every stock had missing bars ! Sometimes 1 bar in a day but some other times more than 20 bars in a day !

By the way, I determined this using a script I wrote in Wealth-Lab.

It is pretty hard to backtest any system if the data used is unreliable...

What data provider do you guys use to backtest your systems ?
More...
============

I$ land;

By the way I determined this also with ;
''getting my hands dirty with data'' also.

There is another public company [besides exchanges ] that sells ''cleansed'' data.

But not only is it unnecessarily expensive for individuals like me, would rather ,as my retired banker dad used to say,
'' don't be afraid to get your hands dirty'' with the data.

Can & have printed like 5 days intraday data/ save plenty of those weeks for years;
HPQ printer helps, but after you do plenty of that ,
cheaper & more helpful to ''get hands dirty with data''.

So prefer to simply record candle chart [or bars] info with blue/black/red ink on white notebook paper.:cool:[Do it before end of day] A human brain with discretion can ''cleanse '' focused prices also:cool:
 
tickdata.com is the best source for clean data that I have ever come across - the qcollector software wroks great - but the data it collects from (esignal or quote.com) leave alot to be desired - all the quotes might be there but the time stamps are off
 
The problem with spending all this money to buy all this scrubbed data for testing is that real world trading results are unlikely to resemble results obtained during backtesting. Market data is dirty. If the filtering is not fully disclosed and/or suitable to be applied to real time data then you may be setting yourself up for a lot of duplicated effort.
 
syswizard :

Thank you for the link but I only trade stocks.

trader-rachel :

QCollector + ESignal : just made some tests and I must say the results are not good. There are a lot of missing bars. I made a test program and here is the output for RIMM (May 2005 to Feb. 2006, 5 min bars, reg. trading hours) :

20051007 : BarCount = 46 Some bars are missing !
20051021 : BarCount = 53 Some bars are missing !
20051026 : BarCount = 65 Some bars are missing !
20051125 : BarCount = 42 Some bars are missing !
20051130 : BarCount = 48 Some bars are missing !

In a day, there are 78 5-min bars during the regular trading hours. For RIMM, on the worse day (2005/11/25), there were only 42 bars !

tomhaden :

Data CD : great link, thanks ! I downloaded a sample from their site and I will test it soon to see if their data is reliable.

originalskunk :

Agreed, but what do you suggest we do ?

--

Is anyone with TradeStation ? I would love to test their data.
If someone sends me data (RIMM, PLAY, 5 min bars May 2005-Feb. 2006), I will post the results here.

Why pay for historical data if it is not reliable ?
Anyone working for ESignal or Prophet.net can explain this ?

Thanks !

I$land
 
Has anyone tested the historical data available from IB? I know they offer up to 6 months of 1sec data, but I have not checked to see how accurate or reliable it is.

-- Paccc
 
You must log in or register to reply here.
Back
Top