Algo creation using tick data?

[HedgefunTrader2]

Quote from Steven.Davis:

The proposed chart-pattern on a one second chart with automated trading is quite feasible if the chart-pattern can be made fully objective.

Hi Steven, yes my chart pattern is fully objective. It's essentially a combination of bars and certain volume associated with how those bars form. To me, this is the part of the algo concept that seems the least daunting. My issue has been with trusting data. Admittedly, I've only had esignal and their data isn't consistent. I can pull up a 1 min chart and put up a 60 second chart right next to it and it will show different volume readings. The tick data vs the whole minute data is a major issue for me as my pattern doesn't always form only on whole minute data.

Theoretically, I'm not really sure why a 1 min chart and a 60 sec chart should show any differences. Does anyone know?

Thanks.

 

[Mr_You]

Quote from HedgefunTrader2:
Can anyone recommend a data provider that has clean tick data? I currently am an Esignal subscriber but I don't believe them to have pristine tick data.

You might check into using NinjaTrader along with Kinetick. Kinetick uses DTN/IQFeed as their data provider. You would be using C# to code the strategy. I'm looking to do something similar.

 

[eSignal Support]

Quote from HedgefunTrader2:

Hi Steven, yes my chart pattern is fully objective. It's essentially a combination of bars and certain volume associated with how those bars form. To me, this is the part of the algo concept that seems the least daunting. My issue has been with trusting data. Admittedly, I've only had esignal and their data isn't consistent. I can pull up a 1 min chart and put up a 60 second chart right next to it and it will show different volume readings. The tick data vs the whole minute data is a major issue for me as my pattern doesn't always form only on whole minute data.

Theoretically, I'm not really sure why a 1 min chart and a 60 sec chart should show any differences. Does anyone know?

Thanks.

I did a quick comparison of a 1 min vs a 60S chart of the DAX and every bar matches 100%. With slight variations in time-stamping, occasional differences might be spotted plus 1 min bars are built at the server level while 60S bars are built at the desktop. In general, they will match the majority of the time. If there is an issue, it would help to get a specific example so we can try to duplicate.

Thanks.

 

[uexkuell]

Quote from HedgefunTrader2:
My concern is that what I would need requires a program to follow the SPY second by second, and once my specific pattern appears, a trade is taken. Is that too complicated?

No, it's not really complicated.
Quote data is normally just ticks, not time bars or something the like.


Congrats.
In my view strategies based on tick data are the ones that are worthwhile.
No algo that is moving big amounts of money is based on e.g. 1 minute bars.

Quote from HedgefunTrader2:
Theoretically, I'm not really sure why a 1 min chart and a 60 sec chart should show any differences. Does anyone know?

They can differ because with some quote sources ticks don't carry time information.
Therefore they may be assigned arbitrarily to different time intervals depending on the algorithm.

One more reason why tick based algos are better:
They are more objective.

Quote from sma202:
To people who use tick data, my question is on how you select the appropriate timeframe for backtesting. Do you look at 1 day's worth of tick data or multiple days or just use several hours? If multiple days, it's a given that you will run out of memory soon enough. What are the approaches to backtest a few months of tick data?

Memory problems:
Not necessarily.
If you look e.g. at DAX futures you'll find 40k trades per day on average.
Assume 1 year worth of data: 200 days x 40k x 20Bytes/trade = 160 MB. If you have 2 GB main memory that's less than 10% memory usage.

Backtesting / timeframe:
It makes no sense to look 10 years back.
The advent of always smarter algos changes the trading environment pretty fast.
Though it certainly is helpful to look 1 or 2 years back in order to get a sufficient data base.

Backtesting in general:
For tick data you can do something better:
Calculate probabilities for the occurence of certain conditions.
This can give you a guidance in realtime.

 

[HedgefunTrader2]

For 8/2/2011, my 1st 1 min bar and the same 60 second bar have a different low (.80 vs .77) as well as different volume figures (for SPY, 9:30 EST). Check it out and let me know if you see these differences. I see them all the time, enough for it to be a big issue for my algo concept based on tick data.

Quote from eSignal Support:

I did a quick comparison of a 1 min vs a 60S chart of the DAX and every bar matches 100%. With slight variations in time-stamping, occasional differences might be spotted plus 1 min bars are built at the server level while 60S bars are built at the desktop. In general, they will match the majority of the time. If there is an issue, it would help to get a specific example so we can try to duplicate.

Thanks.

 

[gmst]

hedgefundtrader,


just a suggestion. sometimes your computer clock is few seconds different from your broker's/data provider clock. So, if 1 minute chart is getting formed in one place and 60s in different place, differences are to be expected. Hope it helps

 

[HedgefunTrader2]

Quote from gmst:

hedgefundtrader,


just a suggestion. sometimes your computer clock is few seconds different from your broker's/data provider clock. So, if 1 minute chart is getting formed in one place and 60s in different place, differences are to be expected. Hope it helps

Thanks GMST, it's definitely a possibility. But when I open and close esignal after market hours and compare data, I still see differences in volumes. Usually I find the OHLC for both whole minute and tick data associated with that whole minute to line up perfectly, but it's the volumes that tend to differ the most.

 

[HedgefunTrader2]

For the record, my intention is NOT to bash Esignal on this issue, I'm just looking for solutions.

 

[eSignal Support]

Quote from HedgefunTrader2:

For 8/2/2011, my 1st 1 min bar and the same 60 second bar have a different low (.80 vs .77) as well as different volume figures (for SPY, 9:30 EST). Check it out and let me know if you see these differences. I see them all the time, enough for it to be a big issue for my algo concept based on tick data.

We're taking a look. Seems to be related to the official marketcenter open prices and aggregate volume assigned to each one. The 60S range bar is using those values to calculate the bar where the 1 min interval does not appear to be. Not sure why quite yet but if theory holds true, this would most happen right at open or close, where as bars during the rest of the day match. Does that ring true in your experience? Any example you can find that counters the theory?

Thanks.

 

[HedgefunTrader2]

Quote from eSignal Support:

We're taking a look. Seems to be related to the official marketcenter open prices and aggregate volume assigned to each one. The 60S range bar is using those values to calculate the bar where the 1 min interval does not appear to be. Not sure why quite yet but if theory holds true, this would most happen right at open or close, where as bars during the rest of the day match. Does that ring true in your experience? Any example you can find that counters the theory?

Thanks.

Ok, I see what you mean about open/close. So here is an example from today (8/03/11) that isn't at the open or close. The 15min chart vs the 900s chart (same time frame, just whole vs tick data).

The 900s chart has the 10:30 and 10:45 bars as having increasing volume while the same two bars on the 15 min chart show volume decreasing from the 10:30 to 10:45 bars. All times are EST I am giving you.

900 sec chart has 18,778,263 and 18,830,688 (10:30 and 10:45 bars)
15 min chart has 19,069,570 and 18,124,934 (10:30 and 10:45 bars)