Over the past couple of years, we (a small group) have developed an automated intraday trading program in C++ using the Genesis API, which has the ability to execute a given strategy in real-time (live) and test the strategy over historical data (backtesting).
One problem we have had for quite some time, and have not yet been able to solve is outlined below. For those who have experience with Genesis, E-signal historical data, or can shed any other light on this issue, your input is much appreciated. If you are interested in a more detailed discussion, feel free to PM me.
The problem: (bear with me, this is long)
We get our real-time tick data through the Genesis API, and build OHLC bars from this data to run our strategies. They might be 1, 3, or 5 minute bars, for example, it all depends on the strategy. We then execute our strategy based, partially based on these candles.
We cannot get historical data from Genesis, but we are able to get it from E-signal. We can get upto 6 months of 1 minute bars from them for any stock, so we use this data for our backtesting, since we can build any size bar from 1 minute bars. We realize that by using this data, our exit price may sometimes be a bit off, and we accept that, but our entries, if they are based on bar data only (and often they are), should be perfectly reproducible.
The problem is that we find that the bars we generate from the Genesis feed are always somewhat different from the E-signal historical data, so that if we test a day we just traded, the results sometimes match well, but other times the bars are just different enough that the backtesting trades differ greatly from the live trades, rendering the backtesting nearly useless.
We first suspected synchronization to be an issue, however we now use the ECN timestamp on each print to build our bars, so our local clock is not involved in the system any more.
We also suspected that the "filtering" of certain prints (out of the money, etc.) in the historical data may result in the differences, but we don't see BIG differences in a few spots, we often see VERY SLIGHT differences in almost every candle... almost as if the time window for each bar was shifted by a few seconds, which made us suspect synchronization in the first place.
So, the question is, has anyone experienced a similar situation? For those who use the Genesis API, what historical data source do you use and do you find that it matches your Genesis tick data perfectly? Do you do any sort of additional tick filtering? For those who use E-signal data for backtesting your system that operates on a non-E-signal data feed, do you find that your historical data matches your live data?
Any help would be much appreciated, and if anyone has other problems with a Genesis-based trading system, or anything else I can help with, I would be glad to offer my advice in return.
Thanks.
One problem we have had for quite some time, and have not yet been able to solve is outlined below. For those who have experience with Genesis, E-signal historical data, or can shed any other light on this issue, your input is much appreciated. If you are interested in a more detailed discussion, feel free to PM me.
The problem: (bear with me, this is long)
We get our real-time tick data through the Genesis API, and build OHLC bars from this data to run our strategies. They might be 1, 3, or 5 minute bars, for example, it all depends on the strategy. We then execute our strategy based, partially based on these candles.
We cannot get historical data from Genesis, but we are able to get it from E-signal. We can get upto 6 months of 1 minute bars from them for any stock, so we use this data for our backtesting, since we can build any size bar from 1 minute bars. We realize that by using this data, our exit price may sometimes be a bit off, and we accept that, but our entries, if they are based on bar data only (and often they are), should be perfectly reproducible.
The problem is that we find that the bars we generate from the Genesis feed are always somewhat different from the E-signal historical data, so that if we test a day we just traded, the results sometimes match well, but other times the bars are just different enough that the backtesting trades differ greatly from the live trades, rendering the backtesting nearly useless.
We first suspected synchronization to be an issue, however we now use the ECN timestamp on each print to build our bars, so our local clock is not involved in the system any more.
We also suspected that the "filtering" of certain prints (out of the money, etc.) in the historical data may result in the differences, but we don't see BIG differences in a few spots, we often see VERY SLIGHT differences in almost every candle... almost as if the time window for each bar was shifted by a few seconds, which made us suspect synchronization in the first place.
So, the question is, has anyone experienced a similar situation? For those who use the Genesis API, what historical data source do you use and do you find that it matches your Genesis tick data perfectly? Do you do any sort of additional tick filtering? For those who use E-signal data for backtesting your system that operates on a non-E-signal data feed, do you find that your historical data matches your live data?
Any help would be much appreciated, and if anyone has other problems with a Genesis-based trading system, or anything else I can help with, I would be glad to offer my advice in return.
Thanks.
