Bars created from broker real-time data feed differ from bars created from eSignal

[abattia]

I have noticed that the intraday bars created from real-time data obtained via my broker’s data feed often differ from those re-created next day from eSignal data.

As a result, a "back simulation" of a given strategy over “historical” eSignal data will often show different trades from those that actually happened.

As far as possible, I would like to eliminate this difference between the two data sets. What can I do?

Does there exist a set of “absolute” data that is produced (and is available) in real-time?

Is eSignal data “absolute” in this sense?

Or will it always be the case that intraday data re-loaded the next day will differ from the data that was available in real-time?

 

[thstart]

Quote from abattia:

I have noticed that the intraday bars created from real-time data obtained via my broker’s data feed often differ from those re-created next day from eSignal data.

As a result, a "back simulation" of a given strategy over “historical” eSignal data will often show different trades from those that actually happened.

As far as possible, I would like to eliminate this difference between the two data sets. What can I do?

Does there exist a set of “absolute” data that is produced (and is available) in real-time?

Is eSignal data “absolute” in this sense?

Or will it always be the case that intraday data re-loaded the next day will differ from the data that was available in real-time?

HFT lead to Market segmentation. Meaning you have to get the data from place you trade - you broker, etc. - this is the most relevant place to get your data.

 

[abattia]

Quote from thstart:

HFT lead to Market segmentation. Meaning you have to get the data from place you trade - you broker, etc. - this is the most relevant place to get your data.

Thanks, thstart

Therefore, I guess backtests/optimizations should always be performed only on historical data produced from the same data source (i.e. in my case, from my broker)?

 

[thstart]

Quote from abattia:

Thanks, thstart

Therefore, I guess backtests/optimizations should always be performed only on historical data produced from the same data source (i.e. in my case, from my broker)?

Exactly! I tested several "independent" data feeds and can tell you - better get the data from the place you are trading.

Drop me a PM - I can tell you more about current situation.

 

[Fireplace]

I have found it's always better to choose the intraday data of an Esignal or a "pay" data feed over a brokers data feed in determining which bars are correct.

Quote from abattia:

I have noticed that the intraday bars created from real-time data obtained via my broker’s data feed often differ from those re-created next day from eSignal data.

 

[Arthur Deco]

Any strategy you develop that is sensitive to such differences won't work anyway. The issue isn't what the data is in the brokerage feed, but what the data is you'll make the trade decisions from, which is ESignal. If there ARE errors in ESignal, which I don't believe, then you want to back test including those errors.

 

[thstart]

Quote from Arthur Deco:

Any strategy you develop that is sensitive to such differences won't work anyway. The issue isn't what the data is in the brokerage feed, but what the data is you'll make the trade decisions from, which is ESignal. If there ARE errors in ESignal, which I don't believe, then you want to back test including those errors.

These are "abstract exercises" not corresponding to reality - you fit and tweak to "abstract" data in hope to find "dependency" and develop a "strategy" but when you try to apply this "strategy" to real trade, you have to place your order to "real" broker with "real" data which are different from the "abstract" neutral data.

Data are very important - they have to correspond to reality and not to be abstract.

The current market structure changed a lot of things one of it - the market data. It needs a new tools, new approach.

 

[Arthur Deco]

From what you just said it is clear that you do not system trade. You are a theoretician.

 

[abattia]

Quote from Arthur Deco:

Any strategy you develop that is sensitive to such differences won't work anyway. The issue isn't what the data is in the brokerage feed, but what the data is you'll make the trade decisions from, which is ESignal. If there ARE errors in ESignal, which I don't believe, then you want to back test including those errors.

Are there differences between realtime intraday eSignal data bars, and historical intraday eSignal data bars?

Do the realtime bars get "corrected" subsequently in any way?

 

[thstart]

Quote from Arthur Deco:

From what you just said it is clear that you do not system trade. You are a theoretician.

Mr. "Deco", please don't pollute the otherwise fruitful discussion with "speculations" about who is who. Obviously you have a lot of time which I don't. I know that trading is "boring" sometimes and from your "posts history" I see sometimes you amuse yourself.

There are threads in this forums I see very fruitful and this was one of it. If you have a "constructive" non personal thoughts you can share it, otherwise filtering is time consuming.

If data analysis is a "science", it follows "data" comes first. Data not corresponding to "reality" lead to erroneous conclusions (climate change data?).

By the way I just got a PM from the original poster which wanted to "know" and not to read "speculations".