Market data lag

[topdog]

Depends on your feed. Most feeds provide a timestamp on the tick, sometimes with a precision of 1/1000th of a second, with the timestamp coming from the exchange.

If your feed is like that, sync your time as much as you can with a known precise time source, then compare the timestamp of the trade with the synced time on your computer. The difference is the lag.

Yes. I think it's mandatory that you have exchange generated timestamps. I still would calculate the deltas between timestamps and my h/w clock ticks. I think this will give me a better accuracy. The exact times aren't so important in this situation. It's more a question of lag.

 

[easymon1]

How do you check your real market data lag ?

You'll need a chronosynclastic infundibulum to obtain extreme accuracy and precision - oh, and a large budget.

https://en.wikipedia.org/wiki/Flash_Boys
"...the faster a market participant's computer system can receive and act on data, the better their edge, and opportunity to profit, with even nanoseconds making a difference."

The first chapter tells the story of a $300 million project from Spread Networks that was underway in mid-2009—the construction of an 827-mile (1,331 km) fiber-optic cable that cuts straight through mountains and rivers from Chicago to New Jersey—with the sole goal of reducing the transmission time for data from 17 to 13 milliseconds.[6] (The construction of the line was dramatized in the 2018 film The Hummingbird Project.)

 

[topdog]

You'll need a chronosynclastic infundibulum to obtain extreme accuracy and precision - oh, and a large budget.

I think you don't have to. You just have to look a time delta between 2 timestamped actions and compare the time with your h/w clock ticks.

For example: You will receive a bid change 12:22:20.150 and the next will come 12:22:20:160. Because the timestamps will come directly from the exchange you know the time spent between these actions is 10 ms. So you should receive the information within 10 ms if not then the excess time will be the lag.

You of course have to optimize your local h/w for the speed so the lag you produce by your local h/w is more or less irrelevant. There is many ways to do this but it's a different topic.

 

[NorgateData]

For example: You will receive a bid change 12:22:20.150 and the next will come 12:22:20:160. Because the timestamps will come directly from the exchange you know the time spent between these actions is 10 ms. So you should receive the information within 10 ms if not then the excess time will be the lag.

No that's not the correct definition of lag. That simply represents the time between quotes. For example, even if you're on the other side of the world, the time between quotes will be 10ms. You could be on the moon and the time between quotes will still be 10ms.

Assuming your PC's clock is correct and synchronized to a known good nearby NTP server, the lag is calculated as:
Your PC clock when you received it - exchange timestamp

This will determine all of the lag elements between the exchange and your PC.

Perhaps another way of thinking about "lag" is pretending you are on the moon and someone on earth sends you a radio signal. Assuming you have the greatest equipment available, even at the speed of light, it takes 1.3 seconds for that signal to reach you, so the lag is 1300ms. So a transmission sent at 12:20:20.150 will reach you at 12:20:21.450. A second transmission sent at 12:20:20.160 at 12:20:21.460.

You should also monitor for jitter (basically the variation in lag) indicating that some intermediate point is congested, especially at busy times (open and close).

 

[rb7]

I am asking this because I don't know how to do it.


Thank you.

You just need to compare the exchange time stamp on the trade or bid/ask message that you receive with the time at which you receive them.

 

[topdog]

No that's not the correct definition of lag. That simply represents the time between quotes. For example, even if you're on the other side of the world, the time between quotes will be 10ms. You could be on the moon and the time between quotes will still be 10ms.

Assuming your PC's clock is correct and synchronized to a known good nearby NTP server, the lag is calculated as:
Your PC clock when you received it - exchange timestamp

This will determine all of the lag elements between the exchange and your PC.

Perhaps another way of thinking about "lag" is pretending you are on the moon and someone on earth sends you a radio signal. Assuming you have the greatest equipment available, even at the speed of light, it takes 1.3 seconds for that signal to reach you, so the lag is 1300ms. So a transmission sent at 12:20:20.150 will reach you at 12:20:21.450. A second transmission sent at 12:20:20.160 at 12:20:21.460.

You should also monitor for jitter (basically the variation in lag) indicating that some intermediate point is congested, especially at busy times (open and close).

No.

In my case I am not interested in the lag produced by the internet or connections. I am interested in how much overhead will the data provider do. How good or bad is my data provider.

Then by knowing that the quotes must come in within the 10 ms is a crucial part of the validation. Also this is why I am colocated to minimizing the connection delays. My connection and processing overhead are minimized into a level where they are irrelevant.

 

[topdog]

You just need to compare the exchange time stamp on the trade or bid/ask message that you receive with the time at which you receive them.

Yes this can be done, but as I said it's better to do by comparing delta times. It will give you more accuracy. No need to sync your local h/w clock because you are counting it's ticks only.

 

[NorgateData]

Yes this can be done, but as I said it's better to do by comparing delta times. It will give you more accuracy. No need to sync your local h/w clock because you are counting it's ticks only.

I respectfully disagree.

You're examining jitter with your method. It's quite different to lag (latency).

See my post above about the moon analogy.

 

[topdog]

I respectfully disagree.

You're examining jitter with your method. It's quite different to lag (latency).

See my post above about the moon analogy.

No again.

Jitter is a variance of latencies.

I am not calculating that.

I am calculating how long does it take to receive quote OR trade prints from the exchange -> data provider overhead will change that. I of course accumulate calculations and then average the results to get the answer.

After examining and testing things I now understand that my method works 100% right.

 

[topdog]

Thank you very much for your answers. Was very helpful and helped me to find a solution to my question. Happy trading for all of you.