Market Depth Data, Backtesting and Latency Implications for a Small Trader

I

IAS_LLC

Hello,

I have been working to stand up a C++ based automated/algorithmic trading platform for my own personal use that is based upon the Interactive Brokers API. I'm finally getting to the point where I am ready to start testing my FX Futures strategies (trading at as high of frequency as my technology and bankroll will allow). To give you an idea of how small of Trader I am..... I am planning on trading Globex E-Micro contracts to avoid over-leveraging myself. I am staring with $10k (I know I should have a lot more, but I don't have it right now, and I feel if I fit the strategy to my bankroll, I can be successfu regardless of my bankroll size).

Up until this point, I had always assumed that I would have to pay for the Market Depth Tick Data that I seek, but I guess I was naive to how much it actually costs.

For CME Historic Market Depth Data, It looks like it would be in the $1000 dollar/month range for 6 Contracts! This is unacceptable, given my small portfolio size. I'm still waiting on a quote for their real time service, but I am expecting a number that I can't afford.

It looks like it would be around $30/month if I were to use IBs real time market depth data, which is reasonable for me. If I go this route, I could just put my strategies on hold until I've collected enough data for backtesting.

I have heard IB's real time data isn't really "Real Time", and I want to get a feeling of the implications of using this "lower quality" data. Here are a list of questions I would like your insights on:

1. IBs API doesn't give millisecond level accuracy on its quote timestamps. Any recommendations on how to get around this?

2. What is the effect of using IBs futures market depthdata, which presumably just isnt updated quite as fast as the CME's real time data? Is it essentially just a higher likelihood of slippage?

3. When backtesting order book strategies where you are placing market orders, do you just simulate your fill at the bid/ask and monte carlo your expected slippage?

4. Can you recommend any other "reasonably" priced sources for futures tick level Order Book data? A means of hooking it up to my C++ program is a must.

I apologize if any of my questions are naive or if their solutions are common knowledge. I am a 25yr old control system engineer in the aerospace industry by profession, and all of my trading knowledge has come from reading books and trolling the internet. Anyway, thanks in advance for any insight you may be able to offer!

Thanks,
Brad
 
Brad, I'm a discretionary trader and, at the time I tried IB data, was trading off five minute charts and I found their data to be unacceptable. I believe you'll find that for your purposes it is almost or completely useless.

Don't make the mistake of believing that being undercapitalized is tantamount to failure. You are dealing with that weakness and working around it. Good for you and good luck.
 
I think NxCore is a good, "moderate" budget full feed solution (not sure how much maybe high-few hundred's a month for some full feeds, but don't quote me on that). They sell historical as well, that can be replayed the same way as the real-time stuff. Their default interface is in a Microsoft-flavored C++. Although starting with $10000 total might be a little light for that type of solution.
 
Hmmm, that is too bad. Right now IBs data seems like my best option (if my strategy involves the CME order book).

When you say that the data was unacceptable, what do you mean by that? Was your 5 minute chart just slow or was the data erroneous? From the link the gentleman above provided (I read all 8 pages), it sounded like the 200ms snapshot data that you can get through the API is accurate, but you may be aliasing price movements that happen in smaller time frames. If the data they send is accurate within the time i sent the request and received the info, that should be fine for me. Any price action within that time frame will be considered slippage, and my back testing will attempt to model that. If the data is erroneous or there latency is greater than the rate at which they are updating my feed..... that's where I would have an issue and could get into trouble. I need these equations to be true, or close to it.

TimeQuoteRecieved = TrueDataMeasurementTime + RequestTransmissionLag + MeasurementTransmissionLag

and

TrueDataMeasumentTime = QuoteRequestedTime +/- IbQuoteUpdateFreqeuncy


As for being under-capitalized, I know it definitely presents another challenge into an already difficult task, but I feel that if I recognize it, it can be dealt with successfully. I appreciate your encouragement.

Even this data latency business.. as long as I recognize it, and know what I am dealing with..... there is a way to win.
 
Quote from Occam:

I think NxCore is a good, "moderate" budget full feed solution (not sure how much maybe high-few hundred's a month for some full feeds, but don't quote me on that). They sell historical as well, that can be replayed the same way as the real-time stuff. Their default interface is in a Microsoft-flavored C++. Although starting with $10000 total might be a little light for that type of solution.
More...

I just sent a price list request to NxCore, I will let you all know what I find. Based on their website, their data feed seems to be exactly what I am looking for, other than the fact that it appears that it is a Windows application and all of my software is designed to run on a Linux box. That is something I can likely get around with a Windows emulator, such as wine, though.

Thanks!
 
To understand IB feed you have to know that it was designed to update quote board for large number of customers simultaneously and operate in all market conditions. Therefore it is not true "tick" feed but "delta" feed and on top of that it is sampled at a rate of 3 times per second or faster for certain future quotes.

It means that some trades/quotes are not recorded or delivered but it will provide you with updates if bid/ask changes.

I do not know much about DOM/book type of data since I consider them useless but I would suspect those are no better.

For many strategies IB "feed" is more than sufficient but for any low latency strategy it is useless.

On top of that you have to understand how market quote dissemination works; there is certain delay allowed for exchanges between trade/quote and data dissemination; add network hops to data provider plus feed data conditioning plus time to deliver it to you and if you are located somewhere in the country you are dealing with historical data between hundreds of ms to 1s regardless who is delivering the data to you.

If you are considering low latency then familiarize yourself with microstructure of the markets and plan accordingly.
 
Quote from vicirek:

To understand IB feed you have to know that it was designed to update quote board for large number of customers simultaneously and operate in all market conditions. Therefore it is not true "tick" feed but "delta" feed and on top of that it is sampled at a rate of 3 times per second or faster for certain future quotes.

It means that some trades/quotes are not recorded or delivered but it will provide you with updates if bid/ask changes.

I do not know much about DOM/book type of data since I consider them useless but I would suspect those are no better.

For many strategies IB "feed" is more than sufficient but for any low latency strategy it is useless.

On top of that you have to understand how market quote dissemination works; there is certain delay allowed for exchanges between trade/quote and data dissemination; add network hops to data provider plus feed data conditioning plus time to deliver it to you and if you are located somewhere in the country you are dealing with historical data between hundreds of ms to 1s regardless who is delivering the data to you.

If you are considering low latency then familiarize yourself with microstructure of the markets and plan accordingly.
More...

Thanks for the insight! Delta time should work for me, as long at the quote is accurate within the delta window.

I consider what I am going to try to do low latency, but I don't think the industry would. It certainly wouldn't be considered HFT by industry standards. My goal is not to be fast, I just want to be able to capitalize on price movements that occur in the several second to minutes time frame. Several quote/DOM updates per second, provided they are accurate given their resolution, should allow me to do that. I certainly don't want to be short the so called "Latency Option", which is why I am going to put a great deal of effort into conservatively modeling slippage.
 
Are you using actual traded volume or any other 'order flow' related quantity in your analysis? If yes, IB feed will not be sufficient. You can collect full market depth data on futures using IQFeed (lighter and cheaper version of Nanex). It will cost you about $100/month (with exchange fee waivers) plus about $300/year for their api (to be able to collect data).
I would say that if you don't use the actual information hidden in order flow (i.e. analyse transactions, bid/ask changes dynamics etc.) and thus essentially just go off of price, you will be fine with IB feed (or almost any other feed for that matter).
 
Quote from jelite:

Are you using actual traded volume or any other 'order flow' related quantity in your analysis? If yes, IB feed will not be sufficient. You can collect full market depth data on futures using IQFeed (lighter and cheaper version of Nanex). It will cost you about $100/month (with exchange fee waivers) plus about $300/year for their api (to be able to collect data).
I would say that if you don't use the actual information hidden in order flow (i.e. analyse transactions, bid/ask changes dynamics etc.) and thus essentially just go off of price, you will be fine with IB feed (or almost any other feed for that matter).
More...

I am using Bid/Ask size/price from the order book. Is this poor quality from IB? As far as traded volume goes, not so much. Although it would be nice to have a sense of the rate of change of the volume for momentum observation purposes. I may end up just trying the IB data feed and see how well I paper trade with that data. My biggest fear is the data being so bad, my system identifies correlations that exist in the data, but not in reality.



Sorry for the slow reply, I've been away from home for a few days....
 
You must log in or register to reply here.
Back
Top