I agree with your points. I think my main focus at this point is futures, particularly CME and CFE, so maybe it would be worthwhile to buy data from Nanex on only these two. I love that the nanex data gives me just a no hassle way to replay the entire book and I trust their data a lot more than data I would database myself. This is actually something I've considered in the past; especially because I believe they give a 10% discount for a year+ worth of data, so I could've save myself a year + of feed listening and potential headaches that would accompany it. My biggest concern with them though (as I've detailed to @IAS_LLC) is the 25 millisecond stamps. 25 milliseconds is low, but there could be hundreds/thousands of messages in 25 milliseconds, so I just wonder how much closer that gets me to a realistic Backtest than trade only millisecond data from iqFeed. I don't even really know what a realistic 'delta' is for the time it takes me to place a trade. Any feedback from you folks about which would be better? Maybe I could combine the two datasets somehow? Also, I'm sorry, but I can't really tell you what my strategies require, I may be putting the horse before the carriage in that sense. I'm admittedly guilty of just obsessing over data quality and have neglected to really move past it, but I also feel like there are probably worse things to obsess over.
Recording live feeds
- Thread starter cjbuckley4
- Start date
The 25ms time stamps are annoying, but not a deal breaker for me.
1. I cant get executed that fast anyway (maybe someday)
2. The feed does provide exchange sequence numbers, so in theory you could correlate that to achieve sub 25ms time reference. I dont do this, but i think its probably possible
1. I cant get executed that fast anyway (maybe someday)
2. The feed does provide exchange sequence numbers, so in theory you could correlate that to achieve sub 25ms time reference. I dont do this, but i think its probably possible
That's really helpful info. I highly doubt I'll be anywhere even in the 25 realm, but still I feel like that's just something rather basic when you pay this amount for data. I'll probably shoot them an email RE: data pricing Monday. I'm hoping they cover future options also, because if that's the case they've solved a huge logistical problem Ive had with 'rolling' options that I've discussed on other threads.
Last edited:
I read somewhere that they are working a version with 1ms stamps, apparently its a sizeable software change. They do offer futures options. When u buy from them, u purchase an entire exhange (NYMEX,CME,etc) so it has everything. The downside of this is you arent eligible for the globex fee waiver (for real time service) because it also contains data from the trading floor.
That's what I expected given that they trade on the same exchange. I just was skeptical because I know they charge like 600/month for options, but I believe that may be for OPRA only...you'd know better than I do. I only need historical data right now, but if, at some point, NxCore really does come out with a linux and millisecond resolution feed, that would be really good for those of us who are interested in trading from a linux environment. Maybe with .NET 2015 we'll see more APIs for linux happen, although I have no idea how NxCore's particular feed works...pure optimistic conjecture. Even with the 25 ms stamps, I think 100-150 a month is a pretty attractive deal, especially given the possibility of decoding the exchange sequence numbers and matching them with the IQFeed trades. Thanks again for the info @IAS_LLC , you've potentially saved me a lot of headache!
@cjbuckley4, with all due respect, but you keep on talking about data sources. You still have not explained what you would ever need such finely granulated data sets for. If you cannot submit orders at such latencies then why would you need such data? Risking to repeat myself, you need to start at the other end, you need to start with a strategy, holding period, trading frequency decision first before you determine which data you need. You keep on talking about data but what you instead should be focusing on is the strategy and defining what strategy approach suits you. Else you risk wasting a lot more time contemplating data sources when in actuality the thing you should be doing is strategy think.
My justification for purchasing the Nanex historical data isn't really related to my strategy requiring it. I'll openly admit to wanting the best data, but because my efforts are currently focused only on CME and CFE, I could pay (based on a quote on another thread...I shot them an email and will hear back soon I expect) 215-300 a month for quote and trade data on these two plus depth on CME. To get that data from IQFeed is a actually similar price, plus the trouble of building a database and the a program to parse the feed in every night. Not to mention the whole time I don't have the database recording, that money is just wasted. I also have that options rolling problem discussed on our other thread. Using Nanex data would achieve exactly what I want, and all I would need to do is write a program to parse their data into kdb. No more database organization questions, no more wondering how to roll the options, no more EC2, no more questioning whether trade only data is sufficient, no more questioning the logic of my program at ever turn.
It's more of a matter of simplicity...much like you say. I have school, athletics, friends, girlfriend...I don't wanna spend my whole live building a feed handler and database. I think <=300 a month is a fair price to not have to sweat data anymore and focus on strategy design.
You're right though, a lot of the 'quest for perfect data' is motivated by not knowing my strategy yet. I don't know what my strategy is, but I know it will be data driven. I think it's worth it to pay for data to be a nonissue and move forward.
It's more of a matter of simplicity...much like you say. I have school, athletics, friends, girlfriend...I don't wanna spend my whole live building a feed handler and database. I think <=300 a month is a fair price to not have to sweat data anymore and focus on strategy design.
You're right though, a lot of the 'quest for perfect data' is motivated by not knowing my strategy yet. I don't know what my strategy is, but I know it will be data driven. I think it's worth it to pay for data to be a nonissue and move forward.
Last edited:
sure if you indeed develop a strategy that deals with futures data that lists on CME/CFE. If not you wasted all that time and money as well as effort. I can again and again only caution you: You seem to be incredibly drawn to want to trade ultra high frequency strategies, however, chances of you succeeding in this space are infinitesimally small. You are competing with the absolute heavy hitters in financial space who installed micro wave antennas between Aurora and New Jersey already years ago. How do you want to compete in products like ES (S&P mini futures), or other CME products? I can only again recommend you to resort this quest for ultra high frequency to a hobby and side project and to instead urge you to focus on your real edge as a retailer: Mid frequency holdings, less noise, the ability to get in and out quickly at low cost, the ability to stay on the sidelines, and many more advantages that larger players just do not have. You are trying to pee in the home turf of other big boys and it may come back to haunt you. Define your edge first before you decide what data you need. Nobody needs Nanex data sets unless you want to compete in extremely high-ultra high frequency space. Its your money and time you invest in this, I just share my 2 cents and many years trading various asset classes professionally.
P.S.; The only field that I see worthwhile contemplating and that is high frequency is market making listed options. There are various markets where you can obtain market maker status relatively inexpensively and without too many resources. But for that you would need to have a solid understanding of option pricing theory, hedging, and greeks (risk).
P.S.; The only field that I see worthwhile contemplating and that is high frequency is market making listed options. There are various markets where you can obtain market maker status relatively inexpensively and without too many resources. But for that you would need to have a solid understanding of option pricing theory, hedging, and greeks (risk).
I hear you on the necessity of being a medium frequency trader. Nanex data wouldn't get me anywhere near competing with the HFT folks I've spoken to on CME and CFE who have latencies in the 15 microsecond and below area. For every 25 millisecond interval of resolution on Nanex there could be thousands of updates on ES, for instance. I have spoken to the IT folks at a couple of FCMs, and if the are telling the truth then millisecond timestamps may have some relevance, but even that is not my intention. I want to look at medium frequency strategies involving the CME index, energy, and interest rate products in conjunction with the CFE volatility products. Having quotes and trades, and to some extent depth of market data is important to me because I don't want to make incorrect assumptions about liquidity and slippage. To what degree is it really important to the accuracy of my Backtests? I really don't know, but I'd like to not have to stress it. I love the exact replay of the tape and the fact it will take care of all my databasing issues. Querying a table containing the entire tape of an exchange is way simpler than the crazy walk arounds I was coming up with to get exactly the same result from IQFeed data in conjunction with kdb+ on our other thread. At this point the price seems right given the potential of Nanex's data to save me time and anxiety. Plus, if I can't find opportunities on these two exchanges, I'll still have IQFeed and can just pick up a few thousand dollars poorer and more experienced on the other side, and if I like the Nanex data then I can always go back and buy more exchanges.
Last edited: