L1 and L2 data stream question

keep in mind that with respect to screens, the most data changes you can see is limited by the refresh frequency of your monitor (typically 60 per second). a further restriction is that your eye/brain combo really cannot detect changes more frequently than about 150 milliseconds which, by the way, is the time it takes to blink (so now we can quantify the time to which "in a blink of an eye" refers).

given these restrictions and given the fact that data from an exchange is published at a far greater rate than 60 messages per second, we felt that posting changes more frequently could be a waste of cpu. so in later versions of R | Trader we put in a preference that gives the user the ability to limit the number of times per second that the screeen would be painted. by default it is set to 10. we observed that this saved a lot of cpu compared to no limit on the number of times the screen could be painted. if you make a side by side comparison between R | Trader and another screen you might want to set the max update rate to "No Max".
 
Quote from PocketChange:

Unless you are receiving 70 updates per second per quote you are receiving partial data. As far as i know CME only time stamps last trade.

For 6E you would be receiving 770 updates per second. if you are HFT in milliseconds you should colocate at equinix and get a fiber connection to the exchange. [...]
More...

Do you mean that the 6E is receives much more updates per second (x11) vs. ES?

Also here is an excerpt from the latest CME Bulletin:

"Equity Futures Enhancements
Effective Sunday, October 4, 2009, a number of enhancements will be introduced for CME Equity futures and future spreads on CME Globex.

This launch will also result in reduced message response times. Already among the fastest in the industry, this upgrade is expected to reduce response times by 40-50% for CME Equity futures.

Please note: as a result of the reduced message response times, bandwidth utilization in these futures markets may ultimately increase by as much as 20%."
 
Quote from NetTecture:

Sounds more than reasonable. Whever Ninja is involved, I am reluctant to point fingers to anyone else in the first place ;) They are FAR too likely to have some brain dead design decisions or simply programming errors somewhere along the path to start with ;)

With your approach you can acutally compare the data. I would not wonder if anything gets lost in the Ninja internal processing.
More...

To be fair to Ninja, I have spoken to their support guys a lot and they have always been reasonably helpful. I get the impression that everyone at Ninja is aware that the software has some serious architectural problems. Of course, they are never going to admit this publicly because it would be a PR disaster. If you look at the company history (http://www.ninjatrader.com/webnew/futures_trading_platform_company_about.htm) you can see that it's still a relatively immature piece of software. I suspect their software development team is pretty small and that they have done a lot of learning 'on the job' which in turn has led to some poor design decisions.

Like everyone, I have great hopes that they have taken a big step forwards and rearchitected NT7 in a much better fashion.
 
Quote from ScoobyStoo:

To be fair to Ninja, I have spoken to their support guys a lot and they have always been reasonably helpful. I get the impression that everyone at Ninja is aware that the software has some serious architectural problems. Of course, they are never going to admit this publicly because it would be a PR disaster. If you look at the company history (http://www.ninjatrader.com/webnew/futures_trading_platform_company_about.htm) you can see that it's still a relatively immature piece of software. I suspect their software development team is pretty small and that they have done a lot of learning 'on the job' which in turn has led to some poor design decisions.

Like everyone, I have great hopes that they have taken a big step forwards and rearchitected NT7 in a much better fashion.
More...

To be fair that doesn ot change the fact that... NT 6 .5 is crap. Point. THe most buggy software I ever used, only slightly worse than certain versions of Rational Rose.

Hpopes in NT 7... may be unfounded, though, as it looks like some of their decisions are going from "bad" to "worse". We will soon know, though ,and a lot of people (me included) hope.

That said ; found their support ranging from smart and helpfull (Bertran) to utterly stupid (Dierk - seems he does not even read the posts he answers). Which is usual. Sadly, there is only so much that even terrific support can do, when the sofware is just THAT bad ;)

Let's all hope for NT7 ;) Should not be TOO long.
 
Quote from jjw:

keep in mind that with respect to screens, the most data changes you can see is limited by the refresh frequency of your monitor (typically 60 per second). a further restriction is that your eye/brain combo really cannot detect changes more frequently than about 150 milliseconds which, by the way, is the time it takes to blink (so now we can quantify the time to which "in a blink of an eye" refers).

given these restrictions and given the fact that data from an exchange is published at a far greater rate than 60 messages per second, we felt that posting changes more frequently could be a waste of cpu. so in later versions of R | Trader we put in a preference that gives the user the ability to limit the number of times per second that the screeen would be painted. by default it is set to 10. we observed that this saved a lot of cpu compared to no limit on the number of times the screen could be painted. if you make a side by side comparison between R | Trader and another screen you might want to set the max update rate to "No Max".
More...

Thanks for your thoughts Jonathan.

As a pure algo trader I don't do any sort of visual information processing and so I'm not really fussed about UI limitations. During the trading day I'm usually kept occupied with new development work (or debugging data stream synch problems) whilst my production servers sit in their nicely air conditioned data centre doing the actual trading.

I simply use the Ninja API as a cheap and cheerful interface to the ZenFire/Rithmic platform. For this reason I'm also not bothered about the architectural problems inherent in the Ninja UI which plague so many other people.

The reason I noticed the L1 and L2 datastreams getting out of synch was because I started to see some strange behaviour from my iceberg detection algos which cross reference both transaction data (L1) and order book data (L2). This led me to investigate the synchronisation of the 2 datastreams.

When you release the .NET version of the R API then I'll probably dump Ninja completely. I would code my own .NET wrappers for the C++ libraries but to be brutally honest I can't be bothered if you guys are going to release a native .NET implementation later this year. I might still use Ninja for running backtests but that's about it.

P.S. Why did Rithmic take the decision to market their platform under both the Rithmic and ZenFire brands? I think everyone's now aware that they are the same thing but I know it caused quite some confusion for a while. Why bother with the ZenFire brand at all?
 
Quote from NetTecture:

To be fair that doesn ot change the fact that... NT 6 .5 is crap. Point. THe most buggy software I ever used, only slightly worse than certain versions of Rational Rose.

Hpopes in NT 7... may be unfounded, though, as it looks like some of their decisions are going from "bad" to "worse". We will soon know, though ,and a lot of people (me included) hope.
More...

Worked on a project where we had to use Rational Rose a couple of years ago. You're not wrong. Dreadful piece of software. There's an old saying which says 'nobody ever got sacked for buying IBM'. Well it's about time fucking time they did!

What design decisions have they made in NT7? I haven't been keeping up with it's development at all. I thought I'd just wait for the beta.
 
Quote from ScoobyStoo:

Thanks for your thoughts Jonathan.

As a pure algo trader I don't do any sort of visual information processing and so I'm not really fussed about UI limitations. During the trading day I'm usually kept occupied with new development work (or debugging data stream synch problems) whilst my production servers sit in their nicely air conditioned data centre doing the actual trading.

I simply use the Ninja API as a cheap and cheerful interface to the ZenFire/Rithmic platform. For this reason I'm also not bothered about the architectural problems inherent in the Ninja UI which plague so many other people.

The reason I noticed the L1 and L2 datastreams getting out of synch was because I started to see some strange behaviour from my iceberg detection algos which cross reference both transaction data (L1) and order book data (L2). This led me to investigate the synchronisation of the 2 datastreams.

When you release the .NET version of the R API then I'll probably dump Ninja completely. I would code my own .NET wrappers for the C++ libraries but to be brutally honest I can't be bothered if you guys are going to release a native .NET implementation later this year. I might still use Ninja for running backtests but that's about it.

P.S. Why did Rithmic take the decision to market their platform under both the Rithmic and ZenFire brands? I think everyone's now aware that they are the same thing but I know it caused quite some confusion for a while. Why bother with the ZenFire brand at all?
More...

Originally, Rithmic produced its software for alogrithmic traders who trade via programs that they themselves write. Using Rithmic's software a trader/programmer could conecentrate on developing and trading algorithms, instead of dealng with exchange related issues and computing problems. Rithmic also produced 2 screens, R | Trader and R | Manager so that FCM's could view positions, manage risk, place orders on behalf of their clients and pull orders when needed. R | Trader also saved the algorithmic programmers the time effort and money needed to produce their own screen (unles they chose to write one anyway).

At that time our customers were running our software on machines hosted by and connected to the network provided by 7Ticks.

Around the time that Refco blew up we began working with Mirus Futures. Though their clientele was somewhat different from ours (screen based traders versus program based traders) there was no reason why screen based tradears coud not benefit from the same trading platform as algorithmic traders. As Rithmic is not a front end shop we were introduced to Ninja and Ninja incorporated R | API into its screen.

Mirus marketed the complete package (the Rithmic trading and market data platform, running on machines hosted by and connected to the network provided by 7Ticks, with Ninja as the front end, and the service provided by Mirus). It grew in popularity and other IB's joined in. People began asking to sign up for it so it was given a name (the whole package) - Zen-Fire.

Rithmic still mostly provides its software to algorithmic programmers/traders/prop shops. Our marketing to this market segment is low key (consistent with our personalities - our work speaks for itself) but we do get interesting referrals. We have also built some interesting trades ranging from news events to spread scalping to very high speed brain dead scalping, all of which are available to be licensed. Recently our focus has increased to offer connectivity to equities and equity options exchanges - tihs is actually from where our roots spring and is in large part why our futures platform on day one had no issues with futures market data and large scale market data distribution, but, as Conan's biographer states, "That is another story". Look for it over the next couple of months.
 
Thanks for correcting:
Meant 6E level I and Level II streams:
11 quote streams. (10 from level II DOM, 1 Level I).


Quote from Bernard111:

Do you mean that the 6E is receives much more updates per second (x11) vs. ES?

Also here is an excerpt from the latest CME Bulletin:

"Equity Futures Enhancements
Effective Sunday, October 4, 2009, a number of enhancements will be introduced for CME Equity futures and future spreads on CME Globex.

This launch will also result in reduced message response times. Already among the fastest in the industry, this upgrade is expected to reduce response times by 40-50% for CME Equity futures.

Please note: as a result of the reduced message response times, bandwidth utilization in these futures markets may ultimately increase by as much as 20%."
More...
 
Thought I'd update this thread just in case anybody stumbles across it in the future.

The problem with the L1 and L2 datastreams getting out of synch did indeed rest with NinjaTrader's thread switching. Upon further examination the datastream supplied by Zen/Rithmic didn't exhibit the problem.

I have since completely stopped using Ninja as it does not lend itself at all to realtime processing of individual market data messages.
 
You must log in or register to reply here.
Back
Top