Having Issue with NxCore C++ Program ran with WINE

[AdrianHagh81]

Im using the book to create trading signals based upon Machine Learning (thats as deep as ill go), it is not low latency strat, but I will attempt to trade as quickly as possible given my latency considerations.

I'm using Machine learning too, What type of ML algo are you using ? If it's not to deep of course

I'm rather new at this ATS myself, so far have one strat running.

I was thinking of trying rithmic.

I hear Xtrader TT api and gateway are blazing fast but expensive.

And BTW if your using Market orders on some fast moving futures, that'll
change your strategy profile i think.
I'm assuming your targeting a few ticks for each trade since your trading the book ?

why not just use a limit order with a maximum predefined offset from the target.

 

[IAS_LLC]

I am going to use marketable limits. I am using Bayesian Networks primarily.

In college I was able to design an autopilot that was able to successfuly relearn and control an aircraft's dynamics in the event of a severe failure ( losing an aileron, etc..) using neural nerworks. That being said, I dont have a lot of confidence in NNs ......so I do use them, but sparingly.

How about you?

I'm using Machine learning too, What type of ML algo are you using ? If it's not to deep of course

I'm rather new at this ATS myself, so far have one strat running.

I was thinking of trying rithmic.

I hear Xtrader TT api and gateway are blazing fast but expensive.

And BTW if your using Market orders on some fast moving futures, that'll
change your strategy profile i think.
I'm assuming your targeting a few ticks for each trade since your trading the book ?

why not just use a limit order with a maximum predefined offset from the target.

 

[AdrianHagh81]

I am going to use marketable limits. I am using Bayesian Networks primarily.

In college I was able to design an autopilot that was able to successfuly relearn and control an aircraft's dynamics in the event of a severe failure ( losing an aileron, etc..) using neural nerworks. That being said, I dont have a lot of confidence in NNs ......so I do use them, but sparingly.

How about you?

I'm using linear SVM with plain old GP.

Basically a spray and pray approach, but managed to find some gems so far.

on NNs with their non linearity haven't given me any usefull results so far.


Yeah, I used NN in college also, for my personal porn indexer.

It performed pretty well categorizing all the different type of porns.


What instruments are you trading then ?

 

[vicirek]

I think if its on the same OS same machine, the sockets are treated differently.

I can't remember if its linux or windows, but local sockets connections are not really tcp ip socket.
maybe they changed that.

Anyway using shared memory with all the pointer arithmetic is a bit eye straining.

But yeah if your looking for sub 1 ms latency i guess thats one of the best ways.

Correct me if I'm wrong but a socket stream will still have a minimum about 1 ms latency right ?

With direct pointers its probably in the low micro seconds on a standard windows server.


What ethernet card are you using for your low latency strat ?

This link may shed some light on socket loopback latencies:

http://blogs.technet.com/b/wincat/a...ndows-server-2012-tcp-loopback-fast-path.aspx

They are in microseconds. I am occasionally using sockets between computers on local network and latency is under 1ms as well.

There would not be much difference between Linux and Windows sockets because both are based on the same core concept and differ in headers used and slight syntactic differences.

One problem I can see is that loopback is a concept within the OS environment the program is running in. Wine and Linux would occupy separate address space so I leave it to you guys to figure out if loopback or direct addressing would apply. Myself I would not complicate my computing environment on one machine running two OS, emulators etc. Computers are cheap (still) and time is precious.

I had a look at NxCore APi and it runs as close to Windows kernel as possible, it is C based (C++ wrapper available) and optimized for WinApi. The stability of it is assured by redundancy and "self healing" tape so average user is not aware of any issues with connectivity if these are transient. In that case I would stay with Windows instead of putting my program at the mercy of yet another programmer who translates my runtime for different environment.

 

[volpunter]

Yes you are correct, even Linux sockets come with overhead and I just went with what IAS_LLC stated which is that he wants to trade "the book" which implies he needs a low-latency solution. But as I already pointed out there quite a number inconsistencies in his aggregate statements. I point them out for the last time and then I am out of here because I do get the sense that he just looks to justify his use case rather than engage in a logical discussion:

+ He wants to trade the book, which implies he requires a low latency solution
+ He peruses a data feed that clearly does not not cater to Linux solutions, and hence runs code that is optimized for Windows OS and runs it in Wine on a Linux box
+ He peruses sockets rather than shared memory (yes I know sharing memory between a Wine app and an app that runs on pure linux is tricky at times)
+ But most confusing he uses IB to execute yet wants to trade the order book. Highly strange given the time it takes for an order to get to IB's gateway on the local machine and then to IB servers.

But hey, if it works for him good, I just got a bit confused because he seems to have a hard time comprehending that his setup is less than sub-optimal to trade order books....

I think if its on the same OS same machine, the sockets are treated differently.

I can't remember if its linux or windows, but local sockets connections are not really tcp ip socket.
maybe they changed that.

Anyway using shared memory with all the pointer arithmetic is a bit eye straining.

But yeah if your looking for sub 1 ms latency i guess thats one of the best ways.

Correct me if I'm wrong but a socket stream will still have a minimum about 1 ms latency right ?

With direct pointers its probably in the low micro seconds on a standard windows server.


What ethernet card are you using for your low latency strat ?

 

[volpunter]

And this is why I stated you obviously have little to no experience trading and managing risk. Stating that you apply "Machine Learning" is not "deep" nor provides any insight whatsoever. Not that we require insight but applying machine learning algorithms to order book dynamics IS LOW LATENCY, whether you like to hear it or not. But your setup does not look like it is optimized to run low latency algorithms.

And further more, its a clear giveaway that we are talking to a pure developer when you are asking what other brokers are out there that provide a C++ API. If you were a trader you would look at it the other way around: "What broker suits my strategy and trading need/style" and then you go with whatever API they provide, and only then you figure out whether Linux or Windows is the better solution and at the very end you determine the language in which you implement your solution. Locking yourself into a specific language makes little to no sense for someone wanting to engage in this business. You may be better suited to look around for shops that look for developers with C++ skills if you are so adamant about C++.

Im using the book to create trading signals based upon Machine Learning (thats as deep as ill go), it is not low latency strat, but I will attempt to trade as quickly as possible given my latency considerations.

Im not close to going live yet, but im using market orders so rapid confirmation is not a huge concern of mine. That being said, I would love to erase 2-300 ms from the conf time. Can you recommend another low cost futures broker with a c++ api? Im certainly not attatched to IB.

As for sockets being treated differently on the same machine, you are correct. They are "local sockets". Windows calls them direct pipes, linux calls them something similar. What im not sure of is if I can use local sockets to talk to a wine program from a native linux program. Fortunately QT has classes for both local and tcp sockets that have very similar syntax, so when I get to that stage I will try both.

I have no clue what my network card is. When Im up and running, I will run on a remote server, but I havent defined what I require of that server yet. Once my software is finished, back testing and execution modeling will help me to determine what my true latency needs are.

 

[volpunter]

Something I recommended many pages ago. Being married to an operating system and specific language hardly ever proved useful, unless of course all utilized APIs and other entry and exit points support the specific language in question. I early on decided not to participate in the technology race in pure hft space and for that reason Linux never was a consideration for lack of broad API support. But then if someone trades longer frequencies I guess one can even get away with a REST API on any OS. But generally the trade should determine the tools and not the other way around.

This link may shed some light on socket loopback latencies:

http://blogs.technet.com/b/wincat/a...ndows-server-2012-tcp-loopback-fast-path.aspx

They are in microseconds. I am occasionally using sockets between computers on local network and latency is under 1ms as well.

There would not be much difference between Linux and Windows sockets because both are based on the same core concept and differ in headers used and slight syntactic differences.

One problem I can see is that loopback is a concept within the OS environment the program is running in. Wine and Linux would occupy separate address space so I leave it to you guys to figure out if loopback or direct addressing would apply. Myself I would not complicate my computing environment on one machine running two OS, emulators etc. Computers are cheap (still) and time is precious.

I had a look at NxCore APi and it runs as close to Windows kernel as possible, it is C based (C++ wrapper available) and optimized for WinApi. The stability of it is assured by redundancy and "self healing" tape so average user is not aware of any issues with connectivity if these are transient. In that case I would stay with Windows instead of putting my program at the mercy of yet another programmer who translates my runtime for different environment.

 

[IAS_LLC]

I dont understand why using machine learning to generate trading signals from orderbook data has to be low latency? I have data that suggests otherwise, and im very conservative in my fill modeling. I DO model latency in my fills. Sure, trading low latency maximizes profits, but if your alpha generating algorithms take latency into consideration , there is no reason you cant use machine learning on any time frame, although the orderbook does get less useful as the time frame lengthens.

I am not a professional trader, but I most certainly am not a "developer" , as you continue to suggest. I posses advanced engineering degrees , and am responsible for the design of some of the most advanced guidance, estimation, and control algorithms that are being flown in the aerospace industry. I would argue that the risk I manage everyday designing these systems far exceeds the risk any one professional trader does, both in terms of $$$ and lives at stake. Managing trading risk is not rocket science. Im not as clueless as you keep suggesting,

Im not OS or language attatched, I just havent seen a good reason to get away from C++ or Linux, you certainly havent provided any ( other than im stupid, and clueless). I certainly have all of the major languages in my arsenal. Oddly enough, I know how use windows too.

As for the broker thing, admittedly I havent done my homework there. In my defense, im atleast a couple months away from going live anyway.

And this is why I stated you obviously have little to no experience trading and managing risk. Stating that you apply "Machine Learning" is not "deep" nor provides any insight whatsoever. Not that we require insight but applying machine learning algorithms to order book dynamics IS LOW LATENCY, whether you like to hear it or not. But your setup does not look like it is optimized to run low latency algorithms.

And further more, its a clear giveaway that we are talking to a pure developer when you are asking what other brokers are out there that provide a C++ API. If you were a trader you would look at it the other way around: "What broker suits my strategy and trading need/style" and then you go with whatever API they provide, and only then you figure out whether Linux or Windows is the better solution and at the very end you determine the language in which you implement your solution. Locking yourself into a specific language makes little to no sense for someone wanting to engage in this business. You may be better suited to look around for shops that look for developers with C++ skills if you are so adamant about C++.

 

[AdrianHagh81]

I dont understand why using machine learning to generate trading signals from orderbook data has to be low latency? I have data that suggests otherwise, and im very conservative in my fill modeling. I DO model latency in my fills. Sure, trading low latency maximizes profits, but if your alpha generating algorithms take latency into consideration , there is no reason you cant use machine learning on any time frame, although the orderbook does get less useful as the time frame lengthens.

I am not a professional trader, but I most certainly am not a "developer" , as you continue to suggest. I posses advanced engineering degrees , and am responsible for the design of some of the most advanced guidance, estimation, and control algorithms that are being flown in the aerospace industry. I would argue that the risk I manage everyday designing these systems far exceeds the risk any one professional trader does, both in terms of $$$ and lives at stake. Managing trading risk is not rocket science. Im not as clueless as you keep suggesting,

Im not OS or language attatched, I just havent seen a good reason to get away from C++ or Linux, you certainly havent provided any ( other than im stupid, and clueless). I certainly have all of the major languages in my arsenal. Oddly enough, I know how use windows too.

As for the broker thing, admittedly I havent done my homework there. In my defense, im atleast a couple months away from going live anyway.

Take it with a grain of salt but
From my observation, succesfull professionals in other fields have the hardest time transitioning
into trading. And this field does attract the super alpha type personality, which is why I guess this
forum is full of the biggest dicks in the universe, but sometimes those dicks are saying something worth
paying attention too.

 

[IAS_LLC]

Noted.

Take it with a grain of salt but
From my observation, succesfull professionals in other fields have the hardest time transitioning
into trading. And this field does attract the super alpha type personality, which is why I guess this
forum is full of the biggest dicks in the universe, but sometimes those dicks are saying something worth
paying attention too.