arguing to the point and remaining factual is definitely not your strength. I recommend you stick to the blog and journal threads going forward ;-) Oh, should I ever come across issues with Linux then I will contact you ;-)
Having Issue with NxCore C++ Program ran with WINE
- Thread starter IAS_LLC
- Start date
There's nothing inherently difficult running mixed-OS systems as such. Wine, VMWare, and multiple servers are all feasible for this; there may be an extra layer of debugging involved, but debugging such problems as I've encountered them has been far easier than debugging my core code
Most high-volume, automated trading shops run mixed Windows/Unix/Linux systems, with Linux/Unix as their core OS. It's been that way for years and continues today -- just take a look at the job postings.People will tend to use what's best for them and what they know, as IAS_LLC has apparently done. I don't see any reason for this previously-useful thread to degenerate into ad hominem bickering.
Couldn't agree more.
Occam, sure they run "mixed OS systems" but only in the context of running specific, context-focused, software application on those respective system. For example, most banks run UIs, pricing front-ends and what have you on Windows machines, while the backend runs on Linux (though a number of banks have migrated even their pricing farms to Windows servers, but that is irrelevant). However, what I have hardly ever heard or seen is that the same piece of software is supposed to run on multiple systems. It makes little to no sense and anyone doing it is begging for disasters to happen. (I am not talking about Web applications, I talk about apps/APIs that dive into OS specific issues, such as the programming of sockets).
We can always end each and every discussion with the proverbial "if it works for you then good, go for it!". But whether it makes sense, is a logical thing to do, efficient, and adds value is an entirely different issue. Just because IAS_LCC "feels" right about something does not make that at all the right route to go. By all means he can do whatever pleases him, but then he has to accept others telling and proving to him that what he is doing is nonsense. He may reject that criticism but he cannot ignore it.
To get back to the issue at hand, data feed handlers are highly sensitive pieces of the puzzle in systematized trading, a single change of line of code can completely derail the platform. Are you sure you would not want to implement this piece of software in the language the API is targeting and not run it on the OS the language was primarily targeting (if the language is truly OS agnostic then you would not need an emulator, right)? If you are unhappy with the lack of OS or language support by the broker/data vendor then a different data vendor should be considered. Or else use an implementation that is OS/language agnostic (such as FIX). But your ass would get fired in any quant or IT group within a hedge fund or bank if you suggested running the implementation of a data feed handler on an OS that the language it was written in is not fully supporting and potentially raises a lot of issues, even such issues may not right away surface. If you need to run a piece of software on Windows OS then write it in a language and use an IDE that fully supports Windows, and equally if you need to run on Linux. Who in their right mind would program a critical piece of software in .Net on a Windows OS and then run it on Linux (Mono is several .Net versions behind and even then has a lot of compatibility issues. If you don't believe me then take a 1000 lines of code that peruses .Net 4.0 features and tell us how it compiled and ran).
Think about for a second why web apps became so popular and why they work on different OSs...its because there are standards which most major browsers support and you can see the browsers as some sort of sandboxed environment, preventing the web app from actually making OS or hardware specific calls. For that precise reason hardly anyone would write a software application that targets sockets that has to run perfectly fine in Linux/Unix and Windows.
We can always end each and every discussion with the proverbial "if it works for you then good, go for it!". But whether it makes sense, is a logical thing to do, efficient, and adds value is an entirely different issue. Just because IAS_LCC "feels" right about something does not make that at all the right route to go. By all means he can do whatever pleases him, but then he has to accept others telling and proving to him that what he is doing is nonsense. He may reject that criticism but he cannot ignore it.
To get back to the issue at hand, data feed handlers are highly sensitive pieces of the puzzle in systematized trading, a single change of line of code can completely derail the platform. Are you sure you would not want to implement this piece of software in the language the API is targeting and not run it on the OS the language was primarily targeting (if the language is truly OS agnostic then you would not need an emulator, right)? If you are unhappy with the lack of OS or language support by the broker/data vendor then a different data vendor should be considered. Or else use an implementation that is OS/language agnostic (such as FIX). But your ass would get fired in any quant or IT group within a hedge fund or bank if you suggested running the implementation of a data feed handler on an OS that the language it was written in is not fully supporting and potentially raises a lot of issues, even such issues may not right away surface. If you need to run a piece of software on Windows OS then write it in a language and use an IDE that fully supports Windows, and equally if you need to run on Linux. Who in their right mind would program a critical piece of software in .Net on a Windows OS and then run it on Linux (Mono is several .Net versions behind and even then has a lot of compatibility issues. If you don't believe me then take a 1000 lines of code that peruses .Net 4.0 features and tell us how it compiled and ran).
Think about for a second why web apps became so popular and why they work on different OSs...its because there are standards which most major browsers support and you can see the browsers as some sort of sandboxed environment, preventing the web app from actually making OS or hardware specific calls. For that precise reason hardly anyone would write a software application that targets sockets that has to run perfectly fine in Linux/Unix and Windows.
There's nothing inherently difficult running mixed-OS systems as such. Wine, VMWare, and multiple servers are all feasible for this; there may be an extra layer of debugging involved, but debugging such problems as I've encountered them has been far easier than debugging my core code
People will tend to use what's best for them and what they know, as IAS_LLC has apparently done. I don't see any reason for this previously-useful thread to degenerate into ad hominem bickering.
Last edited:
You have made incorrect assumptions about why I am running mixed OS. I am not specifically designing software that can be used on either os, the ability to run either os is just a byproduct of intelligent design. Everything I have written, except for the interface to NxCore is completely platform independent. I didnt have to go out of my way to make it that way, and that doesnt imply I intend to test it on one OS and than trust it will work on another. The only reason I ever ran anything on windows at all was as an attempt to pinoint the source of my problems. An excersize that was successful.
Well again if that works for you then great. I do not write to convince you, I made the effort to write to point out to others that even thinking of running a data feed handler, OMS, execution model or the like on various OSs makes little to no sense. No professional is doing that. You decide on which OS you run such modules and then stick to it. what good does it do to you to be able to run NxCore on Windows if you run your algorithms on a Linux box and vice versa? Why would you even bother to care whether it works on Windows unless of course you are a pure developer (which you sound like you are) rather than a trader concerned with making money. Maybe you look to sell your software to various customers. In that case, if I was your customer I would still stay miles away, I prefer to purchase from someone who has developed software from the ground up on a system that I intend to run such application on.
I have made that decision. ....the data feed handler runs on windows....everything elese is on linux. They communicate through sockets. Its not ideal, id rather it all runs linux and uses shared memory, but a reasonably priced data feed of NxCore quality does not exist for linux. Many NxCore users run under WINE. Im not doing anything crazy in that regard, and there is little to no performance overhead. You seem to be hell bent on criticizing me, which is fine, but you are completely misrepresenting what im doing.
I've been running NxCore for years under Linux in full production; I just rebooted a server that ran NxCore uninterrupted since December 2013 (the reboot was precautionary and had nothing to do with Linux or NxCore). That's a single, uninterrupted NxCore process, running on Wine, nonstop for over 7 months without so much as a hiccup. The SIP itself (and even the exchanges) have had more outages than I've had running NxCore on Linux in the past two years.
In my experience, NxCore running under Wine on Linux is far more reliable than Windows 8 itself, unless you're running some stripped-down server version (which I've never done).
That's not to say that there isn't some initial coding/configuration overhead in setting up a mixed-OS system, which obviously there is. But the tools for this are so much better now that it's not a big issue anymore (for me, at least).
By the way, you do not need .Net to run or interface with NxCore (and hence, if running under Wine, no need for mono, which I've never used).
In my experience, NxCore running under Wine on Linux is far more reliable than Windows 8 itself, unless you're running some stripped-down server version (which I've never done).
That's not to say that there isn't some initial coding/configuration overhead in setting up a mixed-OS system, which obviously there is. But the tools for this are so much better now that it's not a big issue anymore (for me, at least).
By the way, you do not need .Net to run or interface with NxCore (and hence, if running under Wine, no need for mono, which I've never used).
Occam,
That is encouraging information. Would you mind sharing a bit more about your setup?
1) What flavor of Linux are you using?
2) What version of WINE are you running?
3) Which Compiler do you use?
4) What is your method of transfering data from your quote processor to your trading algorithms?
Here are my answers to those questions:
1) Ubuntu 12.04 LTS
2) 1.6
3) x86-w64-mingw32-g++ for quote processing/packaging , g++ for trading algorithms, risk management, and IB API interface.
4) Right now im writing quote updates to binary files with my NxCore quote processing application and reading those files with my trading system, but this is for rapid development only. Doing it like this allows me to process an entire days worth of the globex contracts i trade in 15 minutes or so, but I can probably get much faster if I dont have all of that file I/O. Oddly, it runs faster on windows but I suspect that is an OS resource allocation feature. Eventually data transmission will take place through TCP sockets, likely using the QT QTcpSocket framework. I suspect this may force a compiler change when I get to this point. I will eventually make a QT based GUI for my system, so this isnt a problem.
Thanks In advance
That is encouraging information. Would you mind sharing a bit more about your setup?
1) What flavor of Linux are you using?
2) What version of WINE are you running?
3) Which Compiler do you use?
4) What is your method of transfering data from your quote processor to your trading algorithms?
Here are my answers to those questions:
1) Ubuntu 12.04 LTS
2) 1.6
3) x86-w64-mingw32-g++ for quote processing/packaging , g++ for trading algorithms, risk management, and IB API interface.
4) Right now im writing quote updates to binary files with my NxCore quote processing application and reading those files with my trading system, but this is for rapid development only. Doing it like this allows me to process an entire days worth of the globex contracts i trade in 15 minutes or so, but I can probably get much faster if I dont have all of that file I/O. Oddly, it runs faster on windows but I suspect that is an OS resource allocation feature. Eventually data transmission will take place through TCP sockets, likely using the QT QTcpSocket framework. I suspect this may force a compiler change when I get to this point. I will eventually make a QT based GUI for my system, so this isnt a problem.
Thanks In advance
Can I ask what are you guys doing that requires NXCore ?
Start arbing stocks ? pair trading ?
Order book trading ?
Start arbing stocks ? pair trading ?
Order book trading ?