Best practices?

[nitro]

Quote from ScoobyStoo:

OK, I can see now why you are prepared to accept the tight coupling and the maintenance overhead that this design approach brings. As you said earlier, it all depends on the required speed of execution and my needs are obviously not as great as yours. I have adopted a decoupled approach throughout with a fundamental principle being that I should be able to port the app to a different API with the minimum of hassle.

You got it.

Your design decision has advantages obviously. But if you are trying to shave sub-milliseconds from your latency, you throw out most of this OO stuff when it slows you down, especially if it is in the heart of your market-data handling code or order handler that sees trillions of cycles over the course of a trading day.

Your language of choice also makes a huge difference. For example in heavy multi-threaded code, in Erlang vs C++/C# etc, you do lots of copies (correct functional programming), but you avoid locks. Your allocator then makes a huge difference. It is an interesting race...

 

[ScoobyStoo]

Quote from nitro:

You got it.

Your design decision has advantages obviously. But if you are trying to shave sub-milliseconds from your latency, you throw out most of this OO stuff when it slows you down, especially if it is in the heart of your marketdata handling code or order handler.

Your language of choice also makes a huge difference. For example in heavy multi-threaded code, in Erlang vs C++/C# etc, you do lots of copies, but you avoid locks. Your allocator then makes a huge difference. It is an interesting race...

I don't need the performance of C++ so am using C#, simply because the toolset available is so much richer. The various elements of my market data analysis functionality are running on separate threads. Order management obviously runs on its own thread. Haven't done any profiling yet but with a bit of a kick in the right direction the GC shouldn't be too much of a problem. If the GC proves too troublesome though I can always drop into unmanaged code in the really critical blocks to maintain performance.

 

[nitro]

Quote from ScoobyStoo:

I don't need the performance of C++ so am using C#, simply because the toolset available is so much richer. The various elements of my market data analysis functionality are running on separate threads. Order management obviously runs on its own thread. Haven't done any profiling yet but with a bit of a kick in the right direction the GC shouldn't be too much of a problem. If the GC proves too troublesome though I can always drop into unmanaged code in the really critical blocks to maintain performance.

Just so that you know, 90% of the pros that do this use C++, occasionally with MFC or Linux equivalents, but in modern times it is invariably with STL and Boost, and sometimes the ACE threads library. I have seen some pros do it in C# and get 90% of the performance, but they are very deep understanding of the .Net platform.

The hyper-modern way I have seen this done is with Erlang or Clojure or Scala, but that is a rare skill today, imo.


Either way, you will learn alot.

 

[propseeker]

Quote from nitro:

Even in the order case, I have seen order state get out of sync with API order state, but I have yet to see intra-day positions as reported by the API get out of sync with what it really is.

faulty internal order state implementations does not decrease the value of the concept... it decreases the value of the programmers who botched it.

re: api getting out of sync... i've had api's/lines fail to the point where the exchange had to EMAIL the list of fills that were then MANUALLY updated. with no internal order state tied to data to get some hedges off, we would have been dead flying blind. just because you haven't seen it, doesn't mean it can't or hasn't happened. that's the whole point of failsafes... you protect yourself against the devils that you don't know, and manage the ones that you do.

 

[ScoobyStoo]

Quote from nitro:

Just so that you know, 90% of the pros that do this use C++, occasionally with MFC or Linux equivalents, but in modern times it is invariably with STL and Boost, and sometimes the ACE threads library. I have seen some pros do it in C# and get 90% of the performance, but they are very deep understanding of the .Net platform.

The hyper-modern way I have seen this done is with Erlang or Scala, but that is a rare skill today, imo.


Either way, you will learn alot.

Thanks for the heads up. I looked at Retlang a while back. Out of curiousity, at what level do you see the performance/productivity tradeoff between C++ and C# tipping in favour of C++. As far as I have seen, unless you are operating in the space where >100ms delays severely impact your app then the business case for C++ is pretty weak.

 

[nitro]

Quote from propseeker:

faulty internal order state implementations does not decrease the value of the concept... it decreases the value of the programmers who botched it.

re: api getting out of sync... i've had api's/lines fail to the point where the exchange had to EMAIL the list of fills that were then MANUALLY updated. with no internal order state tied to data to get some hedges off, we would have been dead flying blind. just because you haven't seen it, doesn't mean it can't or hasn't happened. that's the whole point of failsafes... you protect yourself against the devils that you don't know, and manage the ones that you do.

Of course it can happen, I have just never seen it __intra-day__. My point is what the heck are you going to trust no matter how many stale redundant systems you have? You might as well flip a coin.

It's like saying, I am going to keep two stale pieces of bread around in case the other one gets stale. See, I am redundant!

 

[nitro]

Quote from ScoobyStoo:

Thanks for the heads up. I looked at Retlang a while back. Out of curiousity, at what level do you see the performance/productivity tradeoff between C++ and C# tipping in favour of C++. As far as I have seen, unless you are operating in the space where >100ms delays severely impact your app then the business case for C++ is pretty weak.

It is never my choice. If I am offernig my services as a programmer, they hire me as a C++ programmer to do a data feed, that is what I do. If they hire me to do it in C#, that is what I do. If they __ask__ me, I suggest C++ & STL & Boost for the server end, and C# or VB or Java or FLEX or Silverlight or whatever for the front end.

I am waaay more productive in C# over C++, so for my own work that is what I choose. But that is putting the cart before the horse in the case of market-data handling and order handling, imo.

Here is the rub though. If you don't have a direct connection to the exchange for market data or orders, it probably makes no difference between C++ and C#! The lag in the all the hops (software, switches, routers, etc) overwhelm language choice. Only in the most professional situations does it matter.

 

[Corey]

Quote from nitro:

Just so that you know, 90% of the pros that do this use C++, occasionally with MFC or Linux equivalents, but in modern times it is invariably with STL and Boost, and sometimes the ACE threads library. I have seen some pros do it in C# and get 90% of the performance, but they are very deep understanding of the .Net platform.

The hyper-modern way I have seen this done is with Erlang or Clojure or Scala, but that is a rare skill today, imo.


Either way, you will learn alot.

In your opinion, do the type safety guarantees of a functional language add anything to the system? I have started using Haskell more and more lately and I am finding the distinction of 'pure' (functions that will ALWAYS return the same output for a given input) and 'impure' (different output for same input -- i.e. user interfaces, networking, random number generation, et cetera) through Monads really allows me to modularize my code in a clean manner. By doing as much code in 'pure' Haskell as possible, I am more 'sure' of the behavior of my program.

Has anyone else done any serious work in a functional language? I know Jane Street uses O'Caml for this very reason...

Erlang would provide the same benefits, but with that nice built in distributive capabilities (scaling with ease...). After all,"Erlang is the specialist language narrowly focused on networked, highly-scalable concurrent applications." [quoting a LtU post]. A automated trading system really seems to fit the bill there...


As for Clojure/Scala, I wouldn't expect them to outperform C# by too great a margin, given that they still run on the Java platform (though, Java 6 -server is pretty damn fast). Scala still allows you to write code imperatively, so unless you really force yourself to do it the functional way, you aren't getting any benefit from the type-safety. Furthermore, as far as I know, the current JDK doesn't have tail-recursion, which is pretty much a must-have for most functional languages. If you are looking to do some data manipulation with a functional language, check out <a href="http://data-sorcery.org/">Incanter</a>.

 

[nitro]

Quote from Corey:

In your opinion, do the type safety guarantees of a functional language add anything to the system? I have started using Haskell more and more lately and I am finding the distinction of 'pure' (functions that will ALWAYS return the same output for a given input) and 'impure' (different output for same input -- i.e. user interfaces, networking, random number generation, et cetera) through Monads really allows me to modularize my code in a clean manner. By doing as much code in 'pure' Haskell as possible, I am more 'sure' of the behavior of my program.

Has anyone else done any serious work in a functional language? I know Jane Street uses O'Caml for this very reason...

Erlang would provide the same benefits, but with that nice built in distributive capabilities (scaling with ease...). After all,"Erlang is the specialist language narrowly focused on networked, highly-scalable concurrent applications." [quoting a LtU post]. A automated trading system really seems to fit the bill there...


As for Clojure/Scala, I wouldn't expect them to outperform C# by too great a margin, given that they still run on the Java platform (though, Java 6 -server is pretty damn fast). Scala still allows you to write code imperatively, so unless you really force yourself to do it the functional way, you aren't getting any benefit from the type-safety. Furthermore, as far as I know, the current JDK doesn't have tail-recursion, which is pretty much a must-have for most functional languages. If you are looking to do some data manipulation with a functional language, check out <a href="http://data-sorcery.org/">Incanter</a>.

My answer is the last line of the post above:

Quote from nitro:
...
Here is the rub though. If you don't have a direct connection to the exchange for market data or orders, it probably makes no difference between C++ and C#! The lag in the all the hops (software, switches, routers, etc) overwhelm language choice. Only in the most professional situations does it matter.

In the case where you don't have direct connections, why not choose what you are most productive in, what reduces the error rate of your coding, what is most fun, etc?

Imo, no one codes in C++ unless they have to, and you do have to if you want to take advantage of every CPU-cycle of the machine. In fact, in many cases you have to drop down to assembler, or code in SSE3 etc...

http://en.wikipedia.org/wiki/Streaming_SIMD_Extensions

 

[propseeker]

Quote from nitro:

Of course it can happen, I have just never seen it __intra-day__. My point is what the heck are you going to trust no matter how many stale redundant systems you have? You might as well flip a coin.

It's like saying, I am going to keep two stale pieces of bread around in case the other one gets stale. See, I am redundant!

it sounds like the api you're familiar with keeps local state... which brings a little clarity to your points/analogy. however, in my experience, most api's do not. my points reflect this generality and the general nature of the topic. perhaps you should consider the specificity of your situation before continued rebuttal.