You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
Best Language for developing a backtesting platform
- Thread starter murrica
- Start date
I meant to say 5-6 million ticks in case that was not apparent.
Quote from hftvol:
So, after you vectorize your back test how are you gonna handle conditional branches? You basically can only vectorize what is repetitive. Anything else you need to loop through. By the way I process backtesting code inside loops at a rate of about 5-6 ticks per second which pretty much blows away any vectorized code you could write in amibroker or R or matlab. I heavily peruse concurrency and parallelization an I run everything in C#. I have not even hit the ceiling I could easily outsource certain matrix computations in some of my correlation strategies to a GPU. So much to c# not being up for the task.
'Quote from hftvol:
So, after you vectorize your back test how are you gonna handle conditional branches? You basically can only vectorize what is repetitive. Anything else you need to loop through. By the way I process backtesting code inside loops at a rate of about 5-6 ticks per second which pretty much blows away any vectorized code you could write in amibroker or R or matlab. I heavily peruse concurrency and parallelization an I run everything in C#. I have not even hit the ceiling I could easily outsource certain matrix computations in some of my correlation strategies to a GPU. So much to c# not being up for the task.
Oh C# is clearly up to the task. It is just a question of what level someone wants to work at.
Agreed that there are definite limitations to vectorizing in R and there are some tasks for which it is not suited.
If one wants to focus a lot on statistical distributions of results from backtesting, then I like R.
One can always pre-compute indicators in C# and feed the results to R.
Some things are best done in C or C# or Java. I use C# or Java a lot for low level stuff.
But R is definitely widely used in finance. See http://quant.stackexchange.com/ and there are a very large number of R users.
If you vectorize, often you have to effectively compute both sides of the branch and throw away one.
But vectorizing is not as bad as it seems as first. APL was enormously popular in finance and it was all-vectors and arrays.
In those cases where backtesting is about massive iteration, maybe C# is better.
But I do not think backtesting should always be about massive iteration.
Rather, I would prefer to test just a few ranges of parameters and explore the statistics of the results.
It is good to avoid losing sight of the forest among all the trees.
But I do not think backtesting should always be about massive iteration.
Rather, I would prefer to test just a few ranges of parameters and explore the statistics of the results.
It is good to avoid losing sight of the forest among all the trees.
Quote from hftvol:
why did you leave out C#/.Net? Its extremely fast to develop, soon the number of libraries for C# will surpass those available in C++, if it has not already.
C++ still rocks and with new VS2012 you have access to C++ AMP which lets you use GPU accelerators that nicely integrate with VS.
I guess that C++ AMP will be available in C# environment few years later.
Anyway any of the .Net IDE is best suited for fast development and project management is the best in its class not mentioning that applications perform extremely well in terms of speed.
I do not even know why people would venture into other languages except for those who do not want to use Windows platform or develop cross platform applications.
When I develop new strategies I make a clear distinction between
* profiling ideas
and
* rigorously testing ideas
I use R/Matlab very extensively for profiling purposes. If an idea looks palatable then I will move to the next stage and run massive (hundreds of millions of tick data) back tests in my C# testing framework. The framework is 100% identical to the live trading systematic architecture with the only difference that the historical data feed is located in-process with the strategy engine in order to avoid an extra serialization/deserialization step otherwise needed to send messages IPC to other modules. Also the execution engine (which incorporates a fill simulator) is in-process when running historical data for the same reason.
There is no way a solution in Matlab or R would even come close to -performance-wise. Even when running Matlab in parallel mode it still cannot reach the throughput I see in my engine. I tested it and that is my experience.
But whatever suits others, use it! Someone was correct in saying that one should use those tools that one is most proficient in and feels most comfortable with. Hard to beat that.
* profiling ideas
and
* rigorously testing ideas
I use R/Matlab very extensively for profiling purposes. If an idea looks palatable then I will move to the next stage and run massive (hundreds of millions of tick data) back tests in my C# testing framework. The framework is 100% identical to the live trading systematic architecture with the only difference that the historical data feed is located in-process with the strategy engine in order to avoid an extra serialization/deserialization step otherwise needed to send messages IPC to other modules. Also the execution engine (which incorporates a fill simulator) is in-process when running historical data for the same reason.
There is no way a solution in Matlab or R would even come close to -performance-wise. Even when running Matlab in parallel mode it still cannot reach the throughput I see in my engine. I tested it and that is my experience.
But whatever suits others, use it! Someone was correct in saying that one should use those tools that one is most proficient in and feels most comfortable with. Hard to beat that.
Quote from comintel:
In those cases where backtesting is about massive iteration, maybe C# is better.
But I do not think backtesting should always be about massive iteration.
Rather, I would prefer to test just a few ranges of parameters and explore the statistics of the results.
There was an extensive test on AMP and while it came out on top, C# performed very close to it despite not using the GPU in that instance. I forget whether it was Dr.Dobbs or another publication. C# does not need to rely on AMP (C# already has libraries to access GPUs) and C# already now gets awfully close in performance to many problems solved in C++. For example array operations in C++ are really not that much faster anymore compared to array operations or even operations on Lists in C#.
But yes if you wanna get the last little performance gain out of your machine one should resort to C++ and not do it on a Windows OS.
But yes if you wanna get the last little performance gain out of your machine one should resort to C++ and not do it on a Windows OS.
Quote from vicirek:
C++ still rocks and with new VS2012 you have access to C++ AMP which lets you use GPU accelerators that nicely integrate with VS.
I guess that C++ AMP will be available in C# environment few years later.
Anyway any of the .Net IDE is best suited for fast development and project management is the best in its class not mentioning that applications perform extremely well in terms of speed.
I do not even know why people would venture into other languages except for those who do not want to use Windows platform or develop cross platform applications.
Quote from hftvol:
When I develop new strategies I make a clear distinction between
* profiling ideas
and
* rigorously testing ideas
I use R/Matlab very extensively for profiling purposes. If an idea looks palatable then I will move to the next stage and run massive (hundreds of millions of tick data) back tests in my C# testing framework. The framework is 100% identical to the live trading systematic architecture with the only difference that the historical data feed is located in-process with the strategy engine in order to avoid an extra serialization/deserialization step otherwise needed to send messages IPC to other modules. Also the execution engine (which incorporates a fill simulator) is in-process when running historical data for the same reason.
There is no way a solution in Matlab or R would even come close to -performance-wise. Even when running Matlab in parallel mode it still cannot reach the throughput I see in my engine. I tested it and that is my experience.
But whatever suits others, use it! Someone was correct in saying that one should use those tools that one is most proficient in and feels most comfortable with. Hard to beat that.
Perfect! I think we would all admire your approach and agree with it!
I have the impression that the original poster is only at the stage of profiling ideas. If I am right, R could work well for him at this stage.
Quote from comintel:
Perfect! I think we would all admire your approach and agree with it!
I have the impression that the original poster is only at the stage of profiling ideas. If I am right, R could work well for him at this stage.
Yup, profiling. I just have to overcome the initial learning curve.. that is the biggest roadbloack for me.
Even using C# would be less of a brainfuck (not the language).