Creating own trading system

[hftvol]

agree fully, if you must get up and running quick then rather start profiling and back test strategies in R. Given most do not start out with tick based data, the memory limitations and fact that code in R runs MUCH slower than in compiled languages should not matter much. A lot better than using any backtesting retail package.

"Hobby". Yes of course, anything is fine to serve a "hobby" but as you put it, to succeed well......

Quote from Nab:

Subtract all the fancy (and not necessary) GUI and (BS) TA indicator stuff, and you will see that it takes not long to code up a basic trading engine sufficient for many purposes.

Why bother with tradelink, etc. for backtesting if you can write up tools of >= quality in R, Mathematica, Maple, Mathlab, etc. during a weekend ...

In case you are not able to code up the tools ... maybe this is not the right hobby

You might survive with retail shelf-tools, but not really succeed ...

 

[NetTecture]

Quote from hftvol:

I somewhat disagree. You hardly get around multithreaded applications these days even in trading architecture space. Think about the following: You run one application on core1, containing x trading strategies, then you run more more strategies in app2 on core2. So what if you want to manage a total risk book where decisions made in strategies within app1 impact the risk book and may prevent strategies in app2 from submitting new trades? You cannot realize a true global book with your approach. You could trade each app on each core in a closed container fashion, giving it its own risk caps and available capital but that is hardly considered "scaling". Even if you wanted to report to a global risk book, you would have to start sending messages over a message bus which will greatly slow down your back tests and real trading as well. Thus, I see no point why you would try so hard to avoid a multi-threaded approach where all you have to do is context switching but you would have access to shared object state. You can use concurrent/shared collections in C# such as ConcurrentDictionary to start with. The next step would be to peruse the rich task parallel library and spin off tasks. Then you can go a step further and implement an async/await framework and if you really want to see top notch performance and what .Net 4.5 can deliver then you use TPL Dataflow. That is what I am using and I stand by my claim that I have not seen a single back test engine in C++ that would beat mine in C# in terms of performance, throughput wise. Of course I could do the same as you do and run the same app over and over again, even on different machines but when we are talking backtest you hardly get around synchronization issues because of Risk. Of course it can be done but you would have to make a lot of simplifying assumptions. You could ignore global risk and in a second run actually eliminate trades that may have violated global risk limits because other strategies already reached a certain cap.

You avoid all those with a multi-threaded approach. I prefer it this way a thousand times. I only split up strategies when I trade completely different accounts and hence do not need to share risk caps. Even then I frequently sync risk to a global risk book because it makes zero sense, for example to have 10 trades from 10 different strategies result in a huge exposure to eur/usd, for example.

That is what I meant with stages.This also matches the TPL dataflow idea. But within a stage you are deterministic.

 

[hftvol]

what do you mean with "stages"? I am not familiar with this term.

Quote from NetTecture:

That is what I meant with stages.This also matches the TPL dataflow idea. But within a stage you are deterministic.

 

[j2ee]

Quote from hftvol:

agree fully, if you must get up and running quick then rather start profiling and back test strategies in R. Given most do not start out with tick based data, the memory limitations and fact that code in R runs MUCH slower than in compiled languages should not matter much. A lot better than using any backtesting retail package.

"Hobby". Yes of course, anything is fine to serve a "hobby" but as you put it, to succeed well......

Why Amibroker with
1. take $$
2. Limited AFL language, hard to find out how to do whatever

but not Tradelink with
1. free
2. real C#, 1 of the most used programming language



Welcome to take your time to explain

We assume we are talking about someone takes trading seriously with complicated strategy but not very simple customized indicator.

 

[GeorgeClueney]

Quote from hftvol:

oh yes, and of course you easily extend the language functionality if something is not offered out of the box. Yes you should definitely trade your fund's money and fulfill your fiduciary duty in terms of risk management with Amibroker ;-)

If you don't like a software's own language then there is their free AmiBroker C/C++ SDK available for download! If one doesn't like C/C++ there is another AmiBroker SDK for .NET.

BTW you clearly have no knowledge about the software so it's useless talking to a "knowitall" anyway. If you know better go to their Yahoo forum and try out your expertise there. ;-)

 

[hftvol]

sorry I am done explaining. I thought I made my points crystal clear.

Quote from j2ee:

Why Amibroker with
1. take $$
2. Limited AFL language, hard to find out how to do whatever

but not Tradelink with
1. free
2. real C#, 1 of the most used programming language



Welcome to take your time to explain

We assume we are talking about someone takes trading seriously with complicated strategy but not very simple customized indicator.

 

[dom993]

Hftvol, you have very good arguments in favor of professional-grade platforms, however, you are missing the point ... most ETers are not professional, merely trying to get their feet wet, discover and learn, and there is no point for that purpose to spend the kind of money required by a professional platform, and corresponding historical data.

I will contend that most of the "random" results arising in backtesting with retail-grade platforms are self-inflicted, meaning result from programming errors in the strategy, rather than platform instability, and people would get same random results on a professional-grade platform.

There should be a lot less bugs & much better turnaround in fixing them with professional-grade platforms, than with retail ones. Certainly Ninja doesn't fare well at all on that front, and needs at times quite a bit of sofware development to workaround their bugs, and for some other things just needs to be carefully used so to stay away from those bugs.


J2ee, you have to be blind not to realize that NinjaScripts are plain C#, and that Microsoft .NET software components are available to be used in those "scripts". I know someone will come & say Ninja doesn't support .NET past 2.0, this I don't know, and to be honest I have not encountered any situation where it would make a difference for me.

 

[j2ee]

You explain why you think creating our own system is the best solution over here but you didn't explain why you think amibroker is a good choice in your opinion.

Quote from hftvol:

My own comment re paying up (a multiple of what normal retail platforms cost) or building your own was not aimed only at those who want to trade higher frequencies.

Here is why and not all points apply to each retail platform equally, some may apply some not, but I would say that at least one point applies to most all such platforms (Amibroker, MultiCharts, TS, Ninja, RightEdge, Quant House (the retail platform) and a host of others):

* when you run a back test and each time get different results, no matter with what time compression or over ticks, how much confidence you have in your strategy? Verdict: The platform is useless for career systematized trading.

* when you connect to a broker platform and there are disconnects (nobody can avoid that) and your platform cannot recover each and EVERY single time from such disconnect then your platform is useless for career systematized trading.

* when your platform cannot be easily extended, meaning you cannot plug in a different database, log different events, build a new broker interface and hook it up because you want to trade through a different broker that is not supported then your platform is useless for career systematized trading.

* when you look to chart or analyze any of your strategy or trading results (anything, for example the latency of your fills) and your platform does not allow you to do that internally or you cannot easily export the data for later analysis in R or so then your platform is useless for career systematized trading.

* when you look to trade baskets, statistical arbitrage, many different strategies and want to run them in one book with one global risk management variable set and your platform does not allow for that then it is useless for career systematized trading.

* when you look to trade higher frequency and your platform cannot let you easily back test tick based data , or forces you to store tick based data in a proprietary format that is hard to convert to and disallows you to use the data in any other platform then your platform is useless for career systematized trading.

* and finally for total starters: A platform that only allows you to connect to retail data feeds that are full of missing data, erroneous ticks, unstable in terms of connection, if your platform cannot support a high quality data feed then your platform is useless for career systematized trading.

Of course you may want to get into this game for the sake of the excitement and not because you are serious about beating other market players who spent hundreds of thousands of dollars in hard and software and development. Then I strongly advise you to throw and gamble away your money elsewhere because this is EXACTLY what it is if you are not dead serious about it. Anyone who thinks they pay 500-1000 dollars for a full fledged systematized trading engine and then compete with the rest in the market are obviously insane or delusional.

 

[j2ee]

ok so you agree AFL is limited too.

I agree building own system is the best way but only if someone knows how to build a system very well. That is not even a normal programmer can do.

Quote from hftvol:

I do not know why you are so married to this application but please keep both your feet on the ground. AFL cannot handle the "most complicated algorithms". You need reasonable work-arounds and maybe you show us how AFL handles a matrix inversion (it could not as of 2008, and as mentioned I have not evaluated any subsequent versions) without having to link up to external libraries. Also, how to implement multithreaded paradigms? How do you handle database access to those dbs that are not supported out of the box? How do you connect to unsupported broker APIs? Right, you need to code an interface in a compiled language!!! And how do you implement a simple Kalman or Particle filter? AFL is what it is, a scripting language in order to avoid having to learn a programming language and as it is based on C++ (well the whole engine is) there are components and ideas that look familiar to someone who knows C++. That is all to it. It comes with all the limitations that most other scripting languages suffer from as well. But again how it is implemented is extremely smart and sleek.

 

[j2ee]

Quote from hftvol:

sorry I am done explaining. I thought I made my points crystal clear.

For your own system, what language do you choose and why? Why don't you use something like Deltix?