what is the best language that suits massive calculation on every tick?

mcgene4xpro · Jan 24, 2011

Quote from LeeD:

Regarding platform, I'm sure lots of people would recommend their favourite platform (or the one they are developing/sellling) but hear you need to do your own legwork. Make a shortlist of platforms (remember, platforms that are most popular are those that come free in some form, like NinjaTrader and MetaTrader, not necessarily thsoe most suitable). Search the web, read each platform's forums. Pays special attention to platform starting to lag while tracking a busy symbol like S&P 500 emini during increased market activity. Some platforms are much better than others.

If platform supports 1-tick, 5-tick etc bars, this can make programming a strategy easier.

Given you don't consider yourself a "professional programmer", the advantage of using a "platform" is the support. Datafeed and broker interfaces change frequently. As a platform developer yourself you'll have to follow these changes and check if they affect the operation of your platform, modify the platform... then test, test, test. As an end user it's easier if someone does all of this for you. On the downside, a platform would not support all broker features. One platform might not support price modification in existing order (so in a trailing stop you have to cancel the old stop and place a new one). Other platforms wouldn't support OCO (one cancels other) orders etc...

MQL is "compiled" but it's compiled into bytecode. So, by using a laguage that compiles into native code (C++, Delphi, Power Basic) or a laguge that runs in a virtual machine (C#, VB.NET, Java) you will gain at least 10x speed advanatge.

I believe you can plug DLLs into MetaTrader. This may be an easier first option before swutching the platform.

If you do complex calculations, these often fall into one or more popular classes such as matrix algebra or optimisation. For each class there are popular libraries that will likley do the task a few times faster than the code you write yourself (think of Intel MKL or Boost). If you need one of those, this limits the choice of programming laguage.

Similarly, some trading platforms would work with any compiled or NET laguage (in the form of a DLL) while with others you have to use the one the platfrom developers have chosen.

Further, the highest speed gain is usually obtained by improving your own algorithm. Mone calculations that don't have to be in a cycle outside the cycle. See if some calcualtions don't have to be done every tick etc.

When choosing a programming laguage, consider speed of developement versus speed of execution. 10% - 20% gain of C++ versus C# or vice versa is immaterial because computers are becoming faster alll the time. On the other hand, if you are fluent at the laguage, you can easier do things in a different way, which can easily speed execution up 2 times or more.

No, though it might help. Note most trading platfroms (including popular .NET ones) don't support calculating indicators/trading systems in multiple threads. This means they would use multiple cores for data collection, charting and housekeeping but all indicator/strategy logic will be executed on a single core.

If you want to do parallel calculations yourslef, it's an additionla level of complexity (like thread synchronisation and locking) that you might not want to handle. If you have 4 cores, just run 3 instances of your favourite platfrom (and keep 1 core for the operating system housekeeping).

C++ gives the developer more control over exact execution logic. For example, C# effectively enforces the use of built-in collections (list, hashmap etc). They are great for "general" use but the implementation may be massively suboptimal for your specific task. In C++ you can implement your own collections and finetune them for the specific task you are trying to accomplish.

I would strongly advise againts even considering assembly language. With the complexity of what is happening in moderrn processors the usual effect of an experienced C++ programmer trying to write in assembly languge is a massive loss of execution speed. Optimizing C++ compilers are just that good.

I am reading your thread for the second time now

. Great Thanks for sharing your expertise.

uexkuell · Jan 24, 2011

Quote from mcgene4xpro:

Actually, i am not familiar with MQL4. I have used professional coder to design my ATS.

This might be your central problem.

How can you be sure that the coder is writing code that is
- efficient
- truely doing what you want

(and in the long term: not using the code/the strategy behind if it turns out to be profitable?)

mcgene4xpro · Jan 24, 2011

Quote from uexkuell:

This might be your central problem.

How can you be sure that the coder is writing code that is
- efficient
- truely doing what you want

(and in the long term: not using the code/the strategy behind if it turns out to be profitable?)

I review the roles/formulas we have put and compare to the log out put. However, i cannot be sure 100% that he did or didnot.

LeeD · Jan 26, 2011

Quote from mcgene4xpro:

I am reading your thread for the second time now . Great Thanks for sharing your expertise.

You are welcome.

Quote from mcgene4xpro:

I review the roles/formulas we have put and compare to the log out put. However, i cannot be sure 100% that he did or didnot.

Unless the system has particularly complex logic, you should be able to code the backtest in excel. OK, you will be able to test the system this way on just thousands, not millions of data points but that should be enough to figure out if anything goes wrong.

Splice your tick data so that you have one tick every 30 seconds or even every few minutes. Use these in excel, run teh backtest on exactly the same sample. You would expect the trade times and prices shopuld match exactly. If they don't something is different between what the developer coded and your spreadsheet.

Regarding efficiency, you should have a ballpark estimate what computational complexity is involved. Given an interpreted laguage is involved direct benchmarking is more difficult though (unless you can code a simple test, like a large cycle every tick, with the same complexity in the same MQL).

otcstockfund · Jan 26, 2011

i have something done like what your looking for in cobol if your interested

mcgene4xpro · Jan 26, 2011

Quote from otcstockfund:

i have something done like what your looking for in cobol if your interested

I am..

prophet · Jan 28, 2011

Quote from nLepwa:

Distribute the work, parametrize your strategy and don't give the parameters

That's very bad advice.

I specialize in highly parameterized strategies and optimization frameworks that are pretty good at rapidly solving 100+ dimension parameter spaces. Of course I also rely on intuition and experience. Its not that difficult actually.

The thought of deliberately over parameterizing a strategy for sake of obscurity is a very flawed concept. Simple intuition and statistical analysis will fill in many parameters. Aggressive optimization will do the rest.

mcgene4xpro · Jan 28, 2011

Quote from prophet:

That's very bad advice.

I specialize in highly parameterized strategies and optimization frameworks that are pretty good at rapidly solving 100+ dimension parameter spaces. Of course I also rely on intuition and experience. Its not that difficult actually.

The thought of deliberately over parameterizing a strategy for sake of obscurity is a very flawed concept. Simple intuition and statistical analysis will fill in many parameters. Aggressive optimization will do the rest.

That is exactly what i am afraid from..

Any suggestion?

prophet · Feb 1, 2011

Quote from mcgene4xpro:

That is exactly what i am afraid from..

Any suggestion?

1) Purchase Matlab and devote a solid 3 months to learning to program in it. Why Matlab? Matlab is designed for scientists, engineers, data exploration, rapid model/algorithm development. It has the best and most complete math and visualization capabilities out-of-the-box. That's why it costs $$$. Matlab is also fast to learn because it is very high level and avoids the complexity/verbosity that application developers demand in lower-level languages like Java and C++. If Matlab is too expensive there is also R, Gauss, Octave and SciLab, but many quant/algo developers consider these to be a "poor man's Matlab" because many capabilities are missing, slow or buggy.

2) Collaborate with others in your same situation. Learn from them. You will make mistakes. If you overcome your mistakes and financial losses you may become a successful quant/algo developer. Many of us started with quant/algo trading, lost a lot of money, got discouraged, but came back to it stronger.

3) Start trading your system or shop your systems around. Do not give out your code or even a parametrized version, except under written contract. Try to maintain some advantage. Like keep your best system private but share a less profitable system to test the trust in your partnership.