You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
Simultaneous calculations, programmatic question
- Thread starter mcgene4xpro
- Start date
Quote from mcgene4xpro:
in parallel
That's brilliant lets not overwhelm with details....
Is your spec running x parallel threads each performing 1000-10000 operations.
Or running 1000-10000 parallel threads, each performing x operations?
What is x?
What is simple-moderate mathematical operations? is it the operations defined here: http://download.oracle.com/javase/6/docs/api/java/lang/Math.html or does it require access to data structures (array, map etc..).
Java on a regular current-gen quad core should be more than sufficient to handle most cases unless your spec really is to run 1000-10000 parallel threads....
not sure what to say about using nvda gpu to run ats..never done it, never seen anyone do it professionally...
Quote from byteme:
You haven't really provided enough info to give a sensible answer though.
Do all of the calculations have to finish before the next tick arrives? Or can the calculations arrive some time later in a continuous stream as fast as new ticks come along but with an initial delay?
All calculations have to finish before the coming of the next tick. The idea is my trading logic dealing with large number of trades " 100-1000" trades and each has separate conditions to be calculated. So, each trade has to be considered separately. That is why i would like to achieve this parallel massive calculations. Sorry, if my terms are not programmatically sound 100%. I am a trader and not a programmer.
As i said, i am using an MQL4 programmer and he tried but we noticed this problem. However, with using less trade number, things becomes better and no conflict was occured. However, the optimal situation for my logic is to trade massive number of trades. That is why i am thinking to migrate to a higher langauge " C++ or Java" and using some API to connect with brokers.
Anyway, i have to stay with MQL4 for a while because i invested a lot of time, efforts and some money on it and i wish to see some solid and acceptable code before migrating.
The trading system could be applied on a very high frequent trading environment but i have to work with real quantitative pro and real programming nerds and having some more money to invest on this monster. However, i have a plan in my mind.
Till this , i have to keep with the lowest cheapest stupid langauge/MT4 platform/brokers for a while till i feel it is really has a realistic potential then i could invest up to 250k on this once i see/prove it works.
The problem here is i am a trader more than a programmer or quantitiative.. I have the basic core idea and i am trying to build on it with very simple and cheap tools. Very hard but it is really an art.
Thanks
McGene
With a little care your goal is easily doable. I typical modern Intel CPU at 3 GHz process a math instruction in a few clock cycles (let's say 3 on average). So you have about 1billion instructions per second to play with. While things like SQRT and sin(x) take longer it's not computational capacity that will keep you from running at your objective. Poor programming and limited understanding of concurrency can easily squander the capability of your CPU.
Quote from newguy05:
That's brilliant lets not overwhelm with details....
Is your spec running x parallel threads each performing 1000-10000 operations.
Or running 1000-10000 parallel threads, each performing x operations?
What is x?
What is simple-moderate mathematical operations? is it the operations defined here: http://download.oracle.com/javase/6/docs/api/java/lang/Math.html or does it require access to data structures (array, map etc..).
Java on a regular current-gen quad core should be more than sufficient to handle most cases unless your spec really is to run 1000-10000 parallel threads....
not sure what to say about using nvda gpu to run ats..never done it, never seen anyone do it professionally...
Ok, the excel theoretical work shows that the ATS could put trade every 0.1 pip " in case of EU". Practically, this is too difficult due to the real broker/communication limitations. The inherent latency will prevent me to do this. Also, theoretically, each trade could be fragmented into the lowest trade size possible, this means more trade number. This is if i keep working with just one instrument, you could imagine if i work with many currency and stock instruments how many trades i will have to deal with.
So the answer is : running 1000-10000 parallel threads, each performing x operations and X here could be 1 to 10.
Each trade has a separate calculation sequence. In short, i could control the number of trades i would like to play. However, the more, the better.
Now, the calculations for now is generally is simple as your first example:http://download.oracle.com/javase/6/docs/api/java/lang/Math.html
Thanks
McGene
Quote from vikana:
With a little care your goal is easily doable. I typical modern Intel CPU at 3 GHz process a math instruction in a few clock cycles (let's say 3 on average). So you have about 1billion instructions per second to play with. While things like SQRT and sin(x) take longer it's not computational capacity that will keep you from running at your objective. Poor programming and limited understanding of concurrency can easily squander the capability of your CPU.
So are you saying it is achievable even with MQL4 and all what i need to get a better programmer and a decent hardware?.
My laptop is Dual-core 2GHZ 2GHZ with 4G ram. but still on backtesting, it cannot handle. I think it might be the code itself and the programing quality. I will try to look deep into this possibility
+1 vikana
To the OP, i dont know why you think it takes so much investment. All you need to do is
1) Sign up for a professional dedicated hosting server (pick one with its data center located close to your broker's). Those run about $100-300 a month.
2) Code your strategy and deploy the binary to the said hosting server
3) Code a simple backtest simulator, basically spits back historical tick data in random x-y milsec intervals.
4) Hook your ats to listen to the backtest simulator, and see the result and fine tune.
The hardware / startup cost etc.. is inconsequential. Just hire a java thread expert. A full time consultant of such skillset from one of the big indian firms run you 150k-250k onsite (annual). If you have one shipped here from india, it's about 100-150k.
Just make sure they actually knows concurrency well and can code it, not just recite the java handbook chapters...
To the OP, i dont know why you think it takes so much investment. All you need to do is
1) Sign up for a professional dedicated hosting server (pick one with its data center located close to your broker's). Those run about $100-300 a month.
2) Code your strategy and deploy the binary to the said hosting server
3) Code a simple backtest simulator, basically spits back historical tick data in random x-y milsec intervals.
4) Hook your ats to listen to the backtest simulator, and see the result and fine tune.
The hardware / startup cost etc.. is inconsequential. Just hire a java thread expert. A full time consultant of such skillset from one of the big indian firms run you 150k-250k onsite (annual). If you have one shipped here from india, it's about 100-150k.
Just make sure they actually knows concurrency well and can code it, not just recite the java handbook chapters...
Quote from mcgene4xpro:
All calculations have to finish before the coming of the next tick.
Can you define what you mean by "tick"? Because "tick" is going to vary a lot depending on where you are getting your data feed from. There are 10Gbps feeds coming out of some places, which (at least for now) puts an upper bound on how fast you need to go, but that won't be the same "tick" you see coming from eg IB across a DSL connection.
Quote from mcgene4xpro:
Ok, the excel theoretical work shows that the ATS could put trade every 0.1 pip " in case of EU". Practically, this is too difficult due to the real broker/communication limitations. The inherent latency will prevent me to do this. Also, theoretically, each trade could be fragmented into the lowest trade size possible, this means more trade number. This is if i keep working with just one instrument, you could imagine if i work with many currency and stock instruments how many trades i will have to deal with.
So the answer is : running 1000-10000 parallel threads, each performing x operations and X here could be 1 to 10.
Each trade has a separate calculation sequence. In short, i could control the number of trades i would like to play. However, the more, the better.
Now, the calculations for now is generally is simple as your first example:http://download.oracle.com/javase/6/docs/api/java/lang/Math.html
Thanks
McGene
Ok but that doesnt really mean you are literally running 1000-10000 parallel threads if each thread's function is just 1-10 basic java math functions. To keep it on the high level, the thread scheduler will continuously run 4 threads (for quad 4 core cpu) while other threads wait. To avoid overhead of creating and destroying many threads, you will create a thread pool. With java 1.5, it's much easier to manage threads, there are many ways to optimize it. Based on what you said so far, java should easily accomplish this. And i dont really think each of those "trade" has to be a separate thread.
And you can always brute force it by running separate java processes on multiple servers.
Latency issue will be minimized by running your ats from a professionally hosted data center as mentioned above (instead of your laptop off a wifi connection lol ).