ha, sure and how do you maintain the fx rates required for conversion? What if you require intra-day conversions as well, not for p&l purposes, but if you for example, ran an fx basket strategy based on tick based data? Then you are out of luck because MC would only let you maintain daily fx conversion data. Also, how do you conveniently keep your fx conversion rates updated? This is exactly where the limitations of off the shelf products kick in.
Trading System Development
- Thread starter gvphubli
- Start date
Lol coming from an industry that have been bailed out so many times by taxpayer
and Fed intervention, if anything that is a sign of bloat and abuse of taxpayer money.
Hey you keep moving the goalpost, that is not what your original assertions are.
Kinda like evolution deniers.
If I follow through on volpunter's line of reasoning I guess in the end
I should build my own customized SSD and write a specialized ASIC to process it.
SQL and columnar databases are too slow and for loosers.
I'll send the VHDL to a fab in taiwan and have them ship me back my
ultimate tick storing trading box.
@volpunter it takes you only a few seconds to process an entire days worth of data? I'm using an I7 and it takes about a minute to process the entire NYMEX/CME feed (simulating trades on 10 or so instruments) . I thought that was pretty good, but I must be cache thrashing if it only takes you a few seconds
I trade only currencies and each currency pair generates around 200k ticks per day at maximum (rough ballpark, variations of course subject to which exact currency pair we talk about), my architecture can process around 5-7 million currency bid/offer updates per second (historical data) so you can do the math as function of number symbols and number days...
nobody claimed retail trading is futile. I manage my own life savings and investments and I pretty much am micromanaging each and every detail when it comes to programming implementations, even when I outsource work to independent contractors. But the difference here is that I am not a lazy couch potato, trying to convince others that a 400 dollar software package forms the basis of a prosperous long-term investing and trading career. I put in insane hours at my venture, I spend weekends and evenings to acquire new programming skills because 40-50 percent of my work and the outcome of it deeply depends on the quality of the software that is run.
And if you cannot do the math that it is entirely possible to load and process several million ticks per second off a historical database then it is you who either has a lack of imagination or worse, knowledge of what is doable.
And if you cannot do the math that it is entirely possible to load and process several million ticks per second off a historical database then it is you who either has a lack of imagination or worse, knowledge of what is doable.
Hmm, unless you've developed a new OS and filesystem I've never heard about
then this I find hard to believe.
Anyone using the standard linux kernel fread() will not accomplish this.
Guys Listening to volpunter makes you think that retail automated trading is hopeless,
well it's not, I'm proof of it
I stick to what I claimed. MC does not implement currency conversions. Do you need proof?
Hey you keep moving the goalpost, that is not what your original assertions are.
Kinda like evolution deniers.
If I follow through on volpunter's line of reasoning I guess in the end
I should build my own customized SSD and write a specialized ASIC to process it.
SQL and columnar databases are too slow and for loosers.
I'll send the VHDL to a fab in taiwan and have them ship me back my
ultimate tick storing trading box.
So just BBO ? I suspect my bottleneck has to do with cache thrashing between process shared memory, I'm sharing 10 levels of depth . Should be easy enough to optimize when the time comes
I digress, this is totally the wrong way of thinking for people starting automated trading.
For example
windows is a 200 dollar software package you can find at wallmart, yet it is still the best state of the art software that an average person can install on their pc for everyday purposes like
browsing, office work and gaming.
Make use of readily available tool, don't bother reinventing the wheel for an extra 0.1% edge.
(unless your HFT or stat arbing).
Sure loading, but you said
Load 7 million ticks and do all that in a second on a commodity quad core ?
no way.
yes only best bid/offer as is convention in spot currency markets. For spot currencies no market depth is generally needed unless you either connect with a firm that shows very low liquidity or trade insanely large positions which should be very unlikely in an algorithmic trading framework. Also, keep in mind that currency volume is reported by choice and hence is most often inaccurate at best and only represents the specific venue,only, anyway.
By the way why do you repeat the "cache thrashing" issue? I do not think it has anything to do with overall performance. I create a new struct for each deserialized tick/quote. Deserialization, time stamp sorting are all highly parallelized processes, and so are performance calculations and reporting. The strategies on purpose are not running in parallel because I need exact synchronicity for portfolio back testing purposes.
By the way why do you repeat the "cache thrashing" issue? I do not think it has anything to do with overall performance. I create a new struct for each deserialized tick/quote. Deserialization, time stamp sorting are all highly parallelized processes, and so are performance calculations and reporting. The strategies on purpose are not running in parallel because I need exact synchronicity for portfolio back testing purposes.