Building a systematic system part one - data capture

[volpunter]

Hardly any strategy which targets long term holdings is fully systematized for good reason. Your diatribes about long term fundamental strategies hence do not apply here at all.

And sorry I really do not have more time or inclination to argue with you about it. We seem to hold very opposing views in almost everything and your approach just does not seem to make any sense to me at all. If OP or others benefit then I am glad. But I do not derive much value from your seeming lack of logic and coherence. Good weekend to you

indeed, they are also systematic trading strategies

and they need pull data, not push

I am perfectly on topic despite your attempt to derail with your Analyst and news analogy, which I also answered

 

[globalarbtrader]

I've stayed out of this debate, since I learnt to my cost that debating with VP is a painful and pointless exercise so I generally keep him on ignore. However I feel I should correct some misconceptions - since its my thread I actually know what its supposed to be about.

To quote from the original blog post:

I am trading futures, in a fully automated system, which is relatively slow and where latency is not an issue, using only price data."

"Hardly any strategy which targets long term holdings is fully systematized for good reason."
"No billion dollar fund trades fully systematized strategies. "

This clearly isn't the case. Systematic CTA's that I am aware of in the $1bn plus range include Winton, AHL, Bluetrend (now systematica), Cantab, Transtrend, Aspect. There are many more but you can google them yourself I'm sure.

Then outside of the CTA space you have the systematic global macro type guys; there is Bridgewater, nearly $100bn nowadays and FX concepts to name just a couple. Then you have the large systematic equity neutral funds. AHL ran an internal fund like this running at $2bn; and there are quite a few more.

I could go on, and on, and on, but I think I've made the point. And importantly all these places are trading relatively slowly.

Most high frequency shops tend to be small (virtu and a couple of others being some high profile exceptions). The reasons are obvious - you don't need large capital to hold, and many High Frequency strategies don't scale well to much larger positions so can't trade with larger capital. So there probably aren't any multibillion dollar High Frequency funds (although you could argue that DE shaw and Rennisance are exceptions; although I don't really understand eithier business so I don't feel qualified to say); all the big systematic funds I've mentioned above trade slowly.

Its a common misconception amongst the general public that "high frequency trading" / "systematic strategies" / "algorithimic strategies" are all the same thing. High frequency is just one part of the larger systematic trading universe.

"Algorithmic" seems to be have been highjacked by the high frequency crowd, and I strongly disagree with that appropriation.

There is in fact a huge universe of difference between the guy scalping or spoofing with microsecond latency; and the stuff these large systematic funds and myself am doing; so for example on the fastest market I trade my average holding period is 4 weeks. On the slowest its 2.5 months.

The techniques involved in high frequency trading are completely different from the longer latency stuff. I wouldn't presuppose to tell a High Frequency trader who to capture data, since I know nothing about that world. Equally I'm surprised when someone from that other world gives me advice that isn't relevant to me. For example these comments show a clear misunderstanding of the slow trading world:

"algorithmic futures trading is anything but slow"

I disagree, it can be very fast, or it can be incredibly slow.

"Tick based historical data are nowadays very easy to come by so I am not sure why you make the latency of your system a function of the availability of your captured data. You can cheaply purchase high precision futures contract exchange data."

I don't. I trade slowly because that's what I know how to do. Every time I do the maths on the costs of high frequency trading I can't work out how it can be profitable. Most retail punters who try to trade fast lose money. Why should I compete in a game I don't know how to play where the odds are stacked against me?

As a minor point the history of available tick data doesn't go back far enough for my kind of trading. Again the rules are different with slow trading. You need decades of low frequency data to test it, not a few years of tick data.

"Spikes and cleaning -> There should not be different options. When you receive a price of zero or one that lies x standard deviations away from the previously traded prices then that is an erroneous quote, period. You filter it out and are done. "

No, it could be a real price - depending on the value of x there is still a probability (unless you've set x too large so it never triggers). When you are trading relatively slowly you have the luxury of checking to see if its real or not.

Side issue:

Although I generally agree with the comments about fundamental data (I've used factset myself), the post wasn't about that, but about technical data (that would be another post; which I may well do actually). I'm fairly sure that the universe of people deploying relatively slow systematic trading on the back of earnings announcements (a large chunk of the equity neutral world) is much larger than those trying to react to them with low latency feed analysers.

 

[volpunter]

I've stayed out of this debate, since I learnt to my cost that debating with VP is a painful and pointless exercise so I generally keep him on ignore. However I feel I should correct some misconceptions - since its my thread I actually know what its supposed to be about.

To quote from the original blog post:


"Hardly any strategy which targets long term holdings is fully systematized for good reason."
"No billion dollar fund trades fully systematized strategies. "

This clearly isn't the case. Systematic CTA's that I am aware of in the $1bn plus range include Winton, AHL, Bluetrend (now systematica), Cantab, Transtrend, Aspect. There are many more but you can google them yourself I'm sure.

Then outside of the CTA space you have the systematic global macro type guys; there is Bridgewater, nearly $100bn nowadays and FX concepts to name just a couple. Then you have the large systematic equity neutral funds. AHL ran an internal fund like this running at $2bn; and there are quite a few more.

I could go on, and on, and on, but I think I've made the point. And importantly all these places are trading relatively slowly.

all of your mentioned funds do not trade fully systematized fundamental strategies. When I said that no billion dollar AUM shop trades systematized strategies I mean fundamental strategies and systematized means fully systematized. All the funds you mentioned fall into this category. Most of the funds you mentioned use quantitative overlays, use algos for trade execution, or run purely non-fundamental fully systematized funds. But not one firm to my knowledge runs a fully systematized strategy book with fundamental approach.

Most high frequency shops tend to be small (virtu and a couple of others being some high profile exceptions). The reasons are obvious - you don't need large capital to hold, and many High Frequency strategies don't scale well to much larger positions so can't trade with larger capital. So there probably aren't any multibillion dollar High Frequency funds (although you could argue that DE shaw and Rennisance are exceptions; although I don't really understand eithier business so I don't feel qualified to say); all the big systematic funds I've mentioned above trade slowly.

Its a common misconception amongst the general public that "high frequency trading" / "systematic strategies" / "algorithimic strategies" are all the same thing. High frequency is just one part of the larger systematic trading universe.

"Algorithmic" seems to be have been highjacked by the high frequency crowd, and I strongly disagree with that appropriation.

I never made any reference to that. Not sure why you are talking about this here. Nobody touched on this topic.

There is in fact a huge universe of difference between the guy scalping or spoofing with microsecond latency; and the stuff these large systematic funds and myself am doing; so for example on the fastest market I trade my average holding period is 4 weeks. On the slowest its 2.5 months.

The techniques involved in high frequency trading are completely different from the longer latency stuff. I wouldn't presuppose to tell a High Frequency trader who to capture data, since I know nothing about that world. Equally I'm surprised when someone from that other world gives me advice that isn't relevant to me. For example these comments show a clear misunderstanding of the slow trading world:

"algorithmic futures trading is anything but slow"

You misunderstand my point: I criticized the following you said in your blog : " I am trading futures, in a fully automated system, which is relatively slow and where latency is not an issue, using only price data. " -> I latency is very much an issue , why do you think even long only purely discretionary fund hire execution traders and purchase top notch technology and DMA algorithms? Their average holding period is often many months. Any idea why they might care about latency and best execution? This might not apply to you trading small size but execution most always does matter. And hence latency. If it did not matter why are you not using a dial up connection. You admit latency somehow does matter to you, maybe not as much as to a hft but it still does.

I disagree, it can be very fast, or it can be incredibly slow.

"Tick based historical data are nowadays very easy to come by so I am not sure why you make the latency of your system a function of the availability of your captured data. You can cheaply purchase high precision futures contract exchange data."

I don't. I trade slowly because that's what I know how to do. Every time I do the maths on the costs of high frequency trading I can't work out how it can be profitable. Most retail punters who try to trade fast lose money. Why should I compete in a game I don't know how to play where the odds are stacked against me?

Did I dictate the way you ought to trade? I criticized your claim that intra day data is hard to come by, that it is not clean, that its complicated to deal with, and many other factually incorrect points.

As a minor point the history of available tick data doesn't go back far enough for my kind of trading. Again the rules are different with slow trading. You need decades of low frequency data to test it, not a few years of tick data.

"Spikes and cleaning -> There should not be different options. When you receive a price of zero or one that lies x standard deviations away from the previously traded prices then that is an erroneous quote, period. You filter it out and are done. "

No, it could be a real price - depending on the value of x there is still a probability (unless you've set x too large so it never triggers). When you are trading relatively slowly you have the luxury of checking to see if its real or not.

Lol, if you have the luxury of checking then you are NOT fully systematized. So you actually just talk about a quantitative overlay as supporting system architecture? That is not clear in your blog at all.

Side issue:

Although I generally agree with the comments about fundamental data (I've used factset myself), the post wasn't about that, but about technical data (that would be another post; which I may well do actually). I'm fairly sure that the universe of people deploying relatively slow systematic trading on the back of earnings announcements (a large chunk of the equity neutral world) is much larger than those trying to react to them with low latency feed analysers.

Sure it is larger, nobody argued otherwise.

-> You have hardly responded to any of my constructive criticism in response to your blog which unfortunately contains countless errors and factually incorrect comments. If I were you I would definitely rework that blog post because it pretty negatively reflects on your claimed understanding of systematized trading. Especially you seem to have couple issues understanding how different expirations of the same root simultaneously trade, when the majority of trading "rolls" to the further out contract and also your section on data cleansing contains errors. I kindly pointed them out to you while you throw shit into the fan in response.

 

[Butterfly]

I've stayed out of this debate, since I learnt to my cost that debating with VP is a painful and pointless exercise so I generally keep him on ignore. However I feel I should correct some misconceptions - since its my thread I actually know what its supposed to be about.

...

Side issue:

Although I generally agree with the comments about fundamental data (I've used factset myself), the post wasn't about that, but about technical data (that would be another post; which I may well do actually). I'm fairly sure that the universe of people deploying relatively slow systematic trading on the back of earnings announcements (a large chunk of the equity neutral world) is much larger than those trying to react to them with low latency feed analysers.

Well said, and yeah, VP can become quickly aggressive when he is being called on something he has no knowledge of.

I like your points about the HFTs trying to hijack certain terms that have been traditionally used in other context, like quants and systematic. Before the HFTs era, quants and systematic trading were already present in the financial dictionary, nothing has been invented since, but for some reasons, those low latency guys think they have found the wheel and brag about it like it was their extraordinary discovery. How ignorant. Wake up, it has been around before HFT was even in the game.

 

[volpunter]

systematic trading simply means running a fully automatized trading architecture without manual intervention. Fact remains that there is no known fund that trades long-term systematic trading strategies based on fundamental data. It just does not exist and I explained already why. The vast majority of systematized trading strategies run on an event based architecture. You can question my expertise as much as you like. So far you have hardly presented or backed up your claimed knowledge in any of the threads here while I have walked others through detailed data structures, messaging systems, trading architectures, order management systems and statistical analysis. I leave it to the readers of this forum to draw their own conclusions. You nor OP has ever provided evidence of your skill set, professional experience, or training nor details of anything. I will from now on refrain to comment on the OP's approach to systematic trading but warn others, especially beginners, to take his comments with a huge grain of salt, given that he made uncountable errors and mistakes in his blog.

It is on thing to shoot the messenger because you do not like the way he argues or talks, it is an entirely different thing to deny facts and standard practices. Nobody in the hedge fund, sell side, nor buy side arena trades long-term systematized strategies based on fundamental data. It is nonsense and obviously OP chooses to be stubborn, most likely because he has vested interest to gain credibility via his blog posts, imho, he gains only a negative reputation via displaying his rather rudimentary knowledge and almost naive logic and thought process. He does not even understand how rolls are performed in the futures arena nor does he understand how and why multiple contracts in one futures root contract trade, Eurodollar futures for example trade very actively in at least 20 expiration months at any given point in time, if memory serves well then around 40 forward months are provided by CME to trade in.

Well said, and yeah, VP can become quickly aggressive when he is being called on something he has no knowledge of.

I like your points about the HFTs trying to hijack certain terms that have been traditionally used in other context, like quants and systematic. Before the HFTs era, quants and systematic trading were already present in the financial dictionary, nothing has been invented since, but for some reasons, those low latency guys think they have found the wheel and brag about it like it was their extraordinary discovery. How ignorant. Wake up, it has been around before HFT was even in the game.

 

[Butterfly]

systematic trading simply means running a fully automatized trading architecture without manual intervention. Fact remains that there is no known fund that trades long-term systematic trading strategies based on fundamental data. It just does not exist and I explained already why. The vast majority of systematized trading strategies run on an event based architecture. You can question my expertise as much as you like. So far you have hardly presented or backed up your claimed knowledge in any of the threads here while I have walked others through detailed data structures, messaging systems, trading architectures, order management systems and statistical analysis. I leave it to the readers of this forum to draw their own conclusions. You nor OP has ever provided evidence of your skill set, professional experience, or training nor details of anything. I will from now on refrain to comment on the OP's approach to systematic trading but warn others, especially beginners, to take his comments with a huge grain of salt, given that he made uncountable errors and mistakes in his blog.

that's quite rich coming from you when you have never demonstrated yourself anything here, except trumpeting your "so-called" skills.

You don't have access to every trading rooms or Fund managers office, so how can you make such claims except for being completely ignorant. The origin of systematic doesn't come from the electronic trading, but on the concept of systematic rules. You and a few other ignorant players out there think it's all about electronic simply because you have no other knowledge of trading outside electronic, that's why you can't dissociate between systematic and electronic. I suggest you educate yourself a bit about the fundamentals of portfolio management and risk management, maybe then you will get it.

 

[volpunter]

This thread is on systematic trading and you still don't get it.

But hey let's turn things around for a change: mention to us a large fund that runs systematized trading strategies with holding periods in the weeks and months that is based on fundamental data. you have such a big mouth I am sure you can easily come up with one. mention the name please. We are all waiting.

that's quite rich coming from you when you have never demonstrated yourself anything here, except trumpeting your "so-called" skills.

You don't have access to every trading rooms or Fund managers office, so how can you make such claims except for being completely ignorant. The origin of systematic doesn't come from the electronic trading, but on the concept of systematic rules. You and a few other ignorant players out there think it's all about electronic simply because you have no other knowledge of trading outside electronic, that's why you can't dissociate between systematic and electronic. I suggest you educate yourself a bit about the fundamentals of portfolio management and risk management, maybe then you will get it.

 

[Butterfly]

haha, VP getting angry after being exposed for what he is

Systematic trading on Fundamentals was there before you were born and before electronic trading was accessible to "amateurs" like yourself

Sell-side is all about liquidity and market making, which is basically your only experience. But buy-side has been in systematic trading before the appearance of fully electronic trades. The word systematic points to the systematic rules of trading, not their electronic form.

As for naming names, we went through that routine before in the Python thread, where you got owned and exposed again, and when I did name names, you didn't believe me, so what's the point.

Speaking of which, SocGen is now fully endorsing Python as their primary trading tools in trading desk globally. Heard it directly from one of their Head IT Architect.

And yes VP, you are an angry little man :LOL:

 

[volpunter]

Systematic trading is only around for less than 10 years you tool.

Nowhere, including SocGen, is Python used as primary desk trading tool. Who is supposed to use that? Wanna teach traders to program in Python? Even if the Python code was provided by developers then it is just what it is , a frontwnd user interface nothing else. Python is way too slow to ever be used for core computations or to run a backend System. You really have no clue what you are talking about, not even remotely.

I have many years of professional prop trading experience at internal hedge funds. You have no clue about me yet make up lies about a person. That is as low as it gets. Additionally you can search this site for an image of my Master's graduation certificate from Carnegie Mellon. You are resorting to lies just to make a point which I have zero respect for. IGNORE.

haha, VP getting angry after being exposed for what he is

Systematic trading on Fundamentals was there before you were born and before electronic trading was accessible to "amateurs" like yourself

Sell-side is all about liquidity and market making, which is basically your only experience. But buy-side has been in systematic trading before the appearance of fully electronic trades. The word systematic points to the systematic rules of trading, not their electronic form.

As for naming names, we went through that routine before in the Python thread, where you got owned and exposed again, and when I did name names, you didn't believe me, so what's the point.

Speaking of which, SocGen is now fully endorsing Python as their primary trading tools in trading desk globally. Heard it directly from one of their Head IT Architect.

And yes VP, you are an angry little man :LOL:

 

[Butterfly]

...

I have many years of professional prop trading experience at internal hedge funds. You have no clue about me yet make up lies about a person. That is as low as it gets. Additionally you can search this site for an image of my Master's graduation certificate from Carnegie Mellon. You are resorting to lies just to make a point which I have zero respect for. IGNORE.

LOL you are really something VP, and such an angry little character with limited knowledge. There is a far bigger world than prop trading shops in the financial world. Maybe you should get out more.

an image of your Master Degree ? oh boy, how low can you go and idiotic can you be ? who cares about your degree, nobody ask for them, all that matters is what you do and what you know.

Let me summarize your silly claims to evaluate what you know:

1. You have no professional programming skills (by your own admission in an older thread) and yet you lecture everyone on programming patterns and programming design

2. Your financial experience is limited to prop trading and "market maker" hedge funds, and yet you claim you know what everyone else is doing outside your little sphere.

3. You are a pure Microsoftie, that is you know nothing in IT outside MS technologies, and yet you claim Python is inferior for "systematic" trading strategies when in reality every new IT trading projects are now including Python into the pipeline, and moving away from C#.

Wake up, and yes you can put me ignore like you did for everyone who exposed your "intellectual" discrepancies.