System / Setup Design process
- Thread starter liquidtox
- Start date
Onelot, thanks for pointing out ACrary's post. I love that idea. Thanks much!
Hi,
I'm not contradicting anybody here that look for correlations or patterns since if you have read some of my posts (for example this one http://www.elitetrader.com/vb/showthread.php?threadid=14288) you know that I have a deterministic model so trends and patterns really exist for me. But let's pretend that I don't have this model, inside a stochastic framework, I would need more than just visual conviction that there is a trend or pattern. You talk about "good indicator", "pattern" and "trade environments" all these terms are qualitative. For example the famous head and shoulders can be found also in random process that is one of the reason statisticians still deny that there is any proof of pattern. If you have read Nassim Taleb he seems to have the opinion that pattern traders trade with illusions in mind. Some traders are arguing: I make money with patterns it is the proof that patterns exist. No it is the proof that they can make money for this period POINT. That pattern exist is not proved by making money with them. In statistics there is the phenomena of hidden variables well known in psychology research field. Many psychological tests have been disqualified after critical examination of their methodology because a researcher wants so willingly prove his thesis that he eludes some other hypothesis (consciously or more often inconsciously if he is honest): correlation is different from causality.
As an illustration I will take the acrary post which has the merit of describing in details his procedure but from methodological point of view there are some flaws. I will do so later.
I'm not contradicting anybody here that look for correlations or patterns since if you have read some of my posts (for example this one http://www.elitetrader.com/vb/showthread.php?threadid=14288) you know that I have a deterministic model so trends and patterns really exist for me. But let's pretend that I don't have this model, inside a stochastic framework, I would need more than just visual conviction that there is a trend or pattern. You talk about "good indicator", "pattern" and "trade environments" all these terms are qualitative. For example the famous head and shoulders can be found also in random process that is one of the reason statisticians still deny that there is any proof of pattern. If you have read Nassim Taleb he seems to have the opinion that pattern traders trade with illusions in mind. Some traders are arguing: I make money with patterns it is the proof that patterns exist. No it is the proof that they can make money for this period POINT. That pattern exist is not proved by making money with them. In statistics there is the phenomena of hidden variables well known in psychology research field. Many psychological tests have been disqualified after critical examination of their methodology because a researcher wants so willingly prove his thesis that he eludes some other hypothesis (consciously or more often inconsciously if he is honest): correlation is different from causality.
As an illustration I will take the acrary post which has the merit of describing in details his procedure but from methodological point of view there are some flaws. I will do so later.
Quote from NorskTrader:
harrytrader,
My hunch is that if you've got a good indicator, you'll see a pattern that is too consistent across many trade environments to leave you comfortable with the null hypothesis of randomness there, too.
Great discussion so far guys.
Harry, I think you are taking this down the "academic" path. I personally don't care if the patterns that my brain sees exist or not. If I can make money training my brain to see flying purple turtles, then by golly I'm going to look for the turtles. This isn't an academic exercise for me (though that part is interesting). I am here to make money. Know what I mean?
I do agree with you though to a certain extent. I think what some people do is look for patterns, but their subconscious is also looking at *other* things to determine if it is a good trade or not. That's why some of these gurus that make money with patterns can't teach other people to be successful with their patterns. The students don't have their subconscious minds trained to see the extra things that the guru sees. The pattern is really just a framework for the guru.
So the question for the guru is, is the pattern valid or not? I say the pattern is valid if his bank account is growing.
Harry, I think you are taking this down the "academic" path. I personally don't care if the patterns that my brain sees exist or not. If I can make money training my brain to see flying purple turtles, then by golly I'm going to look for the turtles. This isn't an academic exercise for me (though that part is interesting). I am here to make money. Know what I mean?
I do agree with you though to a certain extent. I think what some people do is look for patterns, but their subconscious is also looking at *other* things to determine if it is a good trade or not. That's why some of these gurus that make money with patterns can't teach other people to be successful with their patterns. The students don't have their subconscious minds trained to see the extra things that the guru sees. The pattern is really just a framework for the guru.
So the question for the guru is, is the pattern valid or not? I say the pattern is valid if his bank account is growing.
Liquidtox:
I think it boils down to observation, and adaptation. My style is more in the camp belonging to price pattern setups using support/resistance, and trend lines. This is so because it stems from some of my basic beliefs of the market that 1) S/R is important and 2) price patterns reflect interim struggle between buyers and sellers and how one eventually prevails over the other and starts pushing prices in their favor in a more sustained manner. 3) Once price starts being pushed in a direction, it gives rise to the characteristics of the trend. So, yes, you can say that my beliefs give rise to the tools I choose to express in my methodology.
The price pattern setups I use work well under certain market conditions and not others. There are instances when I see extended dry periods where my setups simply donât show up at all. In my attempts to address these, the market condition (e.g. trending, ranging, chopping) becomes part my criteria in addition to my criteria imposed on the validity of the pattern. If you were to compare the intra-day movement of ES on 2/18 with 2/21 for example, they are two different animals. In the ongoing development of my system, I try to look for new patterns or variations of old patterns that tend to appear under those different market conditions. Many fall to the wayside, worthless to my system; but any new âhopefulsâ must ALWAYS be consistent with my basic beliefs before it is admitted into my repertoire of usable patterns. My take is that there are countless market patterns (and indictors as well) out there that have potentially money-making setups; no doubt about it. But if it doesnât fit my framework, then itâs not my playing territoryâ¦yet.
But finding a collection of patterns to look for in the market as opportunities is just the beginning and IMO, only a small piece of the trading picture. There are of course integral elements like position size, getting out of losses cheaply if the trade fails, getting back in, letting the successful trades run, pyramiding, etc, that have a much larger impact on the outcome of the trade; and indeed over a series of trades.
A useful metaphor I am learning is that the evolutionary development of a working system is a vehicle for adapting to the constantly changing market. But ultimately, I am the one having to drive that vehicle and plough through the market to make money. So the driver has to adapt too.
Kermit
I think it boils down to observation, and adaptation. My style is more in the camp belonging to price pattern setups using support/resistance, and trend lines. This is so because it stems from some of my basic beliefs of the market that 1) S/R is important and 2) price patterns reflect interim struggle between buyers and sellers and how one eventually prevails over the other and starts pushing prices in their favor in a more sustained manner. 3) Once price starts being pushed in a direction, it gives rise to the characteristics of the trend. So, yes, you can say that my beliefs give rise to the tools I choose to express in my methodology.
The price pattern setups I use work well under certain market conditions and not others. There are instances when I see extended dry periods where my setups simply donât show up at all. In my attempts to address these, the market condition (e.g. trending, ranging, chopping) becomes part my criteria in addition to my criteria imposed on the validity of the pattern. If you were to compare the intra-day movement of ES on 2/18 with 2/21 for example, they are two different animals. In the ongoing development of my system, I try to look for new patterns or variations of old patterns that tend to appear under those different market conditions. Many fall to the wayside, worthless to my system; but any new âhopefulsâ must ALWAYS be consistent with my basic beliefs before it is admitted into my repertoire of usable patterns. My take is that there are countless market patterns (and indictors as well) out there that have potentially money-making setups; no doubt about it. But if it doesnât fit my framework, then itâs not my playing territoryâ¦yet.
But finding a collection of patterns to look for in the market as opportunities is just the beginning and IMO, only a small piece of the trading picture. There are of course integral elements like position size, getting out of losses cheaply if the trade fails, getting back in, letting the successful trades run, pyramiding, etc, that have a much larger impact on the outcome of the trade; and indeed over a series of trades.
A useful metaphor I am learning is that the evolutionary development of a working system is a vehicle for adapting to the constantly changing market. But ultimately, I am the one having to drive that vehicle and plough through the market to make money. So the driver has to adapt too.
Kermit
Again, good points, harrytrader.
For the record, though, it may be that my use of the terms "pattern," "trade environments," and the like was confusing. I did not mean the price chart patterns that traders frequently identify, but rather the patterns of correlation between the individual percentage performance (y-axis) of a large number of trades arrayed against an ordinal sequence of the pre-entry values of a given indicator (x-axis).
Then, as for the term "trade environments," I meant that only in the objective sense, using completely observable quantitative metrics, with no room for subjective interpretation.
Then, as for your second paragraph in the quote above, I again wholeheartedly agree. Correlation does not imply causation. Nevertheless, let's also remember which enterprise we're engaged in. In some disciplines, such as medicine, confusing correlation and causation could lead to deadly results. Here, though, if we have a consistent correlation over a large number of trades between an indicator and trade performance, does it matter if the indicator isn't causative? Isn't it sufficient for our purposes that it signals a probabilistic advantage or disadvantage for our trade, whatever the cause? Echoing Biomech, isn't it the equity curve that counts?
I don't believe you should look to your backtesting software to deliver any new system, the best computer in the world is between your ears. I would look at charts with all sorts of indicators until I had what looked like a promising system then use the backtester to prove or disprove or categorize it as working on some type of market.
The very best system I found does some things that are absolute anathema to what some experts say should be done so I would just start with a blank slate and try everything, keep what works for you.
Max
The very best system I found does some things that are absolute anathema to what some experts say should be done so I would just start with a blank slate and try everything, keep what works for you.
Max
You're wrong it is not only a question of academic : it's also a question of knowing if you're trading with a ROBUST system or not. You can have the impression that your system works because of some patterns whereas in fact it was because of a context you didn't see (a context is something people aren't aware but it is like the backbone : even if you don't see it without it your skeleton will not stand by) or another factor. Concretly if you have a robust system and you know it by PROVING IT you will never wonder those kind of questions I have read (and it is just one of many of that kind):
"Date: Thu Feb 20, 2003 11:59 pm
Subject: Anyone else having a tough time?
The system I use to trade the e-minis successfully for the last 6
months has absolutely fallen apart in the last 3 sessions.
It has been worthless!
I am trying to figure out why and was curious if anyone else that
normally does well is having a tough go lately."
Another interesting one:
"I believe my system is pretty reliable but the last few months has
been very difficult....just no follow through.
12/02 and 1/03 ended profitable but were very difficult trading
months.
2/03 has been the toughest month in 2 years...."
So if you say :
"the question for the guru is, is the pattern valid or not? I say the pattern is valid if his bank account is growing"
I will say:
the important question is "WILL it continue ?"
"Date: Thu Feb 20, 2003 11:59 pm
Subject: Anyone else having a tough time?
The system I use to trade the e-minis successfully for the last 6
months has absolutely fallen apart in the last 3 sessions.
It has been worthless!
I am trying to figure out why and was curious if anyone else that
normally does well is having a tough go lately."
Another interesting one:
"I believe my system is pretty reliable but the last few months has
been very difficult....just no follow through.
12/02 and 1/03 ended profitable but were very difficult trading
months.
2/03 has been the toughest month in 2 years...."
So if you say :
"the question for the guru is, is the pattern valid or not? I say the pattern is valid if his bank account is growing"
I will say:
the important question is "WILL it continue ?"
I have talked of Robustness. Robustness is the key in Science and Engineering. You are not obliged to "design" a system, you can just build a system by empirism (experience) without "designing" in the sense without thinking before acting. In fact humanity and our brain has always acted before thinking: it is a natural path of doing things. Think of people who have created vapor machines : they were able to do so even before Sadi Carnot discovered the Entropy Theorem. But with this history perspective example you can foresee the big difference between empirism or technology alone and Science. After Sadi Carnot, because people could now understand the true nature of vapor or energy they were able to make machines that don't explode for any apparent reason after having worked until then 
. They don't just create machines they design them before on paper before creating them for real: they think before acting whereas before they act before thinking. It is an iterative process, one must use both ways, I just put the accent on the other way because it is not the way people usually see.
In Quality Management Robustness is the master concept: it is called Quality Assurance. Quality Assurance doesn't mean you have to throw every bad product on the contrary it means you must avoid to do so by having DESIGNED (THOUGHT or CONCEIVED) THE SYSTEM BEFORE CREATING IT WITH THIS GOAL. That is to say Performance is not the main goal, performance will be a consequence of a robust system. People usually think that performance is antinomy of robustness. It is true if you try to optimise the system alone. But if you design the system for robustness before you will have a skeleton and whatever optimisation you will do the skeleton should prevent you from overoptimisation if it is well designed. You know if it is well designed if only you prove it. I can't detail because that would need an entire book
We can even generalise to ethical problem of science: today many people confuse technology with Science. This is very dangerous: technology proceeds with only goal optimisation. This will lead to kind of high speed but fragile society.
. They don't just create machines they design them before on paper before creating them for real: they think before acting whereas before they act before thinking. It is an iterative process, one must use both ways, I just put the accent on the other way because it is not the way people usually see.In Quality Management Robustness is the master concept: it is called Quality Assurance. Quality Assurance doesn't mean you have to throw every bad product on the contrary it means you must avoid to do so by having DESIGNED (THOUGHT or CONCEIVED) THE SYSTEM BEFORE CREATING IT WITH THIS GOAL. That is to say Performance is not the main goal, performance will be a consequence of a robust system. People usually think that performance is antinomy of robustness. It is true if you try to optimise the system alone. But if you design the system for robustness before you will have a skeleton and whatever optimisation you will do the skeleton should prevent you from overoptimisation if it is well designed. You know if it is well designed if only you prove it. I can't detail because that would need an entire book
We can even generalise to ethical problem of science: today many people confuse technology with Science. This is very dangerous: technology proceeds with only goal optimisation. This will lead to kind of high speed but fragile society.
I'm not entirely disagreeing with you here, Harry. Ideally one would understand exactly why their system is successful. I think a system can be robust though without understanding exactly why and how it works.
I don't have the scientific background that you guys do, but I do have a strong Comp Sci background. It is impossible to understand every detail of a million line program, and it is impossible for that program to work exactly as you intended, but with some extremely thorough testing, you can be 99.9% sure that your program will perform as designed. Also, even the best program requires maintenance. Business conditions change, just as market conditions change. The program must constantly be adapted just as a trading system must be tweaked to handle current market conditions. I'll bet these quotes you have of people's systems falling apart were not constantly monitored and tweaked with and re-backtested etc. to keep their market perspective fresh.
As to your second and third points, I would say there is no way that ANYONE can know that there system will continue. Show me a system that you think will be valid for the next 100 years. All you can do is do as thorough testing as your time allows for as large a number of variables as you can muster, regardless of whether or not you think you understand exactly why your pattern works (which is an entirely different discussion). Then you must constantly monitor it to make sure it is still relevant to the current market.