Test objectively.
Brain is more creative.
Brain is more creative.
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
Is data mining for trading patterns impossible?
- Thread starter bulat
- Start date
Hello:
I just closed out my trading for the day and noticed this thread. I want to mention a couple of things quickly. First, I am glad to see that people have moved toward a practical understanding of what works. There are some good, useable comments regarding characterizing price action. With regard to my previous comments I can offer only a little more, then I have to stop. I think some of the information that traders need to become profitable or to find profitable opportunity is "right in front of you". Look at how predictable markets move intraday, and research the work of people like Cliff Sherry. If you don't like to do the work "by hand" check out the books of George Box and Bill Hunter. Both are excellent mathematicians who have an understanding of how to test and retrieve useful info from samples. I have talked about it before, but to put it in a nutshell, you can make money trading markets that show both stationarity and independence, but you make great money when you find a way to isolate temporal windows caused by non-random behavior. Start with economic reports. OK, led you right up to the door. Today at lunchtime (EST) the markets "turned over". I saw a window of non-random behavior and got short. Because institutions don't want to hold over this weekend, they are exiting with size. They are the ones who will transact at these levels and sustain a move (in this case down). This is just one of many non-random influences that one can find, if they know where to look (is this news to anyone?). You just have to be waiting and get on board.
Good luck everyone.
Lefty
For the attached chart, I have a line through the point where the market turned. Look at how it correlates with the open (bisects the opening price bar). Not that it matters, but I drew the line early in the morning and simply waited for confirmation. Confirmed the trade at 1:20 EST.
I just closed out my trading for the day and noticed this thread. I want to mention a couple of things quickly. First, I am glad to see that people have moved toward a practical understanding of what works. There are some good, useable comments regarding characterizing price action. With regard to my previous comments I can offer only a little more, then I have to stop. I think some of the information that traders need to become profitable or to find profitable opportunity is "right in front of you". Look at how predictable markets move intraday, and research the work of people like Cliff Sherry. If you don't like to do the work "by hand" check out the books of George Box and Bill Hunter. Both are excellent mathematicians who have an understanding of how to test and retrieve useful info from samples. I have talked about it before, but to put it in a nutshell, you can make money trading markets that show both stationarity and independence, but you make great money when you find a way to isolate temporal windows caused by non-random behavior. Start with economic reports. OK, led you right up to the door. Today at lunchtime (EST) the markets "turned over". I saw a window of non-random behavior and got short. Because institutions don't want to hold over this weekend, they are exiting with size. They are the ones who will transact at these levels and sustain a move (in this case down). This is just one of many non-random influences that one can find, if they know where to look (is this news to anyone?). You just have to be waiting and get on board.
Good luck everyone.
Lefty
For the attached chart, I have a line through the point where the market turned. Look at how it correlates with the open (bisects the opening price bar). Not that it matters, but I drew the line early in the morning and simply waited for confirmation. Confirmed the trade at 1:20 EST.
Attachments
OK this is beautiful. Talk about non-random behavior. Any of you looking at CNBC, you saw the "news" that Carl Icahn bought stakes in Rite-Aid, Hewlett-Packard, Siebel. Look a the timing and at the ES chart right now. See the bump up from resistance. Do you think that is coincidence? Big coincidence that the news is released right when the S&P hits that number right? Those of you who have time & sales, look a the blocks that went through right before (and after) that announcement. Those that have access to the Fill or kill pit, take a look there. You know someone was waiting for that one. I should know better than to quit early. See ya
Lefty.
Edit:
That old guy is smart. They keep showing the charts of HP and Rite-Aid. You know the guy made a killing just now. All you had to do was get on that train with small size and add a little bit along the way.
From my point of view this is similar to the thread "trading the FOMC announcement". You can see how it works, but that won't give you the trade. You still have to learn to find the sweet spot and then you have to have the guts to pull the trigger and the nerve to keep your hands off the mouse until you have your profit. Judgement, control, discipline, all the skills have to be there or none of this matters.
Lefty.
Edit:
That old guy is smart. They keep showing the charts of HP and Rite-Aid. You know the guy made a killing just now. All you had to do was get on that train with small size and add a little bit along the way.
From my point of view this is similar to the thread "trading the FOMC announcement". You can see how it works, but that won't give you the trade. You still have to learn to find the sweet spot and then you have to have the guts to pull the trigger and the nerve to keep your hands off the mouse until you have your profit. Judgement, control, discipline, all the skills have to be there or none of this matters.
Quote from flakac:
When you look at so many possibilities, you are guaranteed to find quite a few methods that work incredibly well simply by chance. Even worse, some of these methods will pass any statistical test you throw at it (sharpe ratio, t-test, edge test, correlation test, etc.) since all these test compare the observed results vs what you'd expect randomly. But when you look at a 1,000,000 random samples, you will obviously have some that perform better than 99.999% of random, thus passing any possible test you throw at it.
If you've mined and found 10 systems by looking at 1000 different random relationships (not just optimisations of indicator settings) then you can be 99% sure of your result. I'm saying this half tongue in cheek because when you think about it its a reverse acrary edge test.
If your result shows an Edge Test better than 99% of random or Confidence Interval better than 99% of making a profit then your only remaining question is one of stationarity. How long will the system remain valid? How long has it already been valid?
Quote from toe:
If you've mined and found 10 systems by looking at 1000 different random relationships (not just optimisations of indicator settings) then you can be 99% sure of your result. I'm saying this half tongue in cheek because when you think about it its a reverse acrary edge test.
If your result shows an Edge Test better than 99% of random or Confidence Interval better than 99% of making a profit then your only remaining question is one of stationarity. How long will the system remain valid? How long has it already been valid?
Having conducted this exact experiment, I can tell you that after examining millions of basically random patterns, and finding a whole bunch that are consistently "better than random", I tested the consitently good patterns, on an out of sample time period. About half performed better than random, while the other half performed worse than random. In other words the "better than random" test is fairly useless as a validation criteria for price patterns.
It's definitely not useless. This test is highly related to testing for autocorrelation of returns, and more distantly the Sharpe ratio, win% or profit factor stats. All of these calculations are related mathematically and are very useful for quantitating risk and system character. A robust system (or pattern) will indeed be validated by any of these tests. Autocorrelation has a special advantage in that it may reveal strong negative autocorrelation that can be used for system-of-system designs.Quote from bulat:
Having conducted this exact experiment, I can tell you that after examining millions of basically random patterns, and finding a whole bunch that are consistently "better than random", I tested the consitently good patterns, on an out of sample time period. About half performed better than random, while the other half performed worse than random. In other words the "better than random" test is fairly useless as a validation criteria for price patterns.
As you know, high Sharpe, high autocorrelation systems/patterns are quite hard to find assuming you are testing with a healthy number of trades. Too few trades will give useless, random results regardless of what mathematical formula you use to judge performance. That may be the source of the problem you're having.
However, it is questionable whether any survey of systems or patterns can be made random, in a practical sense. The systems/patterns will always be formed based on specific choices of input data, input normalization, choice of market, basic trade rules, and the time frame. It is unreasonable to suggest these basic factors must always be randomly distributed too.Quote from toe:
If you've mined and found 10 systems by looking at 1000 different random relationships (not just optimisations of indicator settings) then you can be 99% sure of your result. I'm saying this half tongue in cheek because when you think about it its a reverse acrary edge test.
There can be a balance between in-hindsight (in depth) optimization and statistical significance. I have seen both strong testing results, strong production systems and arguments made by profitable system traders that say a large number of trades is the best way to achieve statistical significance and robustness. This assumes execution is properly simulated and all trading costs are figured in.
Quote from bulat:
Having conducted this exact experiment, I can tell you that after examining millions of basically random patterns, and finding a whole bunch that are consistently "better than random", I tested the consitently good patterns, on an out of sample time period. About half performed better than random, while the other half performed worse than random. In other words the "better than random" test is fairly useless as a validation criteria for price patterns.
unfortunately i experienced something similar. i think that people tend to forget that it is not only the way you test something for vailidity (like the edge test), but the way you found that "something" that counts if you try to find future value. i really do not see yet why a curve fitted system should not pass the famous edge test. and the egde test completely ignores aspects like no of parameters if i recall correctly.
actually i still think that alan is supoer"true", but i think his real advantage lies somewhere else, not in the edge test. i tend to think it is the thousands of hours he spend with the markets (basically one top be precise) and with testing hundreds and hundreds of systems. yet i admit i find this answer not completely satisfying either...
peace
Quote from prophet:
It's definitely not useless. This test is highly related to testing for autocorrelation of returns, and more distantly the Sharpe ratio, win% or profit factor stats. All of these calculations are related mathematically and are very useful for quantitating risk and system character. A robust system (or pattern) will indeed be validated by any of these tests. Autocorrelation has a special advantage in that it may reveal strong negative autocorrelation that can be used for system-of-system designs.
As you know, high Sharpe, high autocorrelation systems/patterns are quite hard to find assuming you are testing with a healthy number of trades. Too few trades will give useless, random results regardless of what mathematical formula you use to judge performance. That may be the source of the problem you're having.
totally agreed. it is all different aspects, different "glasses", but it is all looking at the exact same thing. i mainly use ttest, sharpe ratio, hit ratio and profit factor. once in a while i run the edge test. but it usually tells me what i alreday know from the other figures.