Why do I see "Trends" in Randomly Generated Data?
- Thread starter Rahula
- Start date
Maestro,
I do like your thinking style.
Any good chart reader can determine that there is a beautiful young lady in the picture. If they can't, then they just don't "have the gift."
---------------------------------------------------------------------
"Patterns are the fool's gold of financial markets. The Power of chance suffices to create spurious patterns and pseudo-cycles that, for all the world, appear predictable and bankable. But a financial market is especially prone to such statistical mirages. My mathematical models can generate charts that - purely by the operation of random processes - appear to trend and cycle. They would fool any professional "chartist."... They are the inevitable consequence of the human need to find patterns in the patternless."
Mandelbrot
I do like your thinking style.
Any good chart reader can determine that there is a beautiful young lady in the picture. If they can't, then they just don't "have the gift."
---------------------------------------------------------------------
"Patterns are the fool's gold of financial markets. The Power of chance suffices to create spurious patterns and pseudo-cycles that, for all the world, appear predictable and bankable. But a financial market is especially prone to such statistical mirages. My mathematical models can generate charts that - purely by the operation of random processes - appear to trend and cycle. They would fool any professional "chartist."... They are the inevitable consequence of the human need to find patterns in the patternless."
Mandelbrot
Two points. First, this is not my Random 1000. Second, how did you generate bar charts? That presumes a time frame, and I made no assumptions in this regard. I was hoping to see what purely random price action of up, down or sideways on a line chart would look like over a period of about 2,000 or so data points.
Just wanted to say I agree with ProfLogic and Spydertrader that existing price (and volume) historical data can be used to profitably anticipate future price movements. By "historical" in some cases this can be as far back as seconds, I guess some people would call that present, so use whatever definition makes sense.
I sat in a chatroom with ProfLogic a few years ago for a few sessions and observed his live trading for a few hours during one of these. While the trade sample was pretty small (10 trades or so, I'd want to see 100 to have higher statistical confidence of course), his performance was very good. He gave the technical reason behind each trade, which was in sync with his trading methodology as he taught it. As I was a beginning student, I didn't understand all of the reasons, but in my brief interaction at least the trades I did understand I was completely in sync with ProfLogic on. I would need to see more samples to be fully sure but my preliminary thoughts are ProfLogic has the ability to profitably trade using nothing but Price/Volume history for the specific contract/equity in question (without using intermarket data, options data, index arbitrage related data, fundamental data, etc).
I've been reviewing Spydertrader's and Jack Hershey's threads and haven't spent enough time to make any definitive statement, but for various reasons I believe it is extremely promising. I hope to produce some definitive research eventually (automated testing of 1000s of trades on what Hershey calls beginner/forest-level Point 3 trades). I'm pretty far off on starting and completing this for various reasons but if I get to the point of completion, I'll post the results with their permission.
In my personal trading, I don't want to go into explicit detail, but I am trading spot currencies in some cases using purely historical Price data. I will say very few of the ideas I test out actually forward test or trade profitably. It usually takes me researching at least 30 ideas to generate 1 successful trading strategy that delivers actual profits. In my 6+ years I've only discovered 3 strategies that have worked consistently (in real time) out of at least 100. There are an additional 10 or so that I've found that forward-test profitably, but on a poor risk-adjusted basis so I don't use them, or otherwise haven't had time to develop, test, and operate them properly.
I absolutely believe, based on my personal research as well as spending some time with various others, that trading using purely historical price history can be successful. At the same time, finding such strategies can be extremely frustrating (which is a double edged sword; I think the high challenge level is my single favorite aspect of trading).
So people are asking for Price-related edges and if it's possible, here's another one just for additional food for thought. This one is being used today by a small trading group and has generated over a $1M/year the last few years, I think they have about 3 people dedicated to trading it intraday. Again I won't go into the specifics for various reasons but the basic idea is this: you find algorithms to detect fat-fingered trades and then you fade them. These are trades that move the mid-cap equity to a stddev event of 3 or higher, and meeting other characteristics. To be successful you need to scan thousands of these in real-time (the group spent over 500k developing the software to assist in this, I think it might be difficult for individuals traders to develop this in most cases). The idea is they fade this pattern, and 95% of the time the exchanges don't reverse the trade since these are smaller equities, not ES etc. It isn't quite risk-free but it's extremely close, the way they have set things up. Anyhow, much like index-arbitrage this particular one isn't anything many people don't know about, it really comes to execution in terms being able to execute it faster than almost anyone else. But it certainly is an example that illustrates a difference between market data and randomly generated price data.
The bottom line is there are people and firms who use purely historical price/volume data to profitably anticipate future price movements. There are thousands of repeatable, profitable patterns in market data that are not consistently found in randomly generated price data. Randomly generated price data will not show the extremely highly consistant patterns such as fat-fingered trades that can be profitably exploited by some people.
I sat in a chatroom with ProfLogic a few years ago for a few sessions and observed his live trading for a few hours during one of these. While the trade sample was pretty small (10 trades or so, I'd want to see 100 to have higher statistical confidence of course), his performance was very good. He gave the technical reason behind each trade, which was in sync with his trading methodology as he taught it. As I was a beginning student, I didn't understand all of the reasons, but in my brief interaction at least the trades I did understand I was completely in sync with ProfLogic on. I would need to see more samples to be fully sure but my preliminary thoughts are ProfLogic has the ability to profitably trade using nothing but Price/Volume history for the specific contract/equity in question (without using intermarket data, options data, index arbitrage related data, fundamental data, etc).
I've been reviewing Spydertrader's and Jack Hershey's threads and haven't spent enough time to make any definitive statement, but for various reasons I believe it is extremely promising. I hope to produce some definitive research eventually (automated testing of 1000s of trades on what Hershey calls beginner/forest-level Point 3 trades). I'm pretty far off on starting and completing this for various reasons but if I get to the point of completion, I'll post the results with their permission.
In my personal trading, I don't want to go into explicit detail, but I am trading spot currencies in some cases using purely historical Price data. I will say very few of the ideas I test out actually forward test or trade profitably. It usually takes me researching at least 30 ideas to generate 1 successful trading strategy that delivers actual profits. In my 6+ years I've only discovered 3 strategies that have worked consistently (in real time) out of at least 100. There are an additional 10 or so that I've found that forward-test profitably, but on a poor risk-adjusted basis so I don't use them, or otherwise haven't had time to develop, test, and operate them properly.
I absolutely believe, based on my personal research as well as spending some time with various others, that trading using purely historical price history can be successful. At the same time, finding such strategies can be extremely frustrating (which is a double edged sword; I think the high challenge level is my single favorite aspect of trading).
So people are asking for Price-related edges and if it's possible, here's another one just for additional food for thought. This one is being used today by a small trading group and has generated over a $1M/year the last few years, I think they have about 3 people dedicated to trading it intraday. Again I won't go into the specifics for various reasons but the basic idea is this: you find algorithms to detect fat-fingered trades and then you fade them. These are trades that move the mid-cap equity to a stddev event of 3 or higher, and meeting other characteristics. To be successful you need to scan thousands of these in real-time (the group spent over 500k developing the software to assist in this, I think it might be difficult for individuals traders to develop this in most cases). The idea is they fade this pattern, and 95% of the time the exchanges don't reverse the trade since these are smaller equities, not ES etc. It isn't quite risk-free but it's extremely close, the way they have set things up. Anyhow, much like index-arbitrage this particular one isn't anything many people don't know about, it really comes to execution in terms being able to execute it faster than almost anyone else. But it certainly is an example that illustrates a difference between market data and randomly generated price data.
The bottom line is there are people and firms who use purely historical price/volume data to profitably anticipate future price movements. There are thousands of repeatable, profitable patterns in market data that are not consistently found in randomly generated price data. Randomly generated price data will not show the extremely highly consistant patterns such as fat-fingered trades that can be profitably exploited by some people.
I believe in neither. But I do believe in keeping abreast of current market action.
Quote from MAESTRO:
No matter how do you define the âTRENDâ the implications of your definition with regards to trading are the same. You want to guess as accurate as possible based on the observed data whether the market will go up or down. So what ever you used to define the trend will lead to two conclusions:
#1 I am in an âUp-Trendâ therefore I believe it will continue to go up
#2 I am in an âUp-Trendâ but I believe it is about to reverse.
The same things, of course you can say about the Down trend. But it is crucial to understand that at any given point of time the probability of the first two statements is still 50/50. Because if it was not, the Markets would not exist. The only reason for the markets to exist is to discover the most âfairâ price of what ever is trading there. If the trend had any meaning at all the prices would then be predictable and, therefore the markets would destroy them selves. What I am simply saying that in the long run the outcome of any strategy that relies on Price vs. Time type of data is â0â minus spreads and commissions paid. In a short run the equity line might be positive (as any normal random walk) but it will with no doubts revert to â0â in the long run. In other words, using Price-Time data only (regardless of the type of indicators you run on it) is like driving the car only looking at your rear view mirror.
Answering, therefore, to the original observation âWhy do we see trends in the purely random data?â brings everything home. The reason is our psychological necessity to associate the unknown with something that we can relate to. Any Shaman knows this trick very well. So, the answer is not in the patterns but in the reaction of people to them. And this info is available. Its too bad only few of us can see it!
Thanks, I can wait. And if you can't find it, that's okay. But if you do, then please confirm what the random generating criteria are, since I'm specifically interested in an equal chance of a 1-tick up move, a 1-tick down move or no change. I don't even want a bar chart because that requires further assumptions of a random nature or otherwise. A line chart would be most pure and basic, I would think, for illustrative purposes.