Can linear regression analysis really predict the future?
- Thread starter tradrejoe
- Start date
Because if you accept that the markets are Gaussian on the Price/Step plane then at any point of time the probability of the markets to go either up or down is 50/50. That is why any point on the chart picked by any TA method will have the same faith: - 50/50. This is the major difference between TA methods and statistical methods. TA methods are applied to one particular run of the price therefore they are always 50/50 in probability. The statistical methods are different because they deal with the AVERAGE outcome observed on multiple runs. Statistical methods are based on stable distributions and therefore could be used as the solid foundation for the profitable strategies. One could argue that the TA patterns also depict the stable occurrences therefore they are not that different form the statistical methods. It is, however, a wrong conclusion. I will let you discuss WHY as I do not intend to give you all the answers.
Well, I haven't done enough research on the 'price/step' plain to confirm whether or not markets behave in a Gaussian manner -- but I think it is a long shot. By simply changing the step in which price is measured you suddenly transform an auto-regressive, heteroscedastic mean and volatility to being stable? Also, somehow by changing into the price/step plane we completely remove mu and rely only on sigma (which would be your 50/50 of going up or down without a given size of the step)? Seems ... unlikely at best. Does changing to the price/step plane also somehow magically remove the jump process?
Again, as I said -- I think that only certain TA patterns can be used to depict short-term stability in the random-walk's (note, this does not imply GBM -- it could be something far, far more complex) parameters. Channels depict stability in mu and sigma. Trend-lines depict stability in mu. Triangles and pennants depict changes in sigma (though, break-outs of these pennants and triangles raise some interesting questions as to how mu and sigma are interconnected. You can also see that the jump-process (often due to news) also dramatically influences mu and sigma)
I don't need to discuss, as my methods are already profitable for me. I just thought it was an interesting discussion and wanted to jump in the fray.
Corey, I have attached over 20 yrs of S&P500 walks that confirm the empirical data match theoretical avg dev model quite well... until... about 100 steps or so, at which point the avg begins to drift much more positive then the model would suggest. Your notion of drift does not get incorporated into the models earlier data pts. as possibly it is swamped or washed out by the averaging and absolute pre processing step (have to think about it more), however, you can simply use a standard deviation model to confirm the same observation with drift-- shown as sqrt(N*P(1-P)), where p is probability of success trial. The deviation of p from 50% can represent drift (mu), which is a property markets show. There are many other ways to demonstrate proof of drift or positive offset in markets, a great argument for buy and hold in the long run; like it or not (also, another reason why you'll often observe long only strategies tend to perform better than short strategies over the long run).
Secondly, as to why the data starts to deviate after about 100 steps, I believe it has to do with wider (than gaussian) tails in market data beginning to manifest and accumulate (could also be drift accumulating as well). It is similar to the tail behavior you might see in a QQ plot of normal vs. true market behavior. We know that fat tails are more likely to occur about 1/100 events than 1/million (can also see this in power law or 1/f models). So, this is more evidence of my earlier argument that markets are close to gaussian, but worse (wider tails-- jumps, etc).
I would have to think a bit more about the exact incorporation of drift in the simple rw coin toss model Maestro brought up for discussion, although either way it doesn't change my conclusions much.
----------------------------------------
Also, before folks think that this observation is the holy grail to nirvana, keep in mind it is an average of a "VERY" large set of trials (central tendency). So while there is truth in the modeling, the key is that you must establish some type of rule to profit off of it. The information alone does not deviate from anything known in modern literature. It does, however, as Maesto said, open your eyes to many fallacies of common TA if you haven't already established that.
Secondly, as to why the data starts to deviate after about 100 steps, I believe it has to do with wider (than gaussian) tails in market data beginning to manifest and accumulate (could also be drift accumulating as well). It is similar to the tail behavior you might see in a QQ plot of normal vs. true market behavior. We know that fat tails are more likely to occur about 1/100 events than 1/million (can also see this in power law or 1/f models). So, this is more evidence of my earlier argument that markets are close to gaussian, but worse (wider tails-- jumps, etc).
I would have to think a bit more about the exact incorporation of drift in the simple rw coin toss model Maestro brought up for discussion, although either way it doesn't change my conclusions much.
----------------------------------------
Also, before folks think that this observation is the holy grail to nirvana, keep in mind it is an average of a "VERY" large set of trials (central tendency). So while there is truth in the modeling, the key is that you must establish some type of rule to profit off of it. The information alone does not deviate from anything known in modern literature. It does, however, as Maesto said, open your eyes to many fallacies of common TA if you haven't already established that.
FANTASTIC! Now, to correct the drift I can suggest using % values of the current prices rather than the absolute values in points (to specify the step value). The drift occurs due to the change in the probabilities of the steps if the ratio between the step value and the actual price is not taking under consideration. But, overall, FANTASTIC! EXCELLENT JOB!
Of course, % is not the only value that is "probabilistically" acceptable to use...
And the drifts can be isolated as a noise to expose... or it can be diffused if they are relatively singular...
Let the Random (Normal) begin.
Maestro, if I was a chick, I would marry you.
Quote from MAESTRO:
The very last hint I will give you guys is "QUANTUM Random Walk"
Ok, from now on I will only answer PMs from people who will show the progress. Once we establish the circle of interested people we should share the further results privately; after all it is the war out there ...
Quantum Walk???
Never heard of it and here I go again... going through white papers and coding/testing x1000...
Maestro... if you were gay, I'll consider myself as your bitch for the weekend.
Seriously. Thank you.