Can linear regression analysis really predict the future?
- Thread starter tradrejoe
- Start date
As frequent as possible. The frequency is directly responsible for the "smoothness" of your equity curve. If you can afford to trade 200 times a day and make 1 tick after all the expenses on each trade then even a very small skew in the distribution (51/49) will allow you to make reliable profits. The frequency is the most desirable characteristic of your trading. Then, of course, there are many limitations such as cost, time, slippage, fills etc. That is why large hedge funds have their computers "parked" directly at the exchange with very little commissions and lightning speed of grabbing the trades.
I guess I concur with most of what you're saying here. In fact it's not much in contradiction to what I posted earlier, to which you initially sounded quite "contrarian". It appears you're not _that_ contrarian.
Though I persist in thinking that more often that not, in specific moments, prices tend to "have memory" and behave the same way when presented the same context - and that just can't be explained by plain luck, sorry.
In terms of prices (I mainly focus on index futures), at a high level I would describe their evolution in time as a superposition of several phenomenons, "signals" if you will:
1- a high frequency, low amplitude "noise". This is basically bid vs ask come and go moves, this "noise" has 1 to a few ticks of amplitude and is essentially present all the time
2- higher amplitude, slower paced volatily "waves": these would describe the natural ebb flow after a price has moved in one direction it tends to reverse course partially, never goes in a straight line, whatever your timeframe. Amplitude would be the avg amplitude of 1 bar on yr time horizon
and either of, depending on... who knows what:
3a- a trend underlying signal, essentially a linear type signal, up or down, with a varying slope and duration depending on the strength, the motivation behind that trending force
3b- a sinewave type underlying signal, essentially responsible for trade ranges, whose amplitude is linked to volatility and other (??) factors. Phase shifts/modulation cause the cycle to not be necessarily exactly periodic
3c- random price "shocks", large pulses or even oscillations, mostly happening after major economic news or unforseable events etc... Totally non periodic, nearly impossible to predict in terms of when and how much
Well, that's a simplified view of course, but that's the way I see it (from quite a distance ^^).
I'm pretty much in simulating "stuff" (never in bed though
), in fact following our discussion on random walks I'll probably try to spend some time generating some charts based on simple binomial distribution rules initially, then evolving towards a more sophisticated model, e.g. introducing some level of randomization of the step size (in the random walk) and maybe some other "signals" to mix in as per my description above.Just out of curiosity... Since you mentioned it, I want to check and see my double tops and triple bottoms right there!
When you are ready to step up from coin tosses and binomial dist., you can download this free simulator from wolfram based on stable distributions (worse than normal and closer to markets!).
Remember, these are random series based on some underlying random distribution. Seems like we tend to repeat these discussions over and over here, don't we? I see some double bottoms there, anyone else?
One problem with high frequency trading is that you better hope your signal (capital) and edge are large enough to swamp out the noise (slippage/comm) you will encounter. Actually, it's even worse than that, commission is not noise, it's a constant negative bias!
Plot cumulative commission vs. n trades and observe how it works against you as freq increases. Retail traders must consider trade off between high freq. benefits (central tendency approaching expectation in short time frame), and freq*costs (brokers love this). These realities must be taken into consideration as you approach shorter time/increased freq.
Remember, these are random series based on some underlying random distribution. Seems like we tend to repeat these discussions over and over here, don't we? I see some double bottoms there, anyone else?
One problem with high frequency trading is that you better hope your signal (capital) and edge are large enough to swamp out the noise (slippage/comm) you will encounter. Actually, it's even worse than that, commission is not noise, it's a constant negative bias!
Plot cumulative commission vs. n trades and observe how it works against you as freq increases. Retail traders must consider trade off between high freq. benefits (central tendency approaching expectation in short time frame), and freq*costs (brokers love this). These realities must be taken into consideration as you approach shorter time/increased freq.
One question - have any of you taken a daily chart of the dow and put a 20 ema and 50 ema on it.
Look at it for the last two years.
If that is random - I guess that fact that I closed out a trade up 8 percent instead of down 85 was due to the fact my brained tricked me into see a pattern that has been working for years.
I really enjoyed reading about the brain and how we interpret signals around us - but I think you all a refusing to see that patterns which do exist.
seriously put up the 50 ema - look at all the times the market bounced at tell me that market is random.
Please - used your very best argument.
I would love to read it.
Look at it for the last two years.
If that is random - I guess that fact that I closed out a trade up 8 percent instead of down 85 was due to the fact my brained tricked me into see a pattern that has been working for years.
I really enjoyed reading about the brain and how we interpret signals around us - but I think you all a refusing to see that patterns which do exist.
seriously put up the 50 ema - look at all the times the market bounced at tell me that market is random.
Please - used your very best argument.
I would love to read it.
There's your first problem. I haven't even looked at that sample, but cherry picking is not a good way to draw conclusions.
I'm assuming you are referring to crossover type systems, although turning points suffer the same lag problem.
I long ago concluded that the basic 2 optimized average cross type systems don't fare to well in practice over the long run (Although they look good if you capture just the right region, or zoom way out to obscure actual activity). Here's another example of what your mind sees looking at the average crossings, but on further inspection tells something slightly different.
Kudos if it worked great for you many years, but my experience doesn't match that. To be fair, I haven't run many large sample tests using EMA, but I don't expect them to perform that much better than standard MAs due to reasons mentioned above (the zoom in effect of EMA shown above, is similar to the effect you see on simple MAs).
A bit more food for thought...
Not only did he reach similar conclusions, but he also found (as I did) that the common crossover strategies perform so bad, that you can often obtain net positive results by flipping the conventional rules!
I believe this result has something to due with the averaging lag effect, but won't elaborate further.
Also, the sample was something like 5 years (prefer more), but I've seen similar results over different sample periods.
There is a blog out there where he shows that the very long term s&p500 optimized MA cross barely beats buy & hold over the long run (which wasn't the case until the recent meltdown, which made up for a lot of inferior crossovers), and although he argues the merits of MA cross, he conveniently forgets to mention short term tax effect, arguing it is irrelevant.
Secondly, it doesn't make much sense to compare the long run, since you are trading short periods. I find it better to compare large sets of small sample periods.
Not only did he reach similar conclusions, but he also found (as I did) that the common crossover strategies perform so bad, that you can often obtain net positive results by flipping the conventional rules!
I believe this result has something to due with the averaging lag effect, but won't elaborate further.
Also, the sample was something like 5 years (prefer more), but I've seen similar results over different sample periods.
There is a blog out there where he shows that the very long term s&p500 optimized MA cross barely beats buy & hold over the long run (which wasn't the case until the recent meltdown, which made up for a lot of inferior crossovers), and although he argues the merits of MA cross, he conveniently forgets to mention short term tax effect, arguing it is irrelevant.
Secondly, it doesn't make much sense to compare the long run, since you are trading short periods. I find it better to compare large sets of small sample periods.
I need much more explanation on this. In 2d space, splines are impossible, but I'm trying to picture that 3d space moving through time and how that looks but have a rough go of it as you use it here. Each time stamp is part of the 3d shape? How does that change in the next snapshot, then smoothing it out? I can see some usefulness in a trading system, but am really unsure of how a security with only price and volume as an inidcator can be used here. Will you please elaborate?
that was exactly the answer I expected.
Instead putting the average up and taking a look. You told me I was cherry picking and telling me that you could not fit a cross over system to it.
I am just telling you to take a look and see if those bounces are random looking to you.
Go back 10 years I am sure you will see it worked all the way back to the time they had to use simple moving averages.
How do I know... my p&l tells me.
But really - take a look at the chart see how many times the dow traded down to the average and moved off it and tell me it is random.
In addition to all the things that brain has to do in terms of looking for patterns. One of the other things we have to do is to listen to our professors and other acadmics and determine real experience from academic myth.
And this is an acid test.
Can someone look at all those bounces and claim that action looks random.
(by the way the Federal Reserve found that currency markets exhibited a tendency to bounce around certain technical points as well.).
Instead putting the average up and taking a look. You told me I was cherry picking and telling me that you could not fit a cross over system to it.
I am just telling you to take a look and see if those bounces are random looking to you.
Go back 10 years I am sure you will see it worked all the way back to the time they had to use simple moving averages.
How do I know... my p&l tells me.
But really - take a look at the chart see how many times the dow traded down to the average and moved off it and tell me it is random.
In addition to all the things that brain has to do in terms of looking for patterns. One of the other things we have to do is to listen to our professors and other acadmics and determine real experience from academic myth.
And this is an acid test.
Can someone look at all those bounces and claim that action looks random.
(by the way the Federal Reserve found that currency markets exhibited a tendency to bounce around certain technical points as well.).