randomness may seem so unrandom...

V

vladiator

Just a thought for the statistically inclined of you...
I was bored this weekend and did smth for fun. I simulated 50,000 random walks 200 time periods each. No drift nothing. Just pure normal/gaussian noise. Then I cumulated the returns and sorted everything according to the cumulative returns as of the 100th period. Not surprisingly, given the size of the sample, there were some that wondered up/down quite a bit halfway through the sample time. Surprisingly, however, when I looked at the cumulative returns of the top/bottom 100 pseudo stocks from that point and on till the end, there was some stong evidence of reversals. The 100 "stocks" that were the biggest gainers as of the 100th interval had an average return that was negative and was economically and statistically significant at all conventional levels. The p-value was like 0.001 or something if I remember correctly. I looked at the losers as of time 100 and they preceeded to gain from that point until time 200. Also statiscially/economically significant. Now, had I not known the series were random, I might have concluded there is evidence of reversals. (overreaction, etc etc). Is there?
Randomness may seem so misleading...
The reason I'm posting this in the TA subsection is b/c I think a lot of what the conventional TA techniques may find is probably in the randomness that seems systematic category and the users should be very careful...
Good luck to all.
 
Sometimes I think 95% of TA (swing time frame) is just random noise blowing around the real force that drives most stocks: fundamentals. Noone likes to talk about it, but stocks with good earnings will generally beat out their ugly counterparts in the long direction using almost any TA methodology.

Fortunately, 5% is market psychology and that can be exploited using TA.
 
Quote from vladiator:

Just a thought for the statistically inclined of you...
I was bored this weekend and did smth for fun. I simulated 50,000 random walks 200 time periods each. No drift nothing. Just pure normal/gaussian noise. Then I cumulated the returns and sorted everything according to the cumulative returns as of the 100th period. Not surprisingly, given the size of the sample, there were some that wondered up/down quite a bit halfway through the sample time. Surprisingly, however, when I looked at the cumulative returns of the top/bottom 100 pseudo stocks from that point and on till the end, there was some stong evidence of reversals. The 100 "stocks" that were the biggest gainers as of the 100th interval had an average return that was negative and was economically and statistically significant at all conventional levels. The p-value was like 0.001 or something if I remember correctly. I looked at the losers as of time 100 and they preceeded to gain from that point until time 200. Also statiscially/economically significant. Now, had I not known the series were random, I might have concluded there is evidence of reversals. (overreaction, etc etc). Is there?
Randomness may seem so misleading...
The reason I'm posting this in the TA subsection is b/c I think a lot of what the conventional TA techniques may find is probably in the randomness that seems systematic category and the users should be very careful...
Good luck to all.
More...


finding just enough order within the chaotic system to profit is what TA seeks. nothng more, nothing less. thanks for the cool research !

best,

surf:)
 
in addition--- what we are seeking by employing TA does not even have to be "real" order within the chaos. " apparent" order will produce profit . profit/loss is the final, ultimate factor.

best,

surf:cool:
 
****************************************************
I agree in the longer term buy/hold strategy...

How would you trade intraday or swing??? Fundamentals work???
****************************************************
Don't do intraday but as far as swing:

Just meant if I backtest a couple 100 "systems" only one will seem
imo statistically significant. I always then ask myself the question:
can this be explained by market psychology? I then test all
parameters to see if tweaking them is also consistent. If not, then I assume I've taken a Random Walk...
 
vladiator,

Your strong evidence of reversion in random price data definitely gave me a laugh. Hmmm.... Sounds like you have found a delightfully contrarian random number generator. The literature is full of examples of the horrible non-random behavior found in many off-the-shelf random number generators ("Numerical Recipes in C" has an excellent discussion of this issue).

But I know that you are right about how easily randomness can create the illusion of potentially profitable patterns. I know that I scared myself silly when I discovered how easy it was to create trading systems that seemed profitable on random data. As an intellectual exercise in due diligence, feeding random price data into a trading system is a prudent test of the system. Unfortunately, the psychological impact of doing so can be devastating when one discovers how easy it is to generate accidental (and unrepeatable) random profits.

On the other hand, jmcgraw's recent contest to identify random vs. real price series was very instructive. (See The challenge and The results) What was interesting is that most traders did indeed successfully discriminate between real and random price data sets. Either real price data contains nonrandom structure or the chosen random process was not realistic enough. (I do know that my simple test for real vs. random could be easily fooled by a more sophisticated, yet random price data generator.)

Trade well,
Traden4Alpha
 
What you describe here is related to the Levy Law discovered rather recontly (towards 1959 if my memory doesnot fail) which describes the persistence of chance or unfortune. Although the common sense would believe that during a very long play, periods of gain of each player is near 1/2 in the case of two players - to simplify - the most probable value for one player is rather 0 or 1. That is too say, there is probably always somebody who accumulate the chances against the others, this seem to show that Nature is not so fair but favors somehow injustice.

The consequence is some managers thinking they are talentful are not, and others thinking they are bad, are not perhaps more bad or more good than the former :).


Quote from vladiator:

Just a thought for the statistically inclined of you...
I was bored this weekend and did smth for fun. I simulated 50,000 random walks 200 time periods each. No drift nothing. Just pure normal/gaussian noise. Then I cumulated the returns and sorted everything according to the cumulative returns as of the 100th period. Not surprisingly, given the size of the sample, there were some that wondered up/down quite a bit halfway through the sample time. Surprisingly, however, when I looked at the cumulative returns of the top/bottom 100 pseudo stocks from that point and on till the end, there was some stong evidence of reversals. The 100 "stocks" that were the biggest gainers as of the 100th interval had an average return that was negative and was economically and statistically significant at all conventional levels. The p-value was like 0.001 or something if I remember correctly. I looked at the losers as of time 100 and they preceeded to gain from that point until time 200. Also statiscially/economically significant. Now, had I not known the series were random, I might have concluded there is evidence of reversals. (overreaction, etc etc). Is there?
Randomness may seem so misleading...
The reason I'm posting this in the TA subsection is b/c I think a lot of what the conventional TA techniques may find is probably in the randomness that seems systematic category and the users should be very careful...
Good luck to all.
More...
 
If you think more thoroughly it isnot so contrary to common sense because why Nature would do justice ? This would mean some intentionality of social intervention from nature :)
 
Quote from Traden4Alpha:

vladiator,

Your strong evidence of reversion in random price data definitely gave me a laugh. Hmmm.... Sounds like you have found a delightfully contrarian random number generator. The literature is full of examples of the horrible non-random behavior found in many off-the-shelf random number generators ("Numerical Recipes in C" has an excellent discussion of this issue).

But I know that you are right about how easily randomness can create the illusion of potentially profitable patterns. I know that I scared myself silly when I discovered how easy it was to create trading systems that seemed profitable on random data. As an intellectual exercise in due diligence, feeding random price data into a trading system is a prudent test of the system. Unfortunately, the psychological impact of doing so can be devastating when one discovers how easy it is to generate accidental (and unrepeatable) random profits.

On the other hand, jmcgraw's recent contest to identify random vs. real price series was very instructive. (See The challenge and The results) What was interesting is that most traders did indeed successfully discriminate between real and random price data sets. Either real price data contains nonrandom structure or the chosen random process was not realistic enough. (I do know that my simple test for real vs. random could be easily fooled by a more sophisticated, yet random price data generator.)

Trade well,
Traden4Alpha
More...

Yep, it turned out to be a problem with the generator (I wrote the program in VBA within MS Excel. I assumed for 10 mil numbers or so their generator should be OK. When I got the result I was amazed. I re-ran the code and got very similar results. It wasn't funny any more... Statistical significance was there and the series were independent. I thought I was either going nuts or I was onto smth interesting... Then I decided to rerun the code filling in the numbers cross-sectionally (not time-wise as I did at first) - I I'd first generate the 50,000 time one noises, then time two and so on, instead of generating one random walk then the next one.
The funny contrarian result vanished. There must be something with the Excel random number generator that I bumped into.
Luckily I didn't spend more time on it. I was getting quite excited and was about to bug my wife who coded her own super neat generator in C++ for her physics department a couple of years ago... :D
It was fun though...
What would have been really nice whould be if it was some spurious autocorrelation that was attributable to incorrectly specified test statistics. If that was the case, that would cause quite some noise among academics... Too bad :D I'll have to get famous with smth else :D
Good trading to you all.
PS I did look at the contest you mentioned. The graphs look a bit different (not in terms of oscillations but just the bars themselves.) I didn't participate not to make a fool of myself. Indeed I would have. I picked all but one that looked different from the rest, but I wasn't sure which ones were which. And my hunch was the opposite of what the truth turned out to be, just eyeballing them. So had I participated, I'd have gotten a 1/10 :D
 
You must log in or register to reply here.
Back
Top