Why do I see "Trends" in Randomly Generated Data?

Quote from Jerry030:

Could you suggest what you mean to look for in reading the book?

I have it. So if I know what you comment is directed to it would help.
More...

I don't know which order to give you some pages to read.

It is one of the funniest books a person will ever read.

Read pp 273 to see how niave these guys were by nearly the end of the book.

For kicks also read pp 197 from the paragraph beginning: " We should have focused on this earlier...

These guys have real money and real people are fundung them to do what they do.

There is a theme in the book about how one after another people leave as a consequence of their personal realizations.... These people only fight the system for so long before they quit.... The reasons they quit are kinda compelling in some ways.

Bass was handicapped and just doi9ng what people like him do. Think of Covell, Derman, Mehrling, and Lowenstein as similar birds.

As the read got funnier and funnier I got out little 3M tabs and clocked thirty such things as those mentioned above.

People read this stuff and, apparently, are taken in by the fact that the writers don't really catch on to what is going on. How can Bass write up stuff the way he did if he knew anything?

The TA cutesy is on page 112.
 
Quote from jack hershey:

I don't know which order to give you some pages to read.

It is one of the funniest books a person will ever read.

Read pp 273 to see how niave these guys were by nearly the end of the book.

For kicks also read pp 197 from the paragraph beginning: " We should have focused on this earlier...

These guys have real money and real people are fundung them to do what they do.

There is a theme in the book about how one after another people leave as a consequence of their personal realizations.... These people only fight the system for so long before they quit.... The reasons they quit are kinda compelling in some ways.

Bass was handicapped and just doi9ng what people like him do. Think of Covell, Derman, Mehrling, and Lowenstein as similar birds.

As the read got funnier and funnier I got out little 3M tabs and clocked thirty such things as those mentioned above.

People read this stuff and, apparently, are taken in by the fact that the writers don't really catch on to what is going on. How can Bass write up stuff the way he did if he knew anything?

The TA cutesy is on page 112.
More...

Jack,
As much as I hate Tucson, I have to make the effort to come out there for a few days. You're amazing and I mean that in a positive way.
 
Quote from ProfLogic:

Jack,
As much as I hate Tucson, I have to make the effort to come out there for a few days. You're amazing and I mean that in a positive way.
More...

Please do. Trading starts at 6:30 am local. One more would be fun. The pool hit 92 again today; we are going to reverse the solar.....LOL.

I sign EKG's weekly....
 
Quote from highlifejoker:

</b>Quote from kut2k2:

Here's one:

http://elitetrader.com/vb/showthread.php?s=&postid=1827852#post1827852

My point is that trends can occur randomly or nonrandomly. Don't assume it has to be one or the other.

But the only trends you can rely on are the nonrandom ones. Once you figure out how to spot them, you're on your way to profits. <b>

this makes no sense whatsoever. if you catch a trend, it doesn't matter if its random or non random. in addition, your entry into the alledged non random trend is random per definition so it's a mute point regarding the randomness or non randomness of a trend. no wonder the market is so RICH....
More...
I'll go slowly.

If a trend occurs in a random sequence, you won't know it until after the fact, so you can't "catch it" unless you are a compulsive gambler who always bets with the trend, no matter the source. "Expert" casino craps players bet with the trend and sometimes they win big. But that's nothing compared to the amount of money they've probably lost overall by playing a negative expectation game.

But if the trend is from a nonrandom source, it can be depended on to last for a while or to reoccur if it dissipates. That's the key difference: being able to detect and exploit a nonrandom trend.

As for your claim that entry into any trend has to be random, that's what makes no sense whatsoever.
 
Quote from jack hershey:

Please do. Trading starts at 6:30 am local. One more would be fun. The pool hit 92 again today; we are going to reverse the solar.....LOL.

I sign EKG's weekly....
More...

I get Vitamin D overload out there.
I need clouds and rain occasionally.
I will suffer later this year and force myself, promise.
You need to share the EKG though . . . and put ice cubes in the pool.
 
Quote from MAESTRO:

If you agree that the markets consist of multiple random processes with different types of distributions and those distributions are more or less stable...
More...

Just curious what do you mean by distribution being more or less stable?:) :confused:
 
So basically there are two camps: those who believe in patterns and those who don't.

There are known successful (and not so successful) examples of both sides:

Pattern believers:

Victor Niederhoffer (blew up?)
"Niederhoffer, like Buffett and Soros, was a brilliant man. He had a Ph.D. in economics from the University of Chicago. He had pioneered the idea that through proper statistical analysis of patterns in the market an investor could identify profitable anomalies." (THE NEW YORKER, APRIL 22 & 29, 2002)

Toby Crabel (manages $3.2 billion, Niederhoffer's apprentice)
Wrote a book "Day Trading With Short Term Price Patterns and Opening Range Breakout." (1990)

Monroe Trout (retired, networth ~$100 million, Niederhoffer's apprentice)
"I found that prices were not independent. That is, there were some statistically significant patterns." (The New Market Wizards, interview with Monroe Trout)

Alan Crary (aka acrary)
Says he was influenced by Toby Crabel (http://www.elitetrader.com/vb/showthread.php?postid=137766#post137766) and Monroe Trout (http://www.elitetrader.com/vb/showthread.php?postid=858115#post858115)
Here he talks about non-randomness of the markets (http://www.elitetrader.com/vb/showthread.php?postid=140656#post140656)



Pattern non-believers:

Nassim Taleb
"Taleb, by contrast [to Victor Niederhoffer], has constructed a trading philosophy predicated entirely on the existence of black swans—on the possibility of some random, unexpected event sweeping the markets. He never sells options, then. He only buys them. He’s never the one who can lose a great deal of money if G.M. stock suddenly plunges. Nor does he ever bet on the market’s moving in one direction or another." (THE NEW YORKER, APRIL 22 & 29, 2002)

MAESTRO (I only know that his name is Alex): his posts on this thread show his views on the markets.

Also, acrary said he wasn't pattern oriented trader early in his career (here: http://www.elitetrader.com/vb/showthread.php?postid=150348#post150348)



My own research on trying to find meaningful patterns were fruitless. I've coded the whole pattern mining system. I also took care to remove data mining bias. The input data was daily OHLC data (comparing various data points and ranges, aka "binary identifiers"). The system does find significant patterns, but the main point is that they don't pass out-of-sample tests. I've found a few that persisted, but their occurrence frequency were too low to satisfy my needs. Of course this does not mean there are none patterns, because my search space was limited and I've probably missed some important concept that people mentioned above know.

On the other hand, fat-tails in the markets are obvious. It's easy to prove with a simple excel sheet.
 
Quote from Indrionas:

So basically there are two camps: those who believe in patterns and those who don't.

There are known successful (and not so successful) examples of both sides:

Pattern believers:

Victor Niederhoffer (blew up?)
"Niederhoffer, like Buffett and Soros, was a brilliant man. He had a Ph.D. in economics from the University of Chicago. He had pioneered the idea that through proper statistical analysis of patterns in the market an investor could identify profitable anomalies." (THE NEW YORKER, APRIL 22 & 29, 2002)

Toby Crabel (manages $3.2 billion, Niederhoffer's apprentice)
Wrote a book "Day Trading With Short Term Price Patterns and Opening Range Breakout." (1990)

Monroe Trout (retired, networth ~$100 million, Niederhoffer's apprentice)
"I found that prices were not independent. That is, there were some statistically significant patterns." (The New Market Wizards, interview with Monroe Trout)

Alan Crary (aka acrary)
Says he was influenced by Toby Crabel (http://www.elitetrader.com/vb/showthread.php?postid=137766#post137766) and Monroe Trout (http://www.elitetrader.com/vb/showthread.php?postid=858115#post858115)
Here he talks about non-randomness of the markets (http://www.elitetrader.com/vb/showthread.php?postid=140656#post140656)



Pattern non-believers:

Nassim Taleb
"Taleb, by contrast [to Victor Niederhoffer], has constructed a trading philosophy predicated entirely on the existence of black swans—on the possibility of some random, unexpected event sweeping the markets. He never sells options, then. He only buys them. He’s never the one who can lose a great deal of money if G.M. stock suddenly plunges. Nor does he ever bet on the market’s moving in one direction or another." (THE NEW YORKER, APRIL 22 & 29, 2002)

MAESTRO (I only know that his name is Alex): his posts on this thread show his views on the markets.

Also, acrary said he wasn't pattern oriented trader early in his career (here: http://www.elitetrader.com/vb/showthread.php?postid=150348#post150348)



My own research on trying to find meaningful patterns were fruitless. I've coded the whole pattern mining system. I also took care to remove data mining bias. The input data was daily OHLC data (comparing various data points and ranges, aka "binary identifiers"). The system does find significant patterns, but the main point is that they don't pass out-of-sample tests. I've found a few that persisted, but their occurrence frequency were too low to satisfy my needs. Of course this does not mean there are none patterns, because my search space was limited and I've probably missed some important concept that people mentioned above know.

On the other hand, fat-tails in the markets are obvious. It's easy to prove with a simple excel sheet.
More...

Indrionas,

As any good Quantum physicist would tell you, what the observer sees is dependant on their point of view and the instruments they use in observation. Look at a rock in you hand and it appear solid. View it at a magnification approaching the Plank length and it is 99.999999% empty space and the rest is constantly moving strings of energy. Which is true? They both are as the "truth" is totally dependant on the location of the observer. Your question on the markets is the same. Each side in this question is true as the truth is dependant on the point of view and methods of measurement of the different traders.

I've found that certain markets, such as EUR/JPY can be predicted is a very high level of accuracy: PF > 6 and a total account doubling time < 30 days.

The tools needed to "observe" the patterns that enable this kind of trading are advanced Neural Networks. NNs don't lend themselves to the kind of statistical analysis that most participants use in this kind of debate. They want to define a tidy simple relationship "how often does the market move higher when the close breaks the trend line after a key reversal on a Doji Star, etc, etc". These measures are simple things the human mind can understand since it dreamed them up in the first place. They are far too simplistic to find dependable patterns in the market.

In contrast a NN powerful enough to generate consistently profitable trades may use 30 or 40 variables and each one is interrelated to each of the others by a weight to create a network. The complexity of this structure is indicated by the term "black box"....it can't be understood by the human mind. You can print it out but you get 100 pages of numbers that define these interrelationships and weights but it's not something you can understand.

However a well trained NN can trade with a high level of accuracy which is the essence of successful prediction. So the answer to your question on market predictability depends totally on the skill of the observer in selecting the method of observation and the tools used to measure and organize the elements that make it predictable. Methods and tools that are too simplistic give the correct conclusion that it is not predictable. Those sophisticated enough to capture its complex patterns give the also true answer that that it is predictable. I'm among the latter. But is pointless to ague with folks in the former group as their answer is just a true and mine is.

I’d suggest you forget this theoretical debate and focus on actual trading instead. Contact me privately if you’d like some links to resources on repositioning yourself as the Observer.

Jerry030
 
Quote from Jerry030:


The tools needed to "observe" the patterns that enable this kind of trading are advanced Neural Networks. NNs don't lend themselves to the kind of statistical analysis that most participants use in this kind of debate. They want to define a tidy simple relationship "how often does the market move higher when the close breaks the trend line after a key reversal on a Doji Star, etc, etc". These measures are simple things the human mind can understand since it dreamed them up in the first place. They are far too simplistic to find dependable patterns in the market.

In contrast a NN powerful enough to generate consistently profitable trades may use 30 or 40 variables and each one is interrelated to each of the others by a weight to create a network. The complexity of this structure is indicated by the term "black box"....it can't be understood by the human mind. You can print it out but you get 100 pages of numbers that define these interrelationships and weights but it's not something you can understand.
More...


Hi,

I completely understand what you mean. My input was 200-400 binary variables, derived from OHLC data. And output is a single binary variable (1 indicates probable target behavior tomorrow, 0 indicates "stay out"). I tried both approaches: the "naive" brute-force data mining, and neural network. My experience with neural network is that it either does not converge to the specified accuracy level or overfits to the data and fails miserably in out of sample test.

Another thing to note here is that my inputs and output are binary variables. So it actually does not matter what method of mining you use: brute-force that covers the whole search space, or some "intelligent" heuristic method like neural network or genetic algorithm, that cover only some part of the whole search space.
 
You must log in or register to reply here.
Back
Top