Why do I see "Trends" in Randomly Generated Data?

[Jerry030]

Quote from originalskunk:

"Predictions about the future based upon similarities to the past are worthless"
Edward Allen Toppel
Zen in the markets

Toppel is a classical mathematician and in the world of classical mathematics he is correct. However his perspective is limited by his world view. We all are. Much of what we think we perceive is determined by our understanding and beliefs, prior to any outside evidence or stimulus. If his point that a set of linear equations won't find predictable patterns, I agree. They are much too primitive to do that.

However I suspect he has little experience with soft computing, neural networks and the like. They are complex enough to find patterns that have predictive value.

Physics is a good analogy. The Newtonians thought in the late 1800's that it was time to find a new profession since they thought everything to discover in physics more or less had been discovered. Then came Einstein, Quantum Mechanics and well, the worldview changed and there was more to discovery.

On that note I'll share that predictable conditions discovered with neural network models seem to obey the Heisenberg Uncertainty Principal. For those not familiar it states that the two primary characteristics of any particle, its locations and velocity are mutually exclusive in that the more precisely you determine its location the less you can know about its velocity and visa versa. So if one is more important you have to pick what you want to know with the realizations that you will loose accuracy on measuring the other.

In my experience in the markets, trade frequency and trade accuracy are like position and velocity in particle physics. You can design a system that has very high levels of one or the other but not both. The more trades you accept the lower the accuracy. The higher the accuracy you want the lower the number of trades you will have.

So those that dream of a high accuracy system that trades every 10 bars will never find it. They can get modest accuracy in terms of Profit Factor but it will never be exceptional. On the other hand if you are willing to have only an occasional trade, say every 25 to 40 bars, extremely high levels of accuracy are possible. The best approach, I think, is to track many markets and go for the high accuracy and low frequency in short term trades.

Jerry030

 

[Jerry030]

Quote from ProfLogic:

Famous and infamous people make uninformed quotes all of the time.

640K ought to be enough for anybody… -- Bill Gates, 1981

A low voter turnout is an indication of fewer people going to the polls… -- Dan Quayle

Airplanes are interesting toys, but they have no military value… -- Marshal Ferdinand Foch in 1911

And there's no doubt in my mind, not one doubt in my mind, that we will fail… -- George W Bush

You have some good quotes there, but I'd leave out the Bush stuff. He is not a knowledgeable person saying something stupid, but a stupid person saying something stupid. That is normal and average for that kind of person.

 

[TraderSystem]

Quote from TraderSystem:

When you flip a coin, it can happen that you get heads 5-6 times in a row; so it looks like a trend; but then it is not so, it is more of a luck or pure chance. We need to see large samples from various traders and let someone analyze the same, to remove bias, before we can state with confidence that a trend exist.

It seems that the above comment was wrong, am sorry if I did not understand the concept. I shall try to read the thread again to understand better. But, if someone gave some real life examples it would be so much easier to understand.

 

[PitchBlack]

Quote from Jerry030:

Toppel is a classical mathematician and in the world of classical mathematics he is correct. However his perspective is limited by his world view. We all are. Much of what we think we perceive is determined by our understanding and beliefs, prior to any outside evidence or stimulus. If his point that a set of linear equations won't find predictable patterns, I agree. They are much too primitive to do that.

However I suspect he has little experience with soft computing, neural networks and the like. They are complex enough to find patterns that have predictive value.

Physics is a good analogy. The Newtonians thought in the late 1800's that it was time to find a new profession since they thought everything to discover in physics more or less had been discovered. Then came Einstein, Quantum Mechanics and well, the worldview changed and there was more to discovery.

On that note I'll share that predictable conditions discovered with neural network models seem to obey the Heisenberg Uncertainty Principal. For those not familiar it states that the two primary characteristics of any particle, its locations and velocity are mutually exclusive in that the more precisely you determine its location the less you can know about its velocity and visa versa. So if one is more important you have to pick what you want to know with the realizations that you will loose accuracy on measuring the other.

In my experience in the markets, trade frequency and trade accuracy are like position and velocity in particle physics. You can design a system that has very high levels of one or the other but not both. The more trades you accept the lower the accuracy. The higher the accuracy you want the lower the number of trades you will have.

So those that dream of a high accuracy system that trades every 10 bars will never find it. They can get modest accuracy in terms of Profit Factor but it will never be exceptional. On the other hand if you are willing to have only an occasional trade, say every 25 to 40 bars, extremely high levels of accuracy are possible. The best approach, I think, is to track many markets and go for the high accuracy and low frequency in short term trades.

Jerry030

There are several very important statements in your posting, with which i agree:

However his perspective is limited by his world view. We all are. Much of what we think we perceive is determined by our understanding and beliefs, prior to any outside evidence or stimulus.
Physics is a good analogy. The Newtonians thought in the late 1800's that it was time to find a new profession since they thought everything to discover in physics more or less had been discovered. Then came Einstein, Quantum Mechanics and well, the worldview changed and there was more to discovery.

Most people do not realize that their judgement is based on their limited knowledge. Limited means that you cannot exclude that things you don’t understand still can exist and/or work. Things that seemed impossible to me when I started trading years ago, are now possible because my knowledge about trading has improved.

I personally experienced that you cannot take everything, that others say, for granted. Especially not if they say that certain things are impossible. When I was in my very early twenties I wanted to start up my first business. So I spoke to several people in my family and nobody was really enthusiastic about my plans. The most heard statement was: what you want to do is impossible, because if it wasn’t, someone else would have done it already a long time ago. I didn’t listen to them and started up my first business. It was an overwhelming success and I sold my business already after 1 year because I already had a new plan to start up. My net profit, cash on the bank, was equal to 20 years of salary that I made at that time. But there was 1 big difference: my profit was cash on the bank, for the 20 years salary I would still have to work 20 years and the net profit would be almost nothing because I would also have 20 years of expenses to pay for. At that time it was already clear to me: listen to others but do what you think you have to do, even if it is the opposite of what all the others say.

In my experience in the markets, trade frequency and trade accuracy are like position and velocity in particle physics. You can design a system that has very high levels of one or the other but not both. The more trades you accept the lower the accuracy. The higher the accuracy you want the lower the number of trades you will have.

My personal experience is that 2 trades a day give a much higher accuracy than 25 trades a day. But it has all to do with the approach. I only trade when “I only have to bow and pick up the money that lies on the ground” ( not a quote from me). When you trade 25 times a day you cannot be exigent, so the risk of inaccuracy is much bigger, and the profit is most of the time much smaller.

 

[Jerry030]

Quote from TraderSystem:

It seems that the above comment was wrong, am sorry if I did not understand the concept. I shall try to read the thread again to understand better. But, if someone gave some real life examples it would be so much easier to understand.

TraderSystem,

You won't find any real life stuff on this thread. Sadly, it's mostly theory and dueling quotes. The debate could have ended long ago by simply posting a price history data set and taking the following steps:

1) Let the "Randomites" test it and certify that all sequences and relationships are random and have no causative relationship or predictive potential.

2) Let the "Predictors" take the certified random data set and apply their algos and models to find predictive elements and see what kind of Profit Factor can be obtained

3) Then the Ramdomites can offer proof that such finding are illusionary.

4) Then the Predictors can trade what they found in real time and post the signals along with their brokerage statements.

5) The real life results would then end the discussion one way or another.

Heck, this gives me an idea for a fresh thread on Elite Trader!

Jerry030

 

[kut2k2]

Quote from TraderSystem:

Quote from TraderSystem:
When you flip a coin, it can happen that you get heads 5-6 times in a row; so it looks like a trend; but then it is not so, it is more of a luck or pure chance. We need to see large samples from various traders and let someone analyze the same, to remove bias, before we can state with confidence that a trend exist.

It seems that the above comment was wrong, am sorry if I did not understand the concept. I shall try to read the thread again to understand better. But, if someone gave some real life examples it would be so much easier to understand.

Here's one:

http://elitetrader.com/vb/showthread.php?s=&postid=1827852#post1827852

My point is that trends can occur randomly or nonrandomly. Don't assume it has to be one or the other.

But the only trends you can rely on are the nonrandom ones. Once you figure out how to spot them, you're on your way to profits.

 

[highlifejoker]

The entire premise of markets being driven by mass human emotion and TA being reflective of this emotion is GARBAGE and OUTDATED by many years.

Markets are NOT DRIVEN BY mass EMOTION. Markets are driven by the movement of capital, much of which is controlled by program trading and large pools aka hedge funds. One man today controls the market moving power of 10000 in the past, and if you think your squigly lines and pretty candles can predict what the BIG BOYS and PROGRAMS are doing you are DELUSIONAL.

With the premise of TA DESTROYED, why do people keep on believing? Cause they have no INFORMATION and have NOTHING else to hang onto a sliver of hope.....

TA ers your trading days are numbered...


bottoms up, socrates.....




HLJ

 

[Jerry030]

Quote from highlifejoker:

The entire premise of markets being driven by mass human emotion and TA being reflective of this emotion is GARBAGE and OUTDATED by many years.

Markets are NOT DRIVEN BY mass EMOTION. Markets are driven by the movement of capital, much of which is controlled by program trading and large pools aka hedge funds. One man today controls the market moving power of 10000 in the past, and if you think your squigly lines and pretty candles can predict what the BIG BOYS and PROGRAMS are doing you are DELUSIONAL.

With the premise of TA DESTROYED, why do people keep on believing? Cause they have no INFORMATION and have NOTHING else to hang onto a sliver of hope.....

TA ers your trading days are numbered...


bottoms up, socrates.....




HLJ

That's great news since the algos will be even easier to anticipate and predict than erratic human emotions ever were.

 

[highlifejoker]

Quote from Jerry030:

That's great news since the algos will be even easier to anticipate and predict than erratic human emotions ever were.

ok, whatever you think.

how about the billion dollar fund manager who had a fight with his wife the night before and decides to liquidate just to burn her ass?

what about seth tobias types? think you can anticipate their moves?

emotion still plays big part, but its now individuals emotion/ideas who control the large capital pools, not the masses of participants----if it ever was the masses. the very premise of TA is destroyed yet the TAers keep coming back for more punishment.

can't predict it, dont try.


HLJ


ps. by the way seth tobias movie/documentary on tonight CNBC.

enjoy!

 

[fluttrader]

Jerry030,



What you define as trade accuracy ? only PF ?