Why do I see "Trends" in Randomly Generated Data?

A significant difference exists between randomly-generated price data and actual market price data.

Just one of an almost limitless number of examples is look at the range of bars that occur immediately after a new High or Low of hour/day/week/year (this is true on virtually all liquid instruments, all time frame settings, all variations). The average range of these particular bars is statistically different than that of randomly generated data (control group data).

And yes there are "edges" related to exploiting knowledge of market microstructure, some of which are large enough to overcome transaction costs and thus present profitable opportunities. Some of the most obvious ones have already been statistically arbitraged away for the most part, but many are undiscovered or have remaining capacity.
 
Quote from ptunic:

A significant difference exists between randomly-generated price data and actual market price data.

Just one of an almost limitless number of examples is look at the range of bars that occur immediately after a new High or Low of hour/day/week/year (this is true on virtually all liquid instruments, all time frame settings, all variations). The average range of these particular bars is statistically different than that of randomly generated data (control group data).

And yes there are "edges" related to exploiting knowledge of market microstructure, some of which are large enough to overcome transaction costs and thus present profitable opportunities. Some of the most obvious ones have already been statistically arbitraged away for the most part, but many are undiscovered or have remaining capacity.
More...

I agree with you for most part but I don't understand why "Some of the most obvious ones have already been statistically arbitraged away for the most part". Do you imply that markets have memory to that extend?

Can you be more specific about the mechanism of "arbitraging away"? This is something stated often but I am suspicious it maybe that some "authority" once claimed it and everyone else repeats it without understanding how it works or even if it is true.

Bill
 
Quote from Rahula:

Now doesn't a random market assumption mean that both mean reversion and trend trading strategies don't have any science to back it up and at the end of the day it all comes down to luck?

I kno......... thread. [/B]
More...
====================
IF? i understand your question -Raha;
look at all /years data[months] QQQQQ, Note trends, both up & down.

Look @ CFC, C, IMB, 2007, those are what we call downtrends;
not random.:cool:

Trading is not luck, but small samples are silly;
trading or running a business isnt easy, but nothing to do with luck.

Small size is wise[as a % of equity]

Good question

murray tt:cool:
 
Quote from intradaybill:

I agree with you for most part but I don't understand why "Some of the most obvious ones have already been statistically arbitraged away for the most part". Do you imply that markets have memory to that extend?

Can you be more specific about the mechanism of "arbitraging away"? This is something stated often but I am suspicious it maybe that some "authority" once claimed it and everyone else repeats it without understanding how it works or even if it is true.

Bill
More...

Let's say you find a valid pattern-- could be fundamental, technical, or other. This one is illegal but is great for illustrative purposes in this example (obviously I don't recommend it): The pattern is that for a particular small pink sheet company, the CEO posts on a private blog his personal thoughts, just for him and his/her management team internally. He only posts it a day before earnings announcement, and gives the earnings numbers in advance.

Except he forgot to get the password working so it is open to everyone. So far you are the only person who found it. Now let's assume you are a hedge fund manager who manages a $1B portfolio. Further let's assume you are in a country without insider trading laws or otherwise don't care about that aspect.

You have an edge. Can you put your entire $1B portfolio towards this? Assume the small company has a market cap of $30 million. There is no way.. at least without buying the company outright. It is very possible, due to not finding enough buyers or sellers, you simply won't be able to trade more than $100k the day before earnings.

Anyhow, let's say you max out the $100k or so. The first interesting effect on this is to a large degree, you are incorporating the information of the earnings announcement earlier than previously; to some degree you are making the market more efficient. Let's say the stock is at $2.00. And you know the CEO is about to announce a much higher earnings per share number than is expected. Thus you buy the day before. Since you are maxing out as much as you can, you will actually fully incorporate this information into the price of the stock. Your actions will move the stock price so that when the earnings number is released, the stock doesn't necessarily move up at all; it's already moved up 10%, 30%, or whatever it did due to your buying activity. Now let's say another hedge fund manager finds out the same blog as you. At this point, they might not enter in the trade as they see the stock price already has moved up within 2 hours of the blog post (about 24 hours ahead of the earnings announcement). Then again, this might encourage them to check the blog more frequently than you, and let's say they find the new blog entry 1 hour before you. They can now get the benefit of this "edge", crowding you out. You might still get some benefit but not nearly as much as before.

This is what I refer to by capacity and arbitrage. The same concept holds true for any traded instrument and virtually any strategy, whether it is something technical like support and resistance, moving averages, something fundamental like earnings or growth rate trends, etc. Of course, something like S&P 500 futures has tens of thousands times as much capacity than a penny stock does due to higher liquidity. It also depends on the timeliness of your informational edge. The smaller the time frame, the lower the capacity because all other things being equal, you can find more counter-parties to trade with the longer you look.
 
Quote from fluttrader:

Can you elaborate why the way of testing was fundamentally flawed ?
More...

Researchers rarely break down their data to its least common denominator before they start analyzing it. They simply don't realize the importance.

This was the case with Gabaix and his team. Most researchers don't understand that market data is different than ordinary math problems so they treat the environment the same. Gabaix's group fully understood that they were analyzing almost 100 million transactions and they understood that each transaction was made up of a specific amount of volume but they didn't specifically analyze the volume just the transactions.

They didn't reduce the problem to the sum of its parts. Problem solving IS an exact science it’s just that one can not always solve a problem for various reasons. The only way to start the process of problem solving though is to first reduce the problem to its least common denominators. Just like building a house, you can't erect the walls without first putting in the foundation.

(well you could but I wouldn't want to live in it)
 
Quote from ProfLogic:

Researchers rarely break down their data to its least common denominator before they start analyzing it. They simply don't realize the importance.

This was the case with Gabaix and his team. Most researchers don't understand that market data is different than ordinary math problems so they treat the environment the same. Gabaix's group fully understood that they were analyzing almost 100 million transactions and they understood that each transaction was made up of a specific amount of volume but they didn't specifically analyze the volume just the transactions.

They didn't reduce the problem to the sum of its parts. Problem solving IS an exact science itÂ’s just that one can not always solve a problem for various reasons. The only way to start the process of problem solving though is to first reduce the problem to its least common denominators. Just like building a house, you can't erect the walls without first putting in the foundation.

(well you could but I wouldn't want to live in it)
More...

Agreed, the seed (first step) is the most important otherwise results are skewed.

Information may be harvested from skewed data, however, if consistently was maintained by the team.
 
Quote from ProfLogic:


Yes there is but you have to be able to define "Trend" first. No definition, no predictability advantage.
More...

... that's the truth of the matter.
 
Quote from intradaybill:

Irrelevant division you made. People don't trade price/volatility but price and as MIT researchers have shown there is a hidden order in the markets that obeys a power law:

http://web.mit.edu/newsoffice/2003/powerlaw.html


Bill
More...

Nice article. Here is an interesting passage:

In their paper, the scientists show that--for the market as a whole and for an individual stock--the daily volume of stocks traded, number of trades and price fluctuations follow power laws.

For example, the number of days when a particular stock price moves by 1 percent will be eight times the number of days when that stock moves by 2 percent, which will in turn be eight times the number of days when that stock moves by 4 percent, which will in turn be eight times the number of days that stock moves by 8 percent, and so on.

The same relationship (called the inverse cubic pattern) characterizes the number of daily trades. A similar power law (the inverse half-cubic pattern) describes the number of shares traded each day.

For instance, if 100,000 shares of Apple stock were traded on 512 days during a certain period, then you can predict that there would be 64 days when 400,000 shares of Apple stock were traded, and eight days when 1,600,000 shares of Apple stock were traded, and one day when 6,400,000 shares of Apple stock were traded.

To understand these patterns, the scientists looked at the size of large traders, such as mutual funds with more than $100 million in assets. They found that their size also follows a power law. The number of funds that manage $1 billion is twice the number of funds with $2 billion, which in turn is twice the number of funds with $4 billion, and so on. (This pattern is called Zipf's Law, named after linguist George Kingsley Zipf, who in the 1930s found the statistical pattern in the frequency of word use in languages.)

The scientists prove that the patterns in daily trades, returns and volume are generated by the actions of large market participants when they trade stock under time pressure. Stocks traded under time pressure by mutual funds cause stock prices to change; these actions are enough to generate these patterns.

Hence, a lot of the large movements of the market, such as the days of small "crashes" when prices seem to move for no good reason, can ultimately be traced back to the behavior of some very large players.
More...
 
Quote from intradaybill:

MIT researchers have shown there is a hidden order in the markets that obeys a power law:
http://web.mit.edu/newsoffice/2003/powerlaw.html
More...
Bill, are you trying to usurp Mark Brown as the ET Village Idiot?

Markets prices can be modeled as samples from a lognormal
distro, not the distro itself. Sampled lognormals often look
pareto in the tails:

http://demonstrations.wolfram.com/PowerLawTailsInLogNormalData/

And a geometric mixture of lognormals is pareto in the
tails, esp the lower tail. Ref Montroll & Schlesinger, 1983
ME Formalism and 1/f Noise, or Allen, Li, Charnof, 2001,
Power Laws and Mixtures of Lognormal Distributions.

Those MIT "scientists" are not all that clever and as for the
SF Institure, are you kidding me, nobody takes anything out
of that less than august body seriously.


-Wildo
 
You must log in or register to reply here.
Back
Top