Quote from David Aronson:
... If it is testable, then how does one determine if the back tested result is due to chance, in the case of back testing a single rule or system, or due to data mining bias in the case of testing many systems and picking the one that performs best.
It seem to me that this is the fundamental question that other books on TA, of which I am aware, and I own about 300 of them, never address.
I am not being argumentative here. But I am being curious. Can you cite some examples of books with useful tidbits or TA knowledge. If they are out there I would like to know of them. Believe me, many of those 300 books just take up space. I do feel badly that you feel your money was wasted. I am sure that you can offer it for sale on amazon used and get at least some of your money back. I own one TA book written by Tony Crabel, turns out people are willing to pay over 1000 for it.
David Aronson
David:
Although I have not read your book yet (arriving this week), Crabel's book is popular now because he was a trader who was able to parlay his research into real money, i.e., he runs a billion dollar hedge fund. The appeal of Crabel's book is that he enumerated hundreds of price patterns, combined them with daily open/close and range sequences, and calculated the P&L for all of these permutations. So, an inexperienced trader would be tempted to pick out the best-performing patterns (your "data mining bias") and probably lose his money. The brilliance of Crabel's book is his analysis of range contraction and expansion for volatility breakout systems. When I develop a system, I prefer to first validate a principle (e.g., mean reversion, volatility expansion, etc.) rather than program a rule. My market framework is much more important to me than a given system, as long as I have a choice of systems that work within that framework (e.g., high vs. low volatility).
Still, your fundamental question has not been addressed in any of the trading literature. But suppose I find what I
think is an edge. Do I really care whether or not it is a real edge if I have a system in place to monitor deterioration in the performance of the system that monitors the so-called edge? Maybe the "edge" stops working right away, maybe it lasts another year. Take the "open range breakout" for example. I recall a January several years ago where there was a trend day almost every day that month. I prefer to follow acrary's example of framing a day's price action (e.g., a trend day) and then mining the conditions that consistently identify trend days, using various time frames. I would tend to trust an edge that operates on a simple principle with less accuracy than an edge that has eight degrees of freedom (variables). My most consistent system is an upward-biased stock trading system that uses a moving average and a calculation based on standard deviation off a weekly low. Two fundamental principles: upward bias of the stock market and mean reversion.
So, I hope your book addresses not only the fundamental question of a true edge, but the degree of reliability that can be expected based on the variables defined by the system, the breadth of the markets tested, and the testing methodology. The greater the variability, the greater the probability that the edge can be chalked up to chance.
Comments?