Quote from intradaybill:
Define:
Actual
Systematic
Objective
Testing
Significant
Predictive
Short term trading
I can bet, those 7 things mean different things to different people. It is possible that if you give an idea to 10 traders to test it you will get 10 different results.
Before you start a long post please keep in mind that extensive experience (mine and of others) has shown that it is not the concepts but their use that makes a difference.
It is too, way too, dogmatic to say that certain methods do not work.
I agree that naive testing does not provide an edge. But those 7 things you mentioned and I listed above offer a very broad spectrum of operation from very naive to extremely sophisticated.
They can mean different things. However, present them to most statisticians and they will tell you the following:
Actual - utilizing testable statistical methods rather than just visual confirmation.
Systematic - the testing method can be repeated over and over again in the same manner using a clearly defined set of steps. It is preferable that the rigorousness of the system be such that others utilizing the same steps should get the same results for the test for the same data.
Objective - (Actual and systematic combined). Furthermore, conclusions are based on test results rather than what outcome the tester was expecting.
Testing - The statistical methods were actually used rather than just assuming the data passes the tests through haphazard testing (in the case of trading, usually just visual confirmation)
Significant - A null and alternate hypothesis were clearly defined before the testing was done for what the result should be. The level of significance would the alpha value chosen (the level of significance) at which the test would be run. I prefer to run tests at an alpha level no less than 0.10. In other words, it is reasonable to expect the outcome of the test on the sample data to be the outcome at least 90% of the time for the overall population. However, many people might feel comfortable with lower alpha levels which is reasonable. Either way, most mainstream indicators would not pass profitability testing at alpha levels < 0.3 which is very unreliable.
Predictive - The testing strongly suggests that the edge is something that should exist in the future. This is perhaps the hardest thing to test for. The most common way is to create a system that yields a certain type of results for one set of data, then yields similar results for a forward looking set of data. Predictive value must be subjected to its own tests and can have its own alpha value as well.
Short term trading - This is the only term that I think can really be very, very vague and can reasonably have "wildly" different definitions. For the sake of this conversation, lets refer to it as any position opened and closed within the same market session (no swing trades). I've never trader forex so I cannot confidently provide a definition for the forex market. If I had to pick, I'd say any position opened and closed within a 12 hour window, but like I said, I've never been a forex guy.
You are certainly right that usage can alter the results. However, I would be right to say that greater than 90% of the time main stream TA fails to give a profitable edge to traders irrespective of usage. The people that are profitable and still use main stream TA (such people are so rare that they can literally be considered statistical anomalies). It is far more likely that other methods are being used for an edge and the concept of an indicator is something completely different for these traders.
The majority of short term traders (as defined above) that say they are profitable only think they are profitable, but over the long run will end up losing money. Hence, the wide belief that mainstream TA is profitable is a bunch of traders succeeding for a short while and propagating the notion that main stream TA is working, then failing with only relatively few people coming back to propagate the reality that their methods never really worked in the first place.
There are certain points I do want to address that I think are needed to properly convey the message I want to get across, but I don't have the time to run the analyses to present that information. When school and work settle down, I will come back to the site for a more in-depth friendly debate.
