Quote from man:
so the problem is to first find a way how to distinguish "true" systems
from "fluke". there are several ways we try to tackle this, one of
them being the choice of what we look for. we want to understand
the effect in the first place, using our human intelligence as a prefilter
before we start a single code line.
what i am saying is that the path you choose before you start testing
is utterly important. alan chose statistical analysis of the whole
time series, like the sp future, to derive trading ideas he would
then follow.
.
Quantitative Investment Management which has a great track record is doing exactly the opposite of what you're saying. See my thread
http://www.elitetrader.com/vb/showthread.php?s=&threadid=115513&
Also there's a superb article by Nassim Taleb which nails the issue:
http://www.elitetrader.com/vb/showthread.php?s=&threadid=124793&highlight=taleb
Quote from the article:
Things, it turns out, are all too often discovered by accident--but we don't see that when we look at history in our rear-view mirrors. The technologies that run the world today (like the Internet, the computer and the laser) are not used in the way intended by those who invented them. Even academics are starting to realize that a considerable component of medical discovery comes from the fringes, where people find what they are not exactly looking for. It is not just that hypertension drugs led to Viagra or that angiogenesis drugs led to the treatment of macular degeneration, but that even discoveries we claim come from research are themselves highly accidental. They are the result of undirected tinkering narrated after the fact, when it is dressed up as controlled research. The high rate of failure in scientific research should be sufficient to convince us of the lack of effectiveness in its design.
If the success rate of directed research is very low, though, it is true that the more we search, the more likely we are to find things "by accident," outside the original plan