You should take his/her comments with regard to using Excel more broadly. What it implies is more important than his/her specific use. The point is: To really do some beneficial backtesting you are going to have to get your hands dirty with some data analysis and manipulation - as opposed to having a pre-packaged program accomplish all aspects of system development.
"Don't optimize, because you will curve fit," is such a hackneyed and trite statement it makes me vomit. Optimae emptor....optimizer beware. Constrained optimization is very useful. Incorporating another issue raised in this thread: the use of multidimensional data representation. The use of 3D graphs can be very useful in evaluating 2 parameter systems (assume profit is on the z-axis/height). One can simply look at what looks like a topographical map for REGIONS of higher profitability, as opposed to single point that happens to be a global max that is surrounded by unprofitable regions. By assessing a bit more of the "larger picture," one can avoid the pitfall of settling on what happens to be a historical global profit maximization point surrounded by unprofitable points in the 2-d region of parameter possibilities. Look for a range of profitable parameter settings....
Let's say Trader A has a system that is profitable (backtested only). He/she has come up with specific parameters by a few random choices. Trader B has come up with a system that optimized his parameter settings such that he maximizes whatever objective function (some risk/reward representation, likely) out of the entire space.......Now, ceteris paribus.....Who is to say that Trader A has happened upon a more profitable surrounding region simply because he didn't use optimization..........in fact take a large enough population of Trader A's and one Trader A has likely settled on the SAME parameter settings as Trader B, only randomly.............ignorance is bliss I suppose....
"Don't optimize, because you will curve fit," is such a hackneyed and trite statement it makes me vomit. Optimae emptor....optimizer beware. Constrained optimization is very useful. Incorporating another issue raised in this thread: the use of multidimensional data representation. The use of 3D graphs can be very useful in evaluating 2 parameter systems (assume profit is on the z-axis/height). One can simply look at what looks like a topographical map for REGIONS of higher profitability, as opposed to single point that happens to be a global max that is surrounded by unprofitable regions. By assessing a bit more of the "larger picture," one can avoid the pitfall of settling on what happens to be a historical global profit maximization point surrounded by unprofitable points in the 2-d region of parameter possibilities. Look for a range of profitable parameter settings....
Let's say Trader A has a system that is profitable (backtested only). He/she has come up with specific parameters by a few random choices. Trader B has come up with a system that optimized his parameter settings such that he maximizes whatever objective function (some risk/reward representation, likely) out of the entire space.......Now, ceteris paribus.....Who is to say that Trader A has happened upon a more profitable surrounding region simply because he didn't use optimization..........in fact take a large enough population of Trader A's and one Trader A has likely settled on the SAME parameter settings as Trader B, only randomly.............ignorance is bliss I suppose....
