i think the basic thing starts with the utility function you choose. whenever it is not a return/risk figure you are already flawed.
random trades produce sharpe ratios above 1.0 in 3% of all cases. thus, if you find a system with a sharpe above 1.0 your chances are about 30:1 that it is not a fit. GIVEN that your underlying systemSearchMethodology is comparable to the random entry test.
assume that you do 10000 random entry tests, resulting about 300 equity curves with a sharpe above 1.0. now assume you take these 300 and you add a single parameter and run these 300 tests again. will you expect NOW to have 9 tests with a sharpe above 1.0, which corresponds to 3% chance? clearly not, since you already start with nonRandom systems but only those that already show a sharpe above 1.0, so you would expect to jump up to about 50% of all systems yield a sharpe above 1.0. BUT, that is still pure chance, no edge at all around here. nothing tradeable.
if you think this example is abstract, then ask yourself how often you started working again on something that looked good once? if this test then was already a result of a fit, your derived variation will be too ...
does it help if you look at the time series itself before you make a system backtest? thus, you are trying to find conditions that predict the market will go up tomorrow and only after you have found them you start a backtest?
i must admit that i still do not see a difference in principal. why shoudl you not be able to fit this kind of statistic analysis as well by introducing many differnet paramters?
yet, i am still puzzled with the fact that the kind of systems derived by such test seem to do better. a possibility i am thinking of is that it is more psychological than one might think in the first place. let me elaborate.
system backtesting is a boring thing. most stuff simply does not work, especially those ideas, that sound super convincing in the first place. so you start playing around, adding something here, tweaking something there and you add up frustration since hardly anything seems to make sense. timeconsuming, frustrating, fitting around. and nothing seems good enough for trading it.
when you look for edge, things do not need to make trading sense in the first place. just a slight shift in hit ratios is already some inspiring "learning effect". you feel like you start to get what this is all about and you add up effect after effect, still most do not work the way you would wish, but you see slight effects all over the place and you enjoy the search process, thus you do more of it. and after all that edge searching you glue that all together, add some stop here and some pyramding there and ready you are.
now was that process "per se" a better weapon against fitting? i'd say not in principle, but very much so in terms of psychologically enabling you to dig deeper and deeper.
sorry ... i think i just wrote that for myself ...
