Quote from ssrrkk:
If that's the case, then the issue is not so much curve fitting (i.e., fitting to a pattern that is not really there), but rather is one of dealing with a finite window of opportunity. In that sense, trading is like any other business, except that the window of opportunity is a lot shorter than in others. In other words, the problem of strategy design becomes not of curve-fitting (chasing mirages) but rather of non-stationarity -- the patterns are there, are statistically significant, are mechanism based, but they disappear as soon as you start exploiting them.
In this scenario which is also well-known and discussed on this board (i.e., non-stationarity), then all one needs to do is find an edge, keep it a secret, but estimate as best as you can the expiration date or window of opportunity. Have several new strategies in the pipeline, and as soon as one expires (even before your PL curve starts going down) you swap with the new strategy and continue the cycle.
So instead of coming up with an adaptive strategy that continuously fits just the parameters, you come up with a pipeline of strategies that are proven to work in the last 1 year or so, run it until your estimated expiration date, then swap to the next in the pipeline and keep going.
Another approach is to come up with a strategy that is highly non-obvious but works. The expiration date on those will likely be longer so you may be able to make more money on those before moving on to the next one.