Optimization

F

funky

When given an optimization of data over 1 year and that same data only over the last 3 months, which is better?

on one hand, the 1 year data is catching more things that have happened, but on the other hand, the 3 month data might be more 'tuned' to the current market conditions.

any tips?
 
I used to think that the last 3 mo's was most valid, but in the Tripack thread it was mentioned that backtesting should capture as many instances of market movement as possible, which I now believe to be true. However, if you backtest any system long enough won't it find dead periods when it was not working? can we learn from that too?

Michael B.
 
...in the past two years I have "developed" too many systems based on 3 month data which were evanescent at best and random results at worst. I'm currently concentrating on finding more stable systems using all of the data E-Signal offers (6 months) and not using those systems until they have demonstrated 3-6 months of stability. That's what the wiser system development heads here told me many months ago, but of course I was smarter than they were.
 
yes, i am finding this as well. maybe its best to just optimize every week or so over the last year's data (so each time you optimize you are moving the year data forward by a week), OR by adding a week each time. from the former, it would be like a moving average of optimization parameters, but the latter would be a different beast. any ideas?
 
since your entries and exits are not going to happen in the "current" market conditions - but rather in future ie unknown market conditions - optimizing for the immediate past is asking for a world of hurt unless you really really understand what you are doing.
 
...I think the question of how often to optimize depends on how dependent the system is on volatility changes. For example, I used to trade two breakout variants (deviants?) which were tolerably profitable, but the key parameters needed to be fine tuned every week or two, and I got tired of it.

IMO what is ideal are systems which exploit intrinsic immutable God-given screw the public market mechanics. In my experience the stops and takes for these don't vary much. I haven't reoptimized the two systems I'm trading now for one and two months respectively. An advantage of that it that it allows much more time for development of new systems, as sustainment of high maintenance systems takes a lot of time.

May ye always find broad optima.
 
Quote from damir00:

since your entries and exits are not going to happen in the "current" market conditions - but rather in future ie unknown market conditions - optimizing for the immediate past is asking for a world of hurt unless you really really understand what you are doing.
More...

Another way of saying this could be - if your results are too sensitive to the tuning of the parameters, perhaps the model doesn't really work and all results are really random.

I would prefer to see profit results that curve away gently when the parameters are varied slightly. That's a good clue to a real edge I think.
 
More data to test on is generally better than less, so the longer time frame would offer a more significant result. However, this is not to say that the shorter, more recent time period should be disregarded. The best use is of both time periods: use the previous year (no overlap with the 3 months) to optimize and then test the resultant optimized system on the 3 months to make sure it works. They call this "walk-forward optimization." Do this with as many periods (of non-overlapping data, if possible) and in as many different markets as possible (even if you don't intend to trade on those markets or instruments) to make sure your system is statistically robust. Then, in every case, do a walk-forward test on data not previously used during the optimization.
 
One thought:
Optimize over a very long time span, but control for volatility of the underlying. Using this, find system profitability per unit of volatility for different system variables.

Then imput volatility and maximize profits after commissions by changing between systems with different levels of profitability per trade.

Of course, this only works if profitability is highly correlated to volatility or implied volatility.

Generally, use excessive commissions to adjust for the fact that optimized results WILL ALWAYS curve fit. Alternatively, run a cumulative walk-forward optimization routine, check the difference between expected and realized levels, then make an adjustment to the current projections.
 
You must log in or register to reply here.
Back
Top