Overoptimization

A

aparsai

I have come to an important conclusion. It looks too obvious to say it out loud but we sometimes need to remind ourselves of obvious and make sure what we have heard is true. Here is my conclusion: "Over-optimization could ruin a trading system."

I had developed a trading system a while ago. The back-test showed profit for a 2 year period. I extended the back-test period to 3 years and then 4 years and in all cases it generated profit. This could be a good sign of a profitable system. Then I tried to optimize the parameters. I managed to almost double the profit factor for the past 3 years but when I extended the test to 4 years the system failed in the 4th year.

The bottom line is that if your system shows profit and when you extend it to the past or the future (via forward-testing) and it still generates profit it could a good system. You do not really need to optimize it to make it better.

Your insight is welcome.
 
I forgot to mention that I'm not against optimization. What I'm trying to say is that do not spend too much time on it as it becomes curve-fitting instead of making a system a better one.
 
1) how many parameters did your system use?

2) did you walk-forward optimize? or just optimize on a static window and examine the results?

rt
 
You don't even need to use an optimizer to get there. If you have 'manually' developed a system with good test results, its still possible that it is a curve-fitted result by chance (that it is the same outcome than with using a lot of optimization cases). If you do a lot of research 'manually', you are bound to find such a case sooner or later.

So you can still become a victim of the data-mining bias even without an optimizer. That's also still true if you use an in-/out of sample testing procedure.
 
Quote from RoughTrader:

1) how many parameters did your system use?

2) did you walk-forward optimize? or just optimize on a static window and examine the results?

rt
More...

For this specific system the parameters were limited to less than 10.

I used MetaTrader optimizer to take care of the job for me.
 
I never optimize more than two parameters at any time and than make sure we do not take the best performing and pick a realistic equity curve and than use some portfolio management tool to build a portfolio of non correlated systems..... the part of your process is the key in my book and missed by most.
 
Quote from aparsai:

For this specific system the parameters were limited to less than 10.

I used MetaTrader optimizer to take care of the job for me.
More...

10 parameters???!!!! Of course it's not going to work! All you are doing is performing an exercise in surface optimization. With that number of parameters, there is also a chance you haven't even found the global optimum, but rather a local one.

Come back to the basics. A sound strategy is a simple one who's underlying logic make sense in the context of the market.

rt
 
You must log in or register to reply here.
Back
Top