I am not wrong. Unless my calculation method is wrong, but I didn't say 3/4 of the time I said OVER 3/4 of the time.
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
When are you optimizing too much?
- Thread starter earlyexit
- Start date
I am not wrong. Unless my calculation method is wrong, but I didn't say 3/4 of the time I said OVER 3/4 of the time.
Quote from Algo_Design_Kid:
OK. I guess this is basically the same thing. But I'll bite.
No, it's not. As I said, you're misunderstanding.
Calculate the PF on your example with 3 $1 losses and 1 $3 win. Until you get the correct answer, which is 1, you still don't understand PF.
Systems with PF of 1 are break-even in the long run. Systems with PF > 1 are profitable. < 1 is unprofitable.
There are other things you would like to know about a system other than PF, but it IS relevant and does separate interesting systems from junk.
Quote from Algo_Design_Kid:
I am not wrong. Unless my calculation method is wrong, but I didn't say 3/4 of the time I said OVER 3/4 of the time.
Yeah, you are wrong. A million people will eventually come in here, tell you you're wrong, and explain it to you.
My bet, though, is that you still won't get it. And that's fine. It doesn't really hurt me that you're wrong.
Quote from The Big D:
Yeah, you are wrong. A million people will eventually come in here, tell you you're wrong, and explain it to you.
My bet, though, is that you still won't get it. And that's fine. It doesn't really hurt me that you're wrong.
Oh my bad baby D I was thinking payoff ratio. A thousand pardons.
Quote from The Big D:
Yeah, you are wrong. A million people will eventually come in here, tell you you're wrong, and explain it to you.
My bet, though, is that you still won't get it. And that's fine. It doesn't really hurt me that you're wrong.
Please stop quoting him, some of us have him on ignore for that reason. Thanks!
Quote from earlyexit:
At what point is optimizing a strategy become too much, and you are just curve fitting?
I have an automated strategy that I wrote that performs pretty well (profitable 76% of the time, 3.24 profit factor). I've been using it for a while, but the optimal values change as time goes on.
For instance, if I optimize over the past 3 months, I get one set of ideal settings, but if I optimize over the past month, I get slightly better results from different settings.
- It is expected that optimal values will change over time. However. If the change is a regime change you will actually be better off only reoptimizing when a regime change is evident. By example, if two regimes are 12 months followed by 2 months, it might appear that optimizing every 1-2 months is optimal, actually the better approach would be only reoptimizing at the 12 month mark or shortly thereafter.
- You are curve fitting when adding optimizations increases risk. This may take the form of increased uncertainty, increased volatility, increased max drawdown, decreased robustness, etc. One example, if you reduce trades in a year from 1200 to 12, profit might look great, however you've lost any statistical evidence that the system is low risk. Another example, if you add in optimizations that are (unknown to you) random, profit might increase, however by adding completely random factors the system risk has increased.
- (Also keep in mind the basic definition of curve fitting. Curve fitting is optimizing without any forward validation. If optimizing over 1 month works better than optimizing over 3 months, you are "curve fitting" or "cherry picking" unless you have another 12-36+ months of data to validate against. Then if with the new data you change your mind and say that optimizing over 2 months works best, you will need to have another 12-36+ months of data to test your new hypothesis against. Otherwise, you are curve fitting.)
Good stuff to think about. Thanks Stoxtrader.
Quote from Stoxtrader:
- It is expected that optimal values will change over time. However. If the change is a regime change you will actually be better off only reoptimizing when a regime change is evident. By example, if two regimes are 12 months followed by 2 months, it might appear that optimizing every 1-2 months is optimal, actually the better approach would be only reoptimizing at the 12 month mark or shortly thereafter.
- You are curve fitting when adding optimizations increases risk. This may take the form of increased uncertainty, increased volatility, increased max drawdown, decreased robustness, etc. One example, if you reduce trades in a year from 1200 to 12, profit might look great, however you've lost any statistical evidence that the system is low risk. Another example, if you add in optimizations that are (unknown to you) random, profit might increase, however by adding completely random factors the system risk has increased.
- (Also keep in mind the basic definition of curve fitting. Curve fitting is optimizing without any forward validation. If optimizing over 1 month works better than optimizing over 3 months, you are "curve fitting" or "cherry picking" unless you have another 12-36+ months of data to validate against. Then if with the new data you change your mind and say that optimizing over 2 months works best, you will need to have another 12-36+ months of data to test your new hypothesis against. Otherwise, you are curve fitting.)
Quote from Algo_Design_Kid:
Oh my bad baby D I was thinking payoff ratio. A thousand pardons.
No problem. It often happens that people confuse the avg. win to avg. loss ratio for the profit factor. I recommend you read this paper carefully for the details. It has all the definitions and it derives the relationship between those two variables and the win rate. One of the best paper I have ever read.
http://tinyurl.com/39tw342