Optimization Paradox: Avg Trade vs. Number of Trades

[farjeon]

What I would do is look at the equity curve of each option and see how it performed in different market conditions. For example if this is a stockmarket system I would like to see how it performs in the bull market of the late 90s, the following bear maket, the low volatility of 03 & 04 and finally the high volatility conditions of today. If option 1 can hold its own in all these enviroments then trade that. However in my experience it looks as though option 2 is the one that might be more robust.

In other words worry about which is the most consistent version of the system rather than which offers the best backtested profit.

 

[cunparis]

Quote from farjeon:

What I would do is look at the equity curve of each option and see how it performed in different market conditions. For example if this is a stockmarket system I would like to see how it performs in the bull market of the late 90s, the following bear maket, the low volatility of 03 & 04 and finally the high volatility conditions of today. If option 1 can hold its own in all these enviroments then trade that. However in my experience it looks as though option 2 is the one that might be more robust.

In other words worry about which is the most consistent version of the system rather than which offers the best backtested profit.

I have thought about this, but then I thought I could optimize it for the bull market and bear market and then trade the strategy optimized for the current conditions. Wouldn't this be more efficient than trying to find a strategy that works in both bull & bear markets?

 

[intradaybill]

Quote from cunparis:

Ok, I made a mistake. The R/R ratio shouldn't change between Option 2 & 3:

Option 3:
Win Rate: 70%
Reward/Risk: 1.5
Profit: $24k
Trades: 100
Avg Trade: $240

R/R ratio is simply the sum of the wins / sum of the losses.

I know it's inevitable that some may feel the need to show they are better than everyone else, it's quite common here on ET. But I'm hoping we can keep this thread on topic without insults and make it interesting for everyone.

What you have done in going from 1 to 3 was to add an additional condition to reduce the number of losing trades and increase the win rate. At the same time, you have also reduced the winning trades.

The ultimate difference between the two systems is their statistical significance. Probably, option 3 is less statistically significant result that option 1 because your trade sample has been reduced by a factor of 3. But that is not necessary. You have to apply statistical analysis techniques to find out whether some of the results in option 3 could have been obtained by chance as compared to results of option 1. This is not an easy study to do. But what you are asking cannot be answered simply on the basis of backtesting results because the ultimate measure is how the two different systems perform in actual trading.

Which brings us to the point: trade both systems and keep a log of the performance. After 6 months you know. Everything else is more or less speculation. Good luck.

 

[TrueStory]

The bigger the sample size, the less likely the system is curve-fit. The more variables, the more likely the system is curve-fit. I always prefer more trades if possible (even given costs of trading) if it is more convincing that the system is not curve-fit.

However, I would be equally concerned with other factors:

1) Out of sample testing
2) Testing other markets
3) Performance across wide parameter values

Beware of creating a system by optimizing 5 indicators until you find something that looks great. If you choose to develop using that method, please consider points 1,2,3 strongly.

 

[zdreg]

Quote from cunparis:

I have thought about this, but then I thought I could optimize it for the bull market and bear market and then trade the strategy optimized for the current conditions. Wouldn't this be more efficient than trying to find a strategy that works in both bull & bear markets?

according to barton biggs we are headed for the biggest bull rally within a bear market.
where does that that leave you?

rallies in bear markets may look like bull markets.
declines in bull markets may look like bear markets

 

[cunparis]

Quote from zdreg:

according to barton biggs we are headed for the biggest bull rally within a bear market.
where does that that leave you?

rallies in bear markets may look like bull markets.
declines in bull markets may look like bear markets

Well if we're in a bear correction in a long term trend then in the intermediate term we're in a bull rally. So I'd use the bull market parameters.

The trick is to identify when we transition from one to the other. When S&P breaks its last high (I think around 900) then we'll be in bull mode.

 

[farjeon]

Quote from cunparis:

I have thought about this, but then I thought I could optimize it for the bull market and bear market and then trade the strategy optimized for the current conditions. Wouldn't this be more efficient than trying to find a strategy that works in both bull & bear markets?

If you can tell when it's a bear and when it's a bull you've got your second system. Trade that along side Option 1 or Option 2 and you should get some nice diversification.