Effeciency of an Auto Strategy

Quote from AnonymousTrader:

Here's an interesting story:

I was approached by someone who wanted to run his strategy auto. We've done some tests (forward with tiny test size) and saw an interesting thing: His strategy had a 85% mean average win rate.

We got into an argument...
He thinks he found the holy grail.
I told him his strategy is not perfectly efficient.

It would be much better if the win rate (winning percentage) was in the low 50s.


What do you guys think?
More...



AT and I actually discussed this topic before so let me put in a few words.

First of all, get some rest - you became completely incoherent.




-----


When rational traders design a mechanical system (either automated or manual) the system functions on the parameters of constraints.

What I mean is: If there was no constraints, every second the system would buy or sell.

Most traders like to avoid losses and thus put on extra constraints to protect the system from losses.

Each extra constriant (if done for the above reason) will attempt to block false positives. The risk of doing so, is that you create false negatives. Therefore, the system will miss positive trades because it is afraid to take losses. Winning % is a good indication (usually) of too many extra constraints. It is no way the only indication, but in more cases than not, reducing winning % (by reducing constraints and thus increasing frequency as a result).


There are many other variables to consider as well. One is $win/$loss ratio. For each dollar lost, how much is gained?

The goal when reducing winning % is to keep the $win/$loss ratio constant.

There are many potential side-benefits to decrease winning percentage by this method.

The method will increase trade frequency. Sometimes this will mean that 1 day holding period will be reduced to some intraday period. That will remove overnight risk from the equation.

Here is an example.

Suppose a strategy with a 85% winning percentage, with a 1:1 $win/$loss ratio with average win of $1.00 and average trades of 100 per day.

Net PNL would be 100 x 0.85 x $1.00 - 100 x (1-0.85) x $1.00 = $70.00 profit.

Suppose we removed extra constraints to give us the following:

55% win percentage,
1:1 $W/$L,
$1.00 average win per trade
1000 trades

The Net PNL would be 1000 x 0.55 x $1.00 - 1000 x (1-0.55) x $1.00 = $100.00

The overall performance increased by $30.00 (a 42% improvement)

In this example I kept the $1.00 average win rate per trade the same. In many cases the $1 will also rise. If it rises to $1.50, the overall result will improve by another 50%:

1000 x 0.55 x $1.50 - 1000 x (1-0.55) x $1.50 = $150

Overall profit increase of 114%


Note: in this example I am using final *net numbers (after fees/commission calculations)



Conclusion:

The high winning percentage is usually an indicator that the system has extra constriants placed that produces too many false negatives. The goal is not to reduce the actual winning percentage by to reduce the false positives to a point where it still makes sense.




If done right, removing these types of constraints (another way is to lax them) will not harm the overall performance but can only improve it (if done right)

[ false positives gained x loss factor ] < [ false negatives removed x win factor]

in case of $W/$L of 1.00 the number of new false positives has to be less than the number of false negatives removed.



Another example:

if the *modification to the strategy increased the number of trades from 100 to only 150.
Remember, the first 100 trades should still be at the same numbers (85% win percentage). The extra 50 trades, will be, for argument sake, at 55% win percentage.
The overall profitability will look like this:




100 x 0.85 x $1.00 - 100 x (1-0.85) x $1.00

+

50 x 0.55 x $1.00 - 50 x (1-0.55) x $1.00

= $75

still a 7% improvement.



Depending on the efficiency of the strategy, some, even with high win% cannot be improved greatly.


I hope this clears many things up. I welcome all comments.


PS:

Be careful when comparing the two examples. In the first one, I assumed the overall strategy win % moved from 85% to 55%

In the last example I assumed that the incremental trades' win% was 55%. The actual overall win% of the new modified strategy would be 75%

PPS:

Note, I also realize that a 55% win percentage on 50 trades means that 27.5 trades will be profitable. I understand that there cannot be 1/2 a trade but I was using arbitrary numbers, please forgive me for that. For consistency, multiply all my numbers by 10, if you wish.
 
I agree with Serge.
And that's why I look more at profit factor (which is win$/loss$ as he mentioned).

Indeed, the goal is to keep profit factor as high as possible.
I watch drawdowns as well, because as win% decreases, the more likely you get a noticeable bad streak. these bad streaks increases drawdowns.

drawdowns are not a bad thing. they are a cost of doing business. However, it has impact on your Sharpe ratio. the higher the ratio, the safer, you can increase your capital at work with the ATS.

it's all a balance, depending on your risk profile.
you want to reduce win% while keeping profit factor higher (which will increase net performance). that's counter intuitive but very important. it's great that Serge pointed this out.
you also want to keep drawdown under control, some watch the equity curve geometry, Sharpe or average and max drawdown. it's all good.
 
There are a couple of points I would like to make in regards to your analyis:

1) if your winning percentage is lower, your odds of going into an extended drawdown increase. I need to check what the equation is (or derive it) but my guess is that it is an exponential function of winning percentage.

2) You give the win/loss as symmetric, $1. But if you think about it that means that the "stop loss" has to be less than the profit target. This leads to 1), i.e., lower winning percentage, but keeping the ratio of win/losses the same implies catching many more moves with extreme entry and exit efficiency.

All this is telling you is what should be mathematically intuitive, that in a closed system, if you adjust one side of the equation, something on the other side of the equal sign has to compensate, i.e., trade frequency goes up along with PnL, risk or ruin goes up accordingly.

nitro
Quote from Serge Pustelnik:

AT and I actually discussed this topic before so let me put in a few words.

First of all, get some rest - you became completely incoherent.




-----


When rational traders design a mechanical system (either automated or manual) the system functions on the parameters of constraints.

What I mean is: If there was no constraints, every second the system would buy or sell.

Most traders like to avoid losses and thus put on extra constraints to protect the system from losses.

Each extra constriant (if done for the above reason) will attempt to block false positives. The risk of doing so, is that you create false negatives. Therefore, the system will miss positive trades because it is afraid to take losses. Winning % is a good indication (usually) of too many extra constraints. It is no way the only indication, but in more cases than not, reducing winning % (by reducing constraints and thus increasing frequency as a result).


There are many other variables to consider as well. One is $win/$loss ratio. For each dollar lost, how much is gained?

The goal when reducing winning % is to keep the $win/$loss ratio constant.

There are many potential side-benefits to decrease winning percentage by this method.

The method will increase trade frequency. Sometimes this will mean that 1 day holding period will be reduced to some intraday period. That will remove overnight risk from the equation.

Here is an example.

Suppose a strategy with a 85% winning percentage, with a 1:1 $win/$loss ratio with average win of $1.00 and average trades of 100 per day.

Net PNL would be 100 x 0.85 x $1.00 - 100 x (1-0.85) x $1.00 = $70.00 profit.

Suppose we removed extra constraints to give us the following:

55% win percentage,
1:1 $W/$L,
$1.00 average win per trade
1000 trades

The Net PNL would be 1000 x 0.55 x $1.00 - 1000 x (1-0.55) x $1.00 = $100.00

The overall performance increased by $30.00 (a 42% improvement)

In this example I kept the $1.00 average win rate per trade the same. In many cases the $1 will also rise. If it rises to $1.50, the overall result will improve by another 50%:

1000 x 0.55 x $1.50 - 1000 x (1-0.55) x $1.50 = $150

Overall profit increase of 114%


Note: in this example I am using final *net numbers (after fees/commission calculations)



Conclusion:

The high winning percentage is usually an indicator that the system has extra constriants placed that produces too many false negatives. The goal is not to reduce the actual winning percentage by to reduce the false positives to a point where it still makes sense.




If done right, removing these types of constraints (another way is to lax them) will not harm the overall performance but can only improve it (if done right)

[ false positives gained x loss factor ] < [ false negatives removed x win factor]

in case of $W/$L of 1.00 the number of new false positives has to be less than the number of false negatives removed.



Another example:

if the *modification to the strategy increased the number of trades from 100 to only 150.
Remember, the first 100 trades should still be at the same numbers (85% win percentage). The extra 50 trades, will be, for argument sake, at 55% win percentage.
The overall profitability will look like this:




100 x 0.85 x $1.00 - 100 x (1-0.85) x $1.00

+

50 x 0.55 x $1.00 - 50 x (1-0.55) x $1.00

= $75

still a 7% improvement.



Depending on the efficiency of the strategy, some, even with high win% cannot be improved greatly.


I hope this clears many things up. I welcome all comments.


PS:

Be careful when comparing the two examples. In the first one, I assumed the overall strategy win % moved from 85% to 55%

In the last example I assumed that the incremental trades' win% was 55%. The actual overall win% of the new modified strategy would be 75%

PPS:

Note, I also realize that a 55% win percentage on 50 trades means that 27.5 trades will be profitable. I understand that there cannot be 1/2 a trade but I was using arbitrary numbers, please forgive me for that. For consistency, multiply all my numbers by 10, if you wish.
More...
 
Quote from MackieMesser:

...and I always want to look at the equity curve. Some systems look so good in the stats, but the equity curve is the gold standard.

m
More...


i look at pf and sharpe and it is really true, without the equity curve you never know the real truth.

i always believed that sharpe ratio is the one and only, now i reconsider.

for those using sharpe, what is your soft hurdle? mine is 2.0 for intraday system single market and 3.0 for a portfolio of intraday futures. all at least three years back, for equity futures i require that they do well before and after 2000.

peace
 
Quote from tntneo:

trade frequency is important as well, DT-waw mentioned it.
I track profit factor in my context (high frequency trading) because it gives a comprehensive view of the efficiency and smoothness of systems.

if you have 5 trades with a profit factor of 3, it is meaningless. Number of trades is of course important to get statistical significance.

profit factor will give you an idea of the slope of your equity curve. the higher the number, the smoother the curve as well. which can be confirmed with max and average drawdowns.

I won't trade anything below PF=2, and I try to have PF higher than that of course.
More...
Thanks tnt, I've always sought a clear thought about PF and yours as contained in here is the clearest I've seen yet.
 
55% win rate with 1000 trades means 45% losers (@ $1 each)

chance of a 5$ drawdown (by 5 consecutive losing trades) :-

( 0.45 ^ 5 ) x (1000-5) = 18.36 times per 1000 trades.




85% win rate with 100 trades means 15% losers (@ $1 each)

chance of a 5$ drawdown (by 5 consecutive losing trades) :-

( 0.15 ^ 5 ) x (100-5) = 0.0072 times per 100 trades.



Give me the second example with 10x leverage any day.



p.s. profit factor is not as good for measuring an entire portfolio as sharpe ratio. reason is while both pf and sharpe measure the risk vs reward (by different means), only sharpe is reliably affected by serial correlation. high serial correlation of equity curve means the bad trades tend to come together resulting in higher drawdowns vs peaks.

imagine if you took one set of daily returnes and arranged them into two series. first series arranged randomly. second series arranged with high serial correlation (good trades together, bad trades together). sharpe ratio on second series would be way lower than first because drawdowns are bigger and peaks smaller in series two. but pf would be exactly the same, no warning is given to serial correlation.
 
sorry the calculation above is slightly out it should read



( 0.45 ^ 5 ) x (1000-5+1) = 18.38 times per 1000 trades.


and


( 0.15 ^ 5 ) x (100-5+1) = 0.0073 times per 100 trades.
 
i have a strategy on paper that has a PF of only 1.4, a modified sharpe of 1.2, but it is on daily data and trades every third day the sp future. thus, since 1985 i have about 1500 trades. so far it does well on paper and i might move it into the market soon.
 
You must log in or register to reply here.
Back
Top