System Performance Score

[kut2k2]

Quote from nonlinear5:

Now you are mixing the absolute P&L and the percentage P&L in the same formula. This will lead to confusing results.

I think if you experiment long enough, you'll come down to this:

SPS = sqrt(trades) * (P / S),
where
trades is the total number of trades
P is the average profit per trade
S is the standard deviation of all trades

This closely relates to Sharpe's ratio and the SQN (System Quality Number). The above formula is what I use to rank and compare my trading systems.

You've completely misinterpreted the formula. There is no absolute P&L. The NOBF is based on percent returns, not absolute returns.

No need for me to "experiment" further. The formula you posted looks pretty much the same as the SQN with all the same flaws.

 

[nonlinear5]

Quote from kut2k2:

The NOBF is based on percent returns, not absolute returns.

Ok, let's try your latest version of the formula, with the percent returns.

System A: [-8, -8, -8, +25, +25]
System B: [-1, +1, +1, +21, +50]

SPS(A) = 0.450763
SPS(B) = 0.446332

So, according to your formula, system A is better than system B, while everyone with common sense would agree that system B is way better.

My own formula rates system B at about 2.3 times better than system A. Additionally, Sharpe's, Sortino, SQN, Kelly, Profit Factor, Max DD, Net profit, and pretty much every other performance measure would also rate system B higher.

 

[kut2k2]

Quote from nonlinear5:

Ok, let's try your latest version of the formula, with the percent returns.

System A: [-8, -8, -8, +25, +25]
System B: [-1, +1, +1, +21, +50]

SPS(A) = 0.450763
SPS(B) = 0.446332

So, according to your formula, system A is better than system B, while everyone with common sense would agree that system B is way better.

My own formula rates system B at about 2.3 times better than system A. Additionally, Sharpe's, Sortino, SQN, Kelly, Profit Factor, Max DD, Net profit, and pretty much every other performance measure would also rate system B higher.

All you've proven is that the NOBF can't handle ridiculous extremes like your system B, which never occur IRL. Calculating the true Kelly fractions gives the correct order for the SPS relationships.

Kelly(A) = .026
NOBF(A) = .018 = about 70% of the true Kelly
NOBF(B) = .0245 = about 3% of the true Kelly
Kelly(B) = .734

SPS(A) = 0.65
SPS(B) = 13.4

That's quite a lot more than your 2.3 ratio.

 

[kut2k2]

Quote from kut2k2:

Quote from nonlinear5:

Ok, let's try your latest version of the formula, with the percent returns.

System A: [-8, -8, -8, +25, +25]
System B: [-1, +1, +1, +21, +50]

SPS(A) = 0.450763
SPS(B) = 0.446332

So, according to your formula, system A is better than system B, while everyone with common sense would agree that system B is way better.

My own formula rates system B at about 2.3 times better than system A. Additionally, Sharpe's, Sortino, SQN, Kelly, Profit Factor, Max DD, Net profit, and pretty much every other performance measure would also rate system B higher.

All you've proven is that the NOBF can't handle ridiculous extremes like your system B, which never occur IRL. Calculating the true Kelly fractions gives the correct order for the SPS relationships.

Kelly(A) = .026
NOBF(A) = .018 = about 70% of the true Kelly
NOBF(B) = .0245 = about 3% of the true Kelly
Kelly(B) = .734

SPS(A) = 0.65
SPS(B) = 13.4

That's quite a lot more than your 2.3 ratio.

In fact, system B is so out of phase with reality that higher-order polynomial approximations of the Kelly fraction get farther away from the exact value rather than the norm of getting closer.

Linear approximation = 0.0245
Cubic approximation = 0.0225
Quintic approximation = 0.0217

:eek:

 

[syswizard]

Bottomline: SPS is a "bust"...correct ?

It's just some sort of statistic, but does not correlate to true performance....right ?

 

[kut2k2]

Quote from syswizard:

Bottomline: SPS is a "bust"...correct ?

It's just some sort of statistic, but does not correlate to true performance....right ?

So what performance metric do you prefer?

 

[syswizard]

Sharpe or Sortino Ratio.

 

[kut2k2]

Quote from syswizard:

Sharpe or Sortino Ratio.

Pick one.

 

[syswizard]

I'd say Sortino then.

 

[kut2k2]

Quote from syswizard:

I'd say Sortino then.

Fail.

One key characteristic of a good performance measure should be its universality.

For example, horsepower, a widely accepted measure of engine performance, is universal: its measurement does not vary from user to user or from year to year. But the Sortino ratio, the Sharpe ratio, and similar ratios all depend on something called "risk-free return", which is wildly open to interpretation. This risk-free return varies from year to year and probably from user to user as well, making it very unreliable, and by extension, making the Sortino ratio unreliable as well.

We want a measure of performance that leaves no value open to misinterpretation because of different benchmarks among users, just as horsepower is not open to misinterpretation by those who know its definition.