System Performance Score

Status
Not open for further replies.
Quote from nonlinear5:

Your "final SPS" is nothing but what you call "Bad Kelly" multiplied by W/L and adjusted for the number of trades. That's it. Again, this is a dubious "improvement". It can be easily shown that your formula overrates W/L. I'll leave the verification up to you.
More...
I PROVED that what I called "Bad Kelly" is not Kelly at all. Would it help you understand it better if I called it "Not Kelly"?

If the SPS values I calculated based on your systems don't make sense, you've yet to point out why.

Let's take another look at your systems:

System E : [-8, -8, -8, +25, +25]
System F : [-8, +8, +8, +168, +400]

If you seriously regard system F as only twice better than system E, you should spend more time critiquing your own measurement than mine.
 
Quote from nonlinear5:

For N >= 1000, min[1, N/1000] always results in 1.

So, it does not make any distinction between system A with 1,000 trades and system B with 10,000 trades. That's way too discontinuous, not to mention that the boundary of "1,000 trades" is too arbitrary.
More...
Then use the Eckhardt criterion instead. He explained why he chose 1800 trades.

But the idea of no upper limit is ridiculous. You can make a lousy system look good by simply padding your trade count when there is no upper limit to the trade count in your performance measure.
 
Quote from kut2k2:


Let's take another look at your systems:

System E : [-8, -8, -8, +25, +25]
System F : [-8, +8, +8, +162, +400]

If you seriously regard system F as only twice better than system E, you should spend more time critiquing your own measurement than mine.
More...

These are not the systems that I gave in my examples, and I never said anything about systems E and F. So, you are misquoting me, and it looks like you are looking to argue just for the sake of arguing. So I am out of here.
 
Quote from nonlinear5:

These are not the systems that I gave in my examples, and I never said anything about systems E and F. So, you are misquoting me, and it looks like that you are looking to argue just for the sake of arguing. So I am out of here.
More...
You kept calling different systems A and B. That would cause confusion when I put them all in one post, so I did the logical thing and relabelled some of them.

Here are the last two systems you posted:

[-8, -8, -8, +25, +25]
[-1, +1, +1, +21, +50]

As you yourself pointed out, the last system is the same performance-wise as

[-8, +8, +8, +168, +400]

Your obtuseness is deliberate and shameful.
 
Quote from gip3:

There's a lot of work backing up the use of sharpe ratio. It's also immediately relateable to statistical significance testing. Sure, it's got short comings, but we can analyze those short comings pretty well within a statistical framework (ie, what's the exact impact of kurtosis on the ability of sharpe to miscapture performance).

Your score has no theory to back up what it's measuring. In fact, it decomposes the trading outcome to a binary world: wins and losses. So I'm entirely at a loss as to why you would find it better? It makes a far more restrictive set of assumptions on the outcome of a trade, and there's no underlying rationale why your variables are laid out that way.
More...
The theory should be obvious based on what I've posted.

The SPS is the Kelly ratio multiplied by the average winning trade return. A logical trader wants both of those to be as high as possible, so increasing either increases the SPS value. I chose the LessBad Kelly formula for ease of calculation since it didn't matter if it was a dead-on match to true Kelly. I'm not trying to calculate true Kelly, I'm trying to develop a logical performance measure. Mission accomplished.

[-8, -8, -8, +25, +25]
[-8, +8, +8, +168, +400]

Your Sharpe ratio says the latter system is barely more than twice better than the former. That defies common sense.
 
Indicative (relative values) only:


System : [-8, -8, -8, +25, +25]
SPS 0.5

System : [-1, +1, +1, +21, +50]
SPS 6

System : [-8, +8, +8, +168, +400]
SPS 22
 
Quote from OddTrader:

Indicative (relative values) only:


System : [-8, -8, -8, +25, +25]
SPS 0.5

System : [-1, +1, +1, +21, +50]
SPS 6

System : [-8, +8, +8, +168, +400]
SPS 22
More...
I don't know what you're measuring but it's not SPS so please stop calling it SPS.

Wouldn't hurt to show your work either.
 
Indicative (relative values) only:


System : [-8, -8, -8, +25, +25]
Relative Performance Index: 0.5

System : [-1, +1, +1, +21, +50]
Relative Performance Index: 6

System : [-8, +8, +8, +168, +400]
Relative Performance Index: 22

Just for fun! :D
 
Quote from kut2k2:

[-8, -8, -8, +25, +25]
[-8, +8, +8, +168, +400]

Your Sharpe ratio says the latter system is barely more than twice better than the former. That defies common sense.
More...

First, I'm not sure what 'common sense' has to do with it. Common sense doesn't really tell me whether the second system is twice as good, or 100x as good.

Now, you are talking about performance comparison given risk aversion (which the sharpe and the classic kelly assume to be risk neutral). Given that risk aversion is subjective and varies individual to individual, I don't believe you've made any contributions there.

Finally, as you now want to do 'how much better' than just 'is it better', are you really saying that a system that does 600 trades is EXACTLY 1.2x better than one that does 500 trades? There's nothing to base this 1.2x being the right factor at all.

Anyway, you've yet to demonstrate how your SPS would rank systems significantly better than IR in any meaningful sense.
 
Status
Not open for further replies.
Back
Top