Trading system results overview

[alpha_monkey]

Quote from Trader666:

This is exactly why people need to stay away from metrics they don't understand. The examples "man" has given are perfectly plausible and shouldn't surprise anyone. Unless, of course, they're under the mistaken impression that Sharpe and PF measure the same thing.

Profit factor is a very simple, "bottom line" metric that is calculated by: total wins/total losses. It (as a metric) is NOT "diminished" with an "increased number of trades" AT ALL and will only decrease with more trades if the system's profitability decreases with more trades. It doesn't take variability of returns into account... just the bottom line.

Sharpe is a measure of reward vs. variability and the denominator of the formula can penalize large gains as well as losses. So a profitable system with a month of exceptionally high performance would be penalized by Sharpe but still have a high PF, while another system steadily returning a tiny amount over the risk-free rate would have a high Sharpe but a low PF (and bottom line).

I fail to see why PF and Sharpe need to be used on an either/or basis. Black and white thinking, to quote spike500.

Given their differences, what's wrong with casting an eye at both at different stages? PF for individual systems in development, Sharpe for the basket of systems in production.

Neither metric conquers all. Funds describe their returns via several different lenses.

AM

 

[man]

Quote from alpha_monkey:

I fail to see why PF and Sharpe need to be used on an either/or basis. Black and white thinking, to quote spike500.

Given their differences, what's wrong with casting an eye at both at different stages? PF for individual systems in development, Sharpe for the basket of systems in production.

Neither metric conquers all. Funds describe their returns via several different lenses.

AM

agree with your spirit. disagree with you on the current issue. the point is that whenever you start real intensive research you need to focus on one main output figure. in my case i have one strategy that consists of several single strategies and we choose at any point fo time automatically which one to trade in which parameterisation. now in order to do so we need a single figure by which we rank. you might have some additional figures like minimum of trades, but these are merely secondary variables. now using sharpe or figures alike for that purpose serves quite well. profit factor fails. period.

does that make pf wrong or meaningless as a secondary thing to look at? surely not. each and every additional aspect is great, welcome, "good" if you will. but this is not the discussion we are having here.

i have the feeling that many people with not too much experience are not aware of these circumstances and follow these sometimes a little weird and bashing discussions, left with the believe "finally they all are right ... somehow". but this is not the case i am afraid. some simply know more than others. (and while i personally claim that i know more about the current subject than some others within this thread i am completely aware about my minor position regarding others on the board. over the years i think people will realise that for example the kind participation acrary showed will never happen again. a pro opening up like that with no other interest for getting something back than a feeling of community ... once in a lifetime.)

why am i so insistant on this whole thing and not shut up? the initial problem of each journey is the target. if your target is vague ... the outcome will be. same in trading. what is your utility function? which figure is it that you want to maximise? i dare to say that almost every trader will finally maximise their sharpe. they might use five other different figures and may not even know sharpe as such, but they maximise their sharpe. why? well return can be easily changed by leverage. draw down? same thing? profit factor (i skip that ... reread the thread ) ...
sharpe does not change with leverage (actually it does a little due to some weakness of the concept, yet i find it bearable and the advantages offset the dis by far.).

usually what single system traders finally use very often is the judgement by looking at the equity curve. is this a strong concept, actually stronger than sharpe as a single figure? no doubt. our internal pattern recognition engine is telling way more than any set of figures. but, as someone pointed out earlier, when it comes to many many tests, you need some figures to use as filter before ... or, to be mor precise, o n e figure ...

 

[m4a1]

man, can you please give the source of your information on Buffet's Sharpe ratio? if you calculated it yourself, can you please show the calculations? did you just use data from 1999-2005?

The first link below is from a Wall Street Journal article written after 2005, and it says Buffet's sharpe ratio is 0.15.
The second link is a research paper that says Buffet's sharpe ratio is 0.786 using data from 1980 to 2000.

http://www.pfblog.com/digest/5063_buffetts_batting_average_is_sinking.shtml
http://66.102.7.104/search?q=cache:...ffet+"sharpe+ratio"&hl=en&gl=hk&ct=clnk&cd=90

Quote from man:

buffet has a modified sharpe of 1.54, with annual return of 22,43%. over thirty years with a multi-billion account you won't here too many comments by any pro that this is anywhere near to "terrible". one of the best, if not the best, quant shop out there, rentec, has a sharpe of 2.6 on their flagship medallion fund. very naive to think you match them when you are trading the same sharpe on a 50k account.

so, i am afraid, your post is as humorous as ungrounded.

other hint. if you had a backtest on intraday data like that:

1999 0.5%
2000 6.5%
2001 -6.2%
2002 10.0%
2003 21.0%
2004 10.5%
2005 6.4%

would you love to trade it?

these are the last seven years of berkshire hathaway. now, if you think that i think buffet doesn't do a great job, please return to the beginning of this post. after the third loop you are allowed to read on ...

 

[man]

did it myself. all years since 1966 out of the report on buffet's site. it is just annual data, but at 40 observations that is bearable.

1965 23,80%
1966 20,30%
1967 11,00%
1968 19,00%
1969 16,20%
1970 12,00%
1971 16,40%
1972 21,70%
1973 4,70%
1974 5,50%
1975 21,90%
1976 59,30%
1977 31,90%
1978 24,00%
1979 35,70%
1980 19,30%
1981 31,40%
1982 40,00%
1983 32,30%
1984 13,60%
1985 48,20%
1986 26,10%
1987 19,50%
1988 20,10%
1989 44,40%
1990 7,40%
1991 39,60%
1992 20,30%
1993 14,30%
1994 13,90%
1995 43,10%
1996 31,80%
1997 34,10%
1998 48,30%
1999 0,50%
2000 6,50%
2001 -6,20%
2002 10,00%
2003 21,00%
2004 10,50%
2005 6,40%

 

[m4a1]

omg man you are using accounting book values to calculate the sharpe ratio.

http://www.berkshirehathaway.com/letters/2005ltr.pdf

Quote from man:

did it myself. all years since 1966 out of the report on buffet's site. it is just annual data, but at 40 observations that is bearable.

1965 23,80%
1966 20,30%
1967 11,00%
1968 19,00%
1969 16,20%
1970 12,00%
1971 16,40%
1972 21,70%
1973 4,70%
1974 5,50%
1975 21,90%
1976 59,30%
1977 31,90%
1978 24,00%
1979 35,70%
1980 19,30%
1981 31,40%
1982 40,00%
1983 32,30%
1984 13,60%
1985 48,20%
1986 26,10%
1987 19,50%
1988 20,10%
1989 44,40%
1990 7,40%
1991 39,60%
1992 20,30%
1993 14,30%
1994 13,90%
1995 43,10%
1996 31,80%
1997 34,10%
1998 48,30%
1999 0,50%
2000 6,50%
2001 -6,20%
2002 10,00%
2003 21,00%
2004 10,50%
2005 6,40%

 

[man]

did i get something wrong? i mean it is basically a stock, so what would you use?

well, that was obviously a more stupid question ...

 

[man]

first column is the stock price, second is the book value.

return 23,62% 20,18%
stdv 29% 16%
mSharpe 0,82 1,25

stock book value
1988 59,32% 20,10%
1989 84,57% 44,40%
1990 -23,05% 7,40%
1991 35,58% 36,90%
1992 29,83% 20,30%
1993 38,94% 14,30%
1994 24,96% 13,90%
1995 57,35% 43,10%
1996 6,23% 31,80%
1997 34,90% 34,10%
1998 52,17% 48,30%
1999 -19,86% 0,50%
2000 26,56% 6,50%
2001 0,48% -6,20%
2002 -3,77% 10,00%
2003 15,81% 21,00%
2004 4,33% 10,50%
2005 0,82% 6,40%

sorry. my fault.

i do not have longer stock returns back. but probably overall the sharpe will be more near 1 than 1.5 since 1966.

overall i would keep up my argument up, though it is obviously not as strong as before.

 

[timmyz]

haha, so as it turned out, it was your post that was "humorous as well as ungrounded"

nobody point out his errors anymore, please. let him bask in his own arrogance and stupidity.

Quote from man:

buffet has a modified sharpe of 1.54, with annual return of 22,43%. over thirty years with a multi-billion account you won't here too many comments by any pro that this is anywhere near to "terrible". one of the best, if not the best, quant shop out there, rentec, has a sharpe of 2.6 on their flagship medallion fund. very naive to think you match them when you are trading the same sharpe on a 50k account.

so, i am afraid, your post is as humorous as ungrounded.

other hint. if you had a backtest on intraday data like that:

1999 0.5%
2000 6.5%
2001 -6.2%
2002 10.0%
2003 21.0%
2004 10.5%
2005 6.4%

would you love to trade it?

these are the last seven years of berkshire hathaway. now, if you think that i think buffet doesn't do a great job, please return to the beginning of this post. after the third loop you are allowed to read on ...

 

[man]

Quote from timmyz:

haha, so as it turned out, it was your post that was "humorous as well as ungrounded"

nobody point out his errors anymore, please. let him bask in his own arrogance and stupidity.

i am afraid my mistake will not cover yours.
and i am afraid that your comment on buffet
which you essentially used to raise questions
on the usefulness of shapre ratio still stands
as ... humorous and ungrounded.

unfortunately that has not changed.

 

[man]

and timmyz, please put me on ignore. the two of us are wasting each others time plus the attention of others. thnx.