the flaws of trade station output

[man]

i personally always thought that tradestation output is weak. so many dollar figures, so weak risk return coverage.
i tried to get used to profit factor as an ingredient, but did not grasp that this concept is the most flawed.

i think as a system trader you are finally interested in your sharpe ratio or your sortino, everything else is less important. now i always had the impression that profit factor and sharpe ratio will have some common ground. did not really give much thought to it, though. now i think profit factor got "It" completely wrong. a system with a profit factor above 2.5 can have a substantially lower sharpe ratio than a system with pf of 1.2. now that might have always been obvious to you, but it was not to me.

reason for this is, that a system which makes two trades everyday, one where it makes 120 dollars and one which looses 100 dollars, will have a profit factor of 1.2, but an infinite sharpe ratio, since every day it prints 20 bucks.

thnx profit factor, so long to you.

 

[Murray Ruggiero]

Quote from man:

i personally always thought that tradestation output is weak. so many dollar figures, so weak risk return coverage.
i tried to get used to profit factor as an ingredient, but did not grasp that this concept is the most flawed.

i think as a system trader you are finally interested in your sharpe ratio or your sortino, everything else is less important. now i always had the impression that profit factor and sharpe ratio will have some common ground. did not really give much thought to it, though. now i think profit factor got "It" completely wrong. a system with a profit factor above 2.5 can have a substantially lower sharpe ratio than a system with pf of 1.2. now that might have always been obvious to you, but it was not to me.

reason for this is, that a system which makes two trades everyday, one where it makes 120 dollars and one which looses 100 dollars, will have a profit factor of 1.2, but an infinite sharpe ratio, since every day it prints 20 bucks.

thnx profit factor, so long to you.

That is a very good point. What about MAR ? That also a good measure.
Another issue is are you evaluating the system with or without money management ?. Once you apply money management to a system you can see a different picture than you do trading a one lot.

 

[man]

all dividing by a figure that is based on small numbers of observations suffer the same problem: high standard error.

the worst is max draw down. this in my eyes the most overrated figure in system context. remember, it is a single observation. statistically it means nil. MAR, as far as i know, divides by the average of the worst three draw downs. well, that is three times more observations than sterling, but still ... huge standard error.

my point is that you need the full distribution of returns to judge what is currently going on. pure statistic will tell you that sooner or later you will profit more than you ever did and you will have a draw down higher than ever before ... your tails are rare but they exist and there is no reason why they should not be exceeded though the overall structure is still valid.

we had numerous discussions here on draw down buying. i think this is a dangerous and highly flawed concept. though i admit i seem to be the only one around with that kind of thinking. my point is that this draw down buying sets in at maybe a dozen observations on a ten year chart. already bearing huge standard error. when we tried to prove a mean reversion tendency on a high frequent base, thus every time "buying" in when the equity curve lost just a few percent, we did not find anything that made sense. now, when you start moving down to fewer and fewer observations, it starts to make sense, but that is a statistical fluke. you just look at an equity curve that ended more or less at the high. that is survivorship bias. plus your statistic suffers bigger and bigger standard error due to reduced observations. so you seem to increase the profitability of your system while in fact you are increasing uncertainty - or, to use a more familiar term, you increase the fitting element.

it boils down to buy in at the maximum draw down, which is almost by definition the best trade you could ever have made on your equity curve. now this trade IS standard error only ...

money management is crucial, crucial, crucial. as is portfolio management.

peace

 

[jerryz]

all else equal, isn't a higher profit factor better?

you gave an example of making $20 a day with 1.2 profit factor. using the same example with 2.5 profit factor and all else equal, you make $150 per day.

Quote from man:

i personally always thought that tradestation output is weak. so many dollar figures, so weak risk return coverage.
i tried to get used to profit factor as an ingredient, but did not grasp that this concept is the most flawed.

i think as a system trader you are finally interested in your sharpe ratio or your sortino, everything else is less important. now i always had the impression that profit factor and sharpe ratio will have some common ground. did not really give much thought to it, though. now i think profit factor got "It" completely wrong. a system with a profit factor above 2.5 can have a substantially lower sharpe ratio than a system with pf of 1.2. now that might have always been obvious to you, but it was not to me.

reason for this is, that a system which makes two trades everyday, one where it makes 120 dollars and one which looses 100 dollars, will have a profit factor of 1.2, but an infinite sharpe ratio, since every day it prints 20 bucks.

thnx profit factor, so long to you.

 

[man]

good question. i would say neither nor.

thus higher profit factor does not tell at all about whether i like that system more or not. my personal utility function is a smooth, upward sloping equity curve. the steeper the better, the smoother the better. now profit factor captures the steepness but tells literally nothing about the smoothness. so it is simply the wrong tool to judge a trading system. meaning, it is not just something i feel i should look in addition, it is simply misguiding. it does not tell me anything. it is like annualised return. who cares as long no "smoothnessComponent" is added ...

 

[jerryz]

if two systems have the same number of trades every month, the one with the higher profit factor will be more consistent in terms of having at least a positive pnl every month.

however, the size of the pnl depends on expectancy. a 1.5 pf system can have a higher expectancy than a 2.5 pf system. therefore, after the same number of trades, it is possible that the 1.5 pf system made more money than the 2.5 pf system. however, the equity curve of the 2.5pf system will be smoother.

therefore, given the same number of trades and the same expectancy, a higher profit factor is better.

do you agree?

 

[man]

Quote from jerryz:

if two systems have the same number of trades every month, the one with the higher profit factor will be more consistent in terms of having at least a positive pnl every month.

however, the size of the pnl depends on expectancy. a 1.5 pf system can have a higher expectancy than a 2.5 pf system. therefore, after the same number of trades, it is possible that the 1.5 pf system made more money than the 2.5 pf system. however, the equity curve of the 2.5pf system will be smoother.

therefore, given the same number of trades and the same expectancy, a higher profit factor is better.

do you agree?

frankly speaking i don't know. i do not understand your terminology. i mean how can you have different profit factor if you have same number fo trades and same expectency?

 

[jerryz]

actually i should revise my last sentence. if two systems have the same number of trades and same profit factor, the one with the higher expectancy is better.

Quote from man:

frankly speaking i don't know. i do not understand your terminology. i mean how can you have different profit factor if you have same number fo trades and same expectency?

 

[man]

Quote from jerryz:

actually i should revise my last sentence. if two systems have the same number of trades and same profit factor, the one with the higher expectancy is better.

still do not get it. profit factor is basically all money made divided by all money lost. dividing this figure by the number of trades would be what i would call expectancy for a single trade. so in my thinking the three make a perfect triangle. so if you keep two the same the third cannot differ.

 

[jerryz]

say you have two systems with the following characteristics:

both have 50% win rate.
System1: winners gain 2 and losers lose 1
System2: winners gain 4 and losers lose 2

Profit factor:
System1: .5*2/.5*1 = 2
System2: .5*4/.5*2 = 2

Expectancy:
System1: .5*2-.5*1 = .5
System2: .5*4-.5*2 = 1

these two systems have the same profit factor but different expectancy.

Quote from man:

still do not get it. profit factor is basically all money made divided by all money lost. dividing this figure by the number of trades would be what i would call expectancy for a single trade. so in my thinking the three make a perfect triangle. so if you keep two the same the third cannot differ.