Trading System Analysis

C

CPTrader

Of the various statistics generated by trading system backtesting software (profit factor, avg winning trade, win/loss%, etc.) which ones do you consider most important and hy? Are there baseline you wish to see for these stats.

Also I recall hearing/reading of a formula called "Edge" that used some mathematical combination of profit factor, win/loss % etc ? Is anyone familiar with this and the exact formula.

Thanks.
 
IMO, the factors most important to me in a good trading system are:

1. Profitability (duh)
2. Lower drawdown
3. Percentage of winning trades >50% (so I don't 2nd guess the system)
4. Reasonable # of trades (so it is statistically valid)
5. Sharpe ratio >1.0 (makes me feel better)

And of course, you should have a robust trading system that you have back tested over the last 30 years with a similar walk forward analysis done on a daily basis which never loses $$ (yeah, right.)
 
...I believe we have corresponded before, so please forgive me if I repeat myself. My priorities are:

stability of the optimized parameters in backtests subsequent to the initial development

net profit after commission and slippage during the period of the testing

net profit per trade

maximum drawdown

ratio of net profit to total losses (R/R)

"quality" (linearity) of the equity curve

number of trades

maximum number of down days

the absurdity (originality) of the system.

Best regards.
 
If your system involves shorting NYSE stocks, then you should consider no fills/bad fills.

I tried to trade a certain kind of negative news method that looks great on paper, but with the demise of bullets, I only get short a fraction of the times that I attempt to.

The only way to test this type of thing is to put on small size trades and look at the results.
 
let me share with all one of my profound revelations.

your should choose perfomance statistics that are so mediocre that no one will want to take notice of them

why? arbitrage and evolution. everyone out there is looking at high %win, high profit factor system, and you can gurantee these system are discovered a hundred times everyday around the world, so as more and more money rush into these system, the profits get arb away. simple.

nobody is going to trade a system with 40% wins, 1.5 profit factor, and 40% drawdowns, and this is the exactly the reason their perforamnce is stable for decades, simply because no one trades them but someone has to do the dirty work.

a good analogy is that of the cockroachs. the cock hasn't change for 400mil years, beause there is simply no evoluationary pressure for it to adapt, why? because no one wants to be a cock, everybody wants to specailized, find their niche, but these enviroments are also very competitive and whatever edge you have get neutralized very quickly.

when you find a systhem that makes your eyeballs pops out, just remember that a thousand others is doing/have done/will be doing the same thing, then you realize your grail is really not that perfect after all..
 
Quote from vegasoul:

let me share with all one of my profound revelations.

your should choose perfomance statistics that are so mediocre that no one will want to take notice of them

why? arbitrage and evolution. everyone out there is looking at high %win, high profit factor system, and you can gurantee these system are discovered a hundred times everyday around the world, so as more and more money rush into these system, the profits get arb away. simple.

nobody is going to trade a system with 40% wins, 1.5 profit factor, and 40% drawdowns, and this is the exactly the reason their perforamnce is stable for decades, simply because no one trades them but someone has to do the dirty work.

a good analogy is that of the cockroachs. the cock hasn't change for 400mil years, beause there is simply no evoluationary pressure for it to adapt, why? because no one wants to be a cock, everybody wants to specailized, find their niche, but these enviroments are also very competitive and whatever edge you have get neutralized very quickly.

when you find a systhem that makes your eyeballs pops out, just remember that a thousand others is doing/have done/will be doing the same thing, then you realize your grail is really not that perfect after all.. [/QUOTE

Interesting (and amusing) comments which I share.

However, my question was asked with the view of determining how to assess sytems. No one has really addressed that....
More...
 
The normal stats are useful to see if you would be comfortable trading the system. To me the only truely useful stat is how much better than random the trades captured by the system are and whether the relationship is stable through time. If not, it's all a curve fit exercise guaranteed to bring disappointment as the market characteristics change.
 
I found this on a website (http://www.wintick.com/6_0/backtesting.asp?ID=34&Func=Report&strType=Stock):

(a) Winning Rate = ( Winning Trades / Total Trades ) • 100%.
(b) Win/Loss Ratio = Avg Winning Trade / Avg Losing Trade.
(c) Profit Factor = Gross Profit / Gross Loss.
Among above three factors, the Profit Factor is the most important. If Profit Factor > 1.0, it indicates that the trading system is a winning system. If Profit Factor > 1.5, it is a tradable system.
AbleSys Index is an overall performance factor, which equals the product of (a), (b) and (c), i.e., AbleSys Index = (a) x (b) x (c) If AbleSys Index = 0.5, the trading system is just breaking even. If AbleSys Index > 0.5, the trading system is a winning system. In general, AbleSys Index > 1.2, the system is a tradable system. Do not trade any symbol or chart with an AbleSys Index less than 1.2 even though the symbol or chart may be one of your favorites.


I think the AbleSys Index is what someone also referred to as Edge. Any opinions on the above?
 
CPTrader,

what kind of system are you developing ... monthly, weekly, daily, intraday, scalping? It all matters in terms of how to evaluate the system. Is your system trading futures, stocks, options or currency? I develop and trade intraday stock trading systems, and the only parameter that matters is cents per share traded. If that can exceed the commission per share, then you have a winning system.

Regards,

Slave2Market
 
CPTrader, I use (a), (b) and (c) in your above post, but didn’t know of this AbleSys, thanks for the tip.

Something that is extremely important for me, is what Kauffman calls in his book “Trading Systems and Methods”, the “efficiency rate”.

The efficiency rate is a/b, where:
a= your net profit
b= the sum of Abs(Ct – Ct-1) where Ct is the closing price of your index on day t.

At the beginning, I was looking for an equity curve as linear as possible. I realized that given the fact that my system is trend following and not intraday, and that the market I am trading isn’t always trending, asking for a linear equity curve was nonsense. Not only this, but it was a sure way to push me to overfit the data.

Instead, I try to keep my efficiency rate as constant as possible, it makes much more sense. If my efficiency rate is constant, then it means that a drop in PnL is accompanied by a drop in volatility of the index. And I’m fine with that: if the market doesn’t move, not making money is perfectly acceptable to me.

This concept of efficiency rate however wouldn’t work in a mean reversal system of course.
 
You must log in or register to reply here.
Back
Top