backtest variables, compare between two systems

Shay said:
regarding Sharpe Ratio, looks nice, but -
it has some disadvantages -
http://www.futuresmag.com/2013/02/01/sortino-ratio-a-better-measure-of-risk
More...
Yes, I too heard about it, but unfortunately had no time yet to study the difference.... ;-)
They are just some rough metrics to make system results comparable to other system results...
I would say PnL% of the invested total money, and MDD% of the invested total money or of PnL%, are the most important ones, IMO...
 
Some comments:
# consecutive winners
# consecutive losers
max drawdown
The idea with the above, is to help determine if current trading results are out of band with the backtest data. (for example: if the strategy over 100 trades had 4 consecutive losers, and your real trade is on the 5th consecutive looser, it may be time to suspect something is wrong!)
The equity curve/graph is the picture worth a thousand words!
Not suggesting these are all viable, but To see a "long list" of metrics to consider, you may reference some of Jake Bernstein's reports -- I think I saw some on UTUBE, with trade performance metrics out the gazoo. Some may have value.
 
Suppose you had an emergency and needed to cash out? On any given day, what are the chances that if you were to cash out, you'd do so profitably?

So another metric might be the total number of days the return was negative. One could prefer the system that gave the greatest odds of cashing out in the positive, on any given day.

Many variations of this can be made.

The meaningful comparison of two systems is dependent upon what you believe is important to you.
 
userque said:
Suppose you had an emergency and needed to cash out? On any given day, what are the chances that if you were to cash out, you'd do so profitably?

So another metric might be the total number of days the return was negative. One could prefer the system that gave the greatest odds of cashing out in the positive, on any given day.

Many variations of this can be made.

The meaningful comparison of two systems is dependent upon what you believe is important to you.
More...
i agree that some metric like this may be important to some people,
but for this specific test, i don't have such value.
 
Shay said:
What do you think we should check when doing backtest ?

Total PnL
% Profit trades
Largest Loss
Largest Win
Therotical % Profit

what else ?
More...


I don't find Sharpe ratios helpful, myself.

I also monitor MFE, MAE and Profit Factor.
 
Xela said:
I don't find Sharpe ratios helpful, myself.
More...
But as system tester you are more or less forced to provide it, because unlike you, some other people like it
because that metric is a widely used industry standard...
 
botpro said:
But as system tester you are more or less forced to provide it, because unlike you, some other people like it
because that metric is a widely used industry standard...
More...
... We're talking about comparing two systems. Not about blowjobs.
 
botpro said:
But as system tester you are more or less forced to provide it, because unlike you, some other people like it
because that metric is a widely used industry standard...
More...


I don't doubt that it would be expected of a professional "systems tester", but I saw no reference at all in the OP to that underlying the question asked here.
 
You must log in or register to reply here.
Back
Top