I know, the term "median maximum drawdown" may sound confusing, so let me explain.
Suppose you have a mechanical trading system S which made N trades, with total return R and the maximum drawdown MD. Let's shuffle the trades (i.e. do a Monte-Carlo simulation). The trades are the same, but the order is different. The total return R will be the same, but the maximum drawdown would be different.
Now, the question is, which "maximum drawdown" should be used for the purposes of the trading system evaluation? There are several candidates:
1. The maximum drawdown from the original sequence of trades (that would be MD)
2. The maximum drawdown from the Monte-Carlo simulation (that would be greater or equal to MD)
3. The average maximum drawdown from the Monte-Carlo simulation (that could be greater or smaller than MD)
4. The median maximum drawdown from the Monte-Carlo simulation (that could be greater or smaller than MD)
I thought about this for a while, and it seems to me that measure (4), the median maximum drawdown, is the most meaningful. What it identifies is the most likely maximum drawdown when trades are reshuffled many times.
In this thread, I don't want to argue about the merits of mechanical or discretionary systems. Let's just see if we can figure out the optimal way to measure the system max drawdown. Thanks.
The maximum drawdown has severe limitations when it comes to measure risk and should be used with cautions. I recommend you reading this paper (http://alumnus.caltech.edu/~amir/mdd-risk.pdf) and the references therein.
In my opinion, Measure 1. (The historical drawdown) has little value as a descriptive statistics and even less value as a predictor of future drawdowns because it is a single number derived from a single sequence of trades and it is going to have a large error associated with it.
Using Monte Carlo simulation, you get the expected distribution of the maximum drawdown from which you can derive confidence intervals on the Maximum Drawdown. This is better as a descriptive statistics (for quoting past performance) but not a very good predictor of future drawdowns.
For a Brownian Motion, we can compute an analytical measure called the Expected Maximum Drawdown (see Magdon-Ismail and Amir Atiya paper) only knowing the average trade return and their standard deviation.
Although theoretical, I found this paper interesting because :
- It shows that the max drawdown increases over time (even for a profitable strategy) : In other words, a deeper drawdown than you already experienced is always ahead of you !
- It provides a scaling law which allows to compare MD quoted over different time intervals, resulting in "normalized" max. drawdown and Calmar ratio measures
- It shows the link between the maximum drawdown and the Sharpe ratio of a strategy.