Is Drawdown and Equity Curve Misleading

[Dhalsim]

If you are running backtests then most system software provides details such as percentage drawdown and dollar drawdown and so forth.

Also, informs you on minimum account size required to trade the strategy.

I find this massively misleading.

E.g.

I have one strategy - i have run the backtest on it. The max close to close drawdown from 2007 - 2014 was $1690 or 1.58%. This is based on $100,000 account size. This is the data generated from running backtest.

- Now this would be very silly to take these numbers at face value.
- The strategy is day trading strat and closes end of day so i take number ticks lost/won on a single trade.
- I then convert this to a percentage based on the entry-price - this way we know how much percentage we gained/lost relative to market (not relative to our account size).
- So this gives much clearer picture of draw-down you will experience and dollar loss.

- The biggest percentage loss was -2.52% relative to market price. This was on 23/10/2008. The dollar loss was only -$629 based on 1 contract traded.

- Now we know this is withing normal range of system so at current prices today then if the market plunges -2.52% and i take a similar loss then my actual dollar loss relative to market price would be 92.8 points on NQ (assuming 3681 last trade price).
This amount to net dollar loss of $1856 (triple what is would have been at 2008 market price).

- This is now a massive difference in dollar terms. Also, if we assume this was the very first trade made when system was started with $100,000 account size then we already have drawdown of 1.856 % (whereas data generated by backtest report shows max close to close drawdown of only 1.58%).

- This single day already proves that basic drawdown figures are useless and everything should be as percentage relative the current market price.

- If you work it my way then Max drawdown on this strategy based on percentage relative market price at time was -5.2%. This translates to 191 NQ points or -$3820 drawdown on the strategy. This is nearly double the system data.


Now for those of you who are using static dollar stops then this will prove disastrous if you dont change them to a percentage loss.
Or if you are using static tick amount as stop you will also suffer.
Just bear in mind that any strategy data these charting packages spew out can often be very misleading.

Moreover, Profit Factor figures (gross gains/gross loss) is also misleading. I might have a great 2013 therefore at current market prices generating massive dollar returns. However, in years when market price was much less then they will have much less impact on dollar gains/dollar losses. Therefore, taking the profit factor overall on the system is again useless and will be weighted favorably/unfavorably to years where market price is higher they by giving ability to generate higher dollar gains or losses.
-this is reason i only use profit factor per year now.
- people testing data 10 year or 20 years back must take this into account and use percentage weighted methodology.

Anyone have any thoughts on this??

 

[bln]

Dont have to make it harder than it is.. and it is a very importand metric.

Drawdown, often refered to as Peak-To-Valley drawdown is measured from max account equity value, this point often refered to as High-Water-Mark. The percentage is then dervived from this.

Account high water mark: $100,000
Account current value: $85,000

((85000/100000)-1)*100 = percentage drawdown

A good system should be regulary be setting new high water marks.

You will often be below is value as it is normal you have drawdown periods, often refered to as "being under water".

 

[Dhalsim]

Dont have to make it harder than it is.. and it is a very importand metric.

Drawdown, often refered to as Peak-To-Valley drawdown is measured from max account equity value, this point often refered to as High-Water-Mark. The percentage is then dervived from this.

Account high water mark: $100,000
Account current value: $85,000

((85000/100000)-1)*100 = percentage drawdown

A good system should be regulary be setting new high water marks.

You will often be below is value as it is normal you have drawdown periods, often refered to as "being under water".

I think you are missing my point. I understand drawdown as an important metric, what i am trying to demonstrate is that for intraday trading systems should measure profit and loss relative to the instrument price being traded.

Otherwise the drawdown amount will always be inaccurate and have far less meaning.

If i have strategy with 2% stop loss (2% based on drop in 2% of index not 2% of account value) then in 2008 when NQ market price was much lower this will translate to far less dollars than a 2% stop loss in 2014 at much higher NQ Market price.
Those of us back-testing in futures markets with single contract back test will have to take this into account when accessing the draw-down implications.

 

[ehsmama]

If you are running backtests then most system software provides details such as percentage drawdown and dollar drawdown and so forth.

Also, informs you on minimum account size required to trade the strategy.

I find this massively misleading.

E.g.

I have one strategy - i have run the backtest on it. The max close to close drawdown from 2007 - 2014 was $1690 or 1.58%. This is based on $100,000 account size. This is the data generated from running backtest.

- Now this would be very silly to take these numbers at face value.
- The strategy is day trading strat and closes end of day so i take number ticks lost/won on a single trade.
- I then convert this to a percentage based on the entry-price - this way we know how much percentage we gained/lost relative to market (not relative to our account size).
- So this gives much clearer picture of draw-down you will experience and dollar loss.

- The biggest percentage loss was -2.52% relative to market price. This was on 23/10/2008. The dollar loss was only -$629 based on 1 contract traded.

- Now we know this is withing normal range of system so at current prices today then if the market plunges -2.52% and i take a similar loss then my actual dollar loss relative to market price would be 92.8 points on NQ (assuming 3681 last trade price).
This amount to net dollar loss of $1856 (triple what is would have been at 2008 market price).

- This is now a massive difference in dollar terms. Also, if we assume this was the very first trade made when system was started with $100,000 account size then we already have drawdown of 1.856 % (whereas data generated by backtest report shows max close to close drawdown of only 1.58%).

- This single day already proves that basic drawdown figures are useless and everything should be as percentage relative the current market price.

- If you work it my way then Max drawdown on this strategy based on percentage relative market price at time was -5.2%. This translates to 191 NQ points or -$3820 drawdown on the strategy. This is nearly double the system data.


Now for those of you who are using static dollar stops then this will prove disastrous if you dont change them to a percentage loss.
Or if you are using static tick amount as stop you will also suffer.
Just bear in mind that any strategy data these charting packages spew out can often be very misleading.

Moreover, Profit Factor figures (gross gains/gross loss) is also misleading. I might have a great 2013 therefore at current market prices generating massive dollar returns. However, in years when market price was much less then they will have much less impact on dollar gains/dollar losses. Therefore, taking the profit factor overall on the system is again useless and will be weighted favorably/unfavorably to years where market price is higher they by giving ability to generate higher dollar gains or losses.
-this is reason i only use profit factor per year now.
- people testing data 10 year or 20 years back must take this into account and use percentage weighted methodology.

Anyone have any thoughts on this??

Just Look at Equity curve. It will tell the whole story. No need to think too much. one picture is worth 100000000 words

 

[Dhalsim]

I am just giving examples for people who backtest with set tick/point stops and how many errors with this kind of backtesting.

Lets say i was backtesting a strategy for CL then lets say i test with set 2 point stop.

In 2007 with CL trading at 57 this amounts to a 3.5% set stop.
At highs in 2008 at price 147, this same 2 point initial stop amounts to only 1.36%. This is a massive difference and hence reason why fixed stops should not be used as super easy to curve fit.

If we use % stop in same scenario then lets say we use intraday stop of 3.5%. So a drop in CL on a given day of 3.5% from entry would stop us out. In 2007 this 3.5% amounts to only -$2000. Whereas a 3.5% stop in 2008 near highs would amount to a massive -$5145.

This is only a one year difference and now we can see that fixed stop and % stop can be vastly different based on price instrument is trading at and therefore can skew results as well as making it super easy to curve fit.


-Me personally when i take drawdown into account i only look at percentage decline relative to market price. I feel that gives truer reflection of drawdown.

 

[trefoil]

It's what happens within a sample population. That's true for all of your results. Max drawdown is useful but in no way should be taken as the actual maximum you will suffer. It will almost always be more, because no matter what you are always only taking a sample of the total population, even if you get all the history you can get.
The 1987 crash went down twice as much as the '29 in a single day. Up until 1987, that 1929 crash was the worst. Now 1987 is. But that doesn't mean it's the worst possible, only that it's the worst that's happened up til now.
To paraphrase a famous person: Trust, but hedge.

 

[Dhalsim]

Here is an example of massive difference if we take each trade as percentage gain/loss based on the relative market price at time of trade.

Then take the % gain/loss on each trade and convert to a dollar value based on current market price (NQ today at 3687).

This chart on right is a truer reflection of system going forwards. Backtest in terms of $profit/loss is not a true reflection going forwards.

Thoughts?? Am i going over the top here??

 

[SIUYA]

I am just giving examples for people who backtest with set tick/point stops and how many errors with this kind of backtesting.

Lets say i was backtesting a strategy for CL then lets say i test with set 2 point stop.

In 2007 with CL trading at 57 this amounts to a 3.5% set stop.
At highs in 2008 at price 147, this same 2 point initial stop amounts to only 1.36%. This is a massive difference and hence reason why fixed stops should not be used as super easy to curve fit.

If we use % stop in same scenario then lets say we use intraday stop of 3.5%. So a drop in CL on a given day of 3.5% from entry would stop us out. In 2007 this 3.5% amounts to only -$2000. Whereas a 3.5% stop in 2008 near highs would amount to a massive -$5145.

This is only a one year difference and now we can see that fixed stop and % stop can be vastly different based on price instrument is trading at and therefore can skew results as well as making it super easy to curve fit.


-Me personally when i take drawdown into account i only look at percentage decline relative to market price. I feel that gives truer reflection of drawdown.

which is why you probably should introduce some sort of volatility element into stops if you want to equalise trades across time and instrument.
Alternatively - correct me if I am wrong. (third beer for the evening )
Your argument is essentially that even though at a price 100 or 1000 or 666 while a 2 tick stop has the same $ amount. (standardized contracts and all) then the % moves of the underlying are different. ....personally - not sure that makes any difference unless you are standardising % moves as part of your stop assessment.
It might mean your contract qty changes as the price changes, based on ATR or similar vol measures - and thats what that is for.

 

[Dhalsim]

Basically i am just saying when looking at your backtest results just be weary of your implied drawdown. As the chart shows a % loss in 2008 will have much different dollar value to the same % loss in 2014.

So backtest might have showed largest losing trade as e.g. -$900. Which might be 3% decline on that particular day relative to its market price. If we incur that same 3% in 2014 (which is possible as it has happened during backtest) then the dollar amount lost will be vaslty greater than -$900.

Its just one thing to keep in mind when accessing your strategy going forward e.g. you might have big losing trade of -$1000 when trading live. You look at your backtest report and it says over last 7 years your largest loss on a trade was -$900. You might thing f*** this strategy sucks. However, as a percentage loss at relative market price this may be a normal withing range loss.



Also, a lot of people myself included only backtest on futures markets with 1 contract unit. Maybe that is the wrong approach as it give unequal weighting relative to market price.

 

[gmst]

Here is an example of massive difference if we take each trade as percentage gain/loss based on the relative market price at time of trade.

Then take the % gain/loss on each trade and convert to a dollar value based on current market price (NQ today at 3687).

This chart on right is a truer reflection of system going forwards. Backtest in terms of $profit/loss is not a true reflection going forwards.

Thoughts?? Am i going over the top here??

Dhalsim, I wish to say "thank you" for making this thread. I am a profitable systematic trader using MC to build and trade strategies but I had missed this point. I have learnt something today because of you.