I understand you can simulate different runs based on different values for winning/losing trades and average profit/losses. But how do you choose the values for the different runs--what's your algorithm for that? At the minimum it seems you're making some assumptions on the distribution of returns.
Help with Monte Carlo Drawdown analysis
- Thread starter MustPlayOptions
- Start date
I understand you can simulate different runs based on different values for winning/losing trades and average profit/losses. But how do you choose the values for the different runs--what's your algorithm for that? At the minimum it seems you're making some assumptions on the distribution of returns.
@NoWorries,
that is quite easy. Winning or losing trades are randomly selected (that's the basis of mcs...) and the model secures, that the basic relation is like in the original system test.
@MustPlayOptions,
my software reports an "Account Drawdown" (= account balance) of a simulation run. In a bad simulation run, the acount runs into negative numbers and that's your risk!
Perhaps the Wealthlab Report has another measurement for it's max DD (Difference highest account to lowest account etc...), based on the historical trades.
But such historical trades don't come again...
You have to"stress"- test your system, and not the historical data. This is the reason that I have another functionality in my software, called "data simulation". There new data can be generated ("data scrambling"), based on the historical data set, but with mcs methodology. Then you can test your system also with this alternative data sets and can do new backtesting and new mcs system simulations and so on...
...some work, but an optimized system development process!
bye,
zentrader
that is quite easy. Winning or losing trades are randomly selected (that's the basis of mcs...) and the model secures, that the basic relation is like in the original system test.
@MustPlayOptions,
my software reports an "Account Drawdown" (= account balance) of a simulation run. In a bad simulation run, the acount runs into negative numbers and that's your risk!
Perhaps the Wealthlab Report has another measurement for it's max DD (Difference highest account to lowest account etc...), based on the historical trades.
But such historical trades don't come again...
You have to"stress"- test your system, and not the historical data. This is the reason that I have another functionality in my software, called "data simulation". There new data can be generated ("data scrambling"), based on the historical data set, but with mcs methodology. Then you can test your system also with this alternative data sets and can do new backtesting and new mcs system simulations and so on...
...some work, but an optimized system development process!
bye,
zentrader
Quote from Volker Butzlaff:
[B@MustPlayOptions,
my software reports an "Account Drawdown" (= account balance) of a simulation run. In a bad simulation run, the acount runs into negative numbers and that's your risk!
Perhaps the Wealthlab Report has another measurement for it's max DD (Difference highest account to lowest account etc...), based on the historical trades.
But such historical trades don't come again...
You have to"stress"- test your system, and not the historical data. This is the reason that I have another functionality in my software, called "data simulation". There new data can be generated ("data scrambling"), based on the historical data set, but with mcs methodology. Then you can test your system also with this alternative data sets and can do new backtesting and new mcs system simulations and so on...
...some work, but an optimized system development process!
[/B]
I don't really know what to say to that. If I had 10k and trusted your max drawdown, I'd be broke. If I had known that the actual drawdown risk was _at least_ 16k, I would cut the trade size in half or use another strategy.
That's really what I'm trying to address - what is my actual risk. So to say that you should ignore the historical data really doesn't make any sense at all to me.
Thanks again for running it but the results just aren't close... Maybe with the full excursion data instead of the end of trade data it would - but you're program seems to ignore that information...
I think my point is this. Assume you have 2 systems that each do 4 trades, with returns as follows:
A. -10 +10 -10 +10
B. -15 +5 -5 +15
these are identical in terms of the inputs supplied to your program, and would lead to identical MC results.
Still system A and B are quite different in terms of the distribution of the trades (e.g. standard deviation and kurtosis), and a proper risk assessment should take that into account, producing different outcomes for both systems.
A. -10 +10 -10 +10
B. -15 +5 -5 +15
these are identical in terms of the inputs supplied to your program, and would lead to identical MC results.
Still system A and B are quite different in terms of the distribution of the trades (e.g. standard deviation and kurtosis), and a proper risk assessment should take that into account, producing different outcomes for both systems.
Quote from Volker Butzlaff:
@NoWorries,
that is quite easy. Winning or losing trades are randomly selected (that's the basis of mcs...) and the model secures, that the basic relation is like in the original system test.
@MustPlayOptions,
my software reports an "Account Drawdown" (= account balance) of a simulation run. In a bad simulation run, the acount runs into negative numbers and that's your risk!
Perhaps the Wealthlab Report has another measurement for it's max DD (Difference highest account to lowest account etc...), based on the historical trades.
But such historical trades don't come again...
You have to"stress"- test your system, and not the historical data. This is the reason that I have another functionality in my software, called "data simulation". There new data can be generated ("data scrambling"), based on the historical data set, but with mcs methodology. Then you can test your system also with this alternative data sets and can do new backtesting and new mcs system simulations and so on...
...some work, but an optimized system development process!
bye,
zentrader
...theoretical you are right, practical not...
Each Monte Carlo Simulation generates another result -that's because of the random component.
In the case of the "most black" swan possible (in the definition of Taleb) the system above can generate (theoretically) 824 loss trades x 268$ loss >= 200K $ Drawdown...
But the mcs based "system simulation" only shows an estimated result under random influences.
If you want to validate the system also under changed market conditions it's necessary to simulate also the data and use so called alternative data and test your system again. If the results remain stable, ok - if not think about it...
Only my two cents.
by,
zentrader
Each Monte Carlo Simulation generates another result -that's because of the random component.
In the case of the "most black" swan possible (in the definition of Taleb) the system above can generate (theoretically) 824 loss trades x 268$ loss >= 200K $ Drawdown...
But the mcs based "system simulation" only shows an estimated result under random influences.
If you want to validate the system also under changed market conditions it's necessary to simulate also the data and use so called alternative data and test your system again. If the results remain stable, ok - if not think about it...
Only my two cents.
by,
zentrader
@MustPlayOptions,
my software was originally developed as an additional test for systems with a high risk control (risk management). In this cases the values for highest loss and the average loss don't differ much.
Your system has an average trade loss of 268, but a high loss of 2259...
For these kind of systems a system simulation can have very big ranges of possible drawdowns.
I've changed the software a little bit to show more "user-friendly", how you can integrate such high losses (which differ much from the average losses) in a simulation, but without a change of the pay-off ratio of your system in this critical phase of losses. Perhaps the attached new gif helps a little bit, to show the scenario better.
The results:
a) with 99% statistical reliability the account dradown will be < 11,557
b) the highest account drawdown was even 33,512 !!!
(so you see the "worth" of statistical statements concerning risk)
But it remains an analysis of the known state (system only tested with historical data) An additional data simulation and new tests make sense anyway...
bye,
zentrader
my software was originally developed as an additional test for systems with a high risk control (risk management). In this cases the values for highest loss and the average loss don't differ much.
Your system has an average trade loss of 268, but a high loss of 2259...
For these kind of systems a system simulation can have very big ranges of possible drawdowns.
I've changed the software a little bit to show more "user-friendly", how you can integrate such high losses (which differ much from the average losses) in a simulation, but without a change of the pay-off ratio of your system in this critical phase of losses. Perhaps the attached new gif helps a little bit, to show the scenario better.
The results:
a) with 99% statistical reliability the account dradown will be < 11,557
b) the highest account drawdown was even 33,512 !!!
(so you see the "worth" of statistical statements concerning risk)
But it remains an analysis of the known state (system only tested with historical data) An additional data simulation and new tests make sense anyway...
bye,
zentrader
Volker,
If you don't use each single trade in your analysis, you leave out important information.
2 systems can have the exact same average winner/loser and number of winning/losing trades BUT they have a different distribution (standard deviation), your MC analysis will produce the same results but they are clearly not the same so you need to take each trade into account.
If you don't use each single trade in your analysis, you leave out important information.
2 systems can have the exact same average winner/loser and number of winning/losing trades BUT they have a different distribution (standard deviation), your MC analysis will produce the same results but they are clearly not the same so you need to take each trade into account.
I've currently come up with a way to calculate a necessary bankroll that makes sense to me, but it's not Monte Carlo based. I'm still interested in what a Monte Carlo analysis would show though if anyone wouldn't mind running one on the excursion data for me please.
The thing I had to take into account was the average and s.d. number of trades that are concurrently open. Without multiple open positions the bankrolls seem to make sense.
The bankrolls given can be pretty high though and I don't know if I'm way overestimating or not. The bankrolls I'm getting are a minimum of 3-4 times the max drawdowns given for the simulation runs and on occasion much much higher.
It wouldn't be much of an issue I guess if I could guarantee single open positions, but with some of the strategies I'm looking at in equities generate multiple limit order signals and than they either take or they don't and there may be many that take at once - and since you can't predict which ones will you can't really control it.
Thanks again for all the replies and help.
The thing I had to take into account was the average and s.d. number of trades that are concurrently open. Without multiple open positions the bankrolls seem to make sense.
The bankrolls given can be pretty high though and I don't know if I'm way overestimating or not. The bankrolls I'm getting are a minimum of 3-4 times the max drawdowns given for the simulation runs and on occasion much much higher.
It wouldn't be much of an issue I guess if I could guarantee single open positions, but with some of the strategies I'm looking at in equities generate multiple limit order signals and than they either take or they don't and there may be many that take at once - and since you can't predict which ones will you can't really control it.
Thanks again for all the replies and help.
@Virgin,
Performance:
2 or 3 seconds on my "3 years old" Pentium for ca. 25,000,000 trades (or 10,000 simulation runs)...
-----
"...If you don't use each single trade in your analysis, you leave out important information...systems can have the exact same average winner/loser and number of winning/losing trades BUT they have a different distribution (standard deviation), your MC analysis will produce the same results but they are clearly not the same so you need to take each trade into account..."
hmm...I don't agree. Why? Let me explain.
Imho there are simply two different risk scenarios concerning the use of trading systems.
1. you control your risk strictly (most of the loss trades will be executed thru stopp losses). In this case the method to use average losses to calculate max account drawdowns is sufficient, because there will be only a small diff between average loss and highest loss. That was the initial assumption fo my mcs implementation.
2. your risk management is relatively weak (the example here shows an average loss of 268, but a max. loss trade of 2,259...).
In this case I use this max. possible loss to calculate the possible risk of the system. To use the historical trades have no advantage. In the opposite: using the max.risk, which the system allows (without risk management), show the risk under bad circumstances.
Imho the characteristics of the system results are more important as the single historical trades. And because this system has to survive in the future and not in the past, it is very important to test also with additional synthetic data. System simulation is better than no simulation. But in combination with data simulation it is a very powerful weapon during the system development process.
by,
zentrader
Performance:
2 or 3 seconds on my "3 years old" Pentium for ca. 25,000,000 trades (or 10,000 simulation runs)...
-----
"...If you don't use each single trade in your analysis, you leave out important information...systems can have the exact same average winner/loser and number of winning/losing trades BUT they have a different distribution (standard deviation), your MC analysis will produce the same results but they are clearly not the same so you need to take each trade into account..."
hmm...I don't agree. Why? Let me explain.
Imho there are simply two different risk scenarios concerning the use of trading systems.
1. you control your risk strictly (most of the loss trades will be executed thru stopp losses). In this case the method to use average losses to calculate max account drawdowns is sufficient, because there will be only a small diff between average loss and highest loss. That was the initial assumption fo my mcs implementation.
2. your risk management is relatively weak (the example here shows an average loss of 268, but a max. loss trade of 2,259...).
In this case I use this max. possible loss to calculate the possible risk of the system. To use the historical trades have no advantage. In the opposite: using the max.risk, which the system allows (without risk management), show the risk under bad circumstances.
Imho the characteristics of the system results are more important as the single historical trades. And because this system has to survive in the future and not in the past, it is very important to test also with additional synthetic data. System simulation is better than no simulation. But in combination with data simulation it is a very powerful weapon during the system development process.
by,
zentrader