Trades and their order are not random. Tell that to any successful fund manager and they will laugh on the floor. A lot of energy and research goes into each and every trade. You can't just flip the order and simulate equity curves. Someone else already mentioned serial correlation I think. This fact renders MCS useless for trading system analysis due to the complexity involved in removing serial correlation.
Systematic traders! Hear ye! Hear ye!
- Thread starter abattia
- Start date
Trades and their order are not random. Tell that to any successful fund manager and they will laugh on the floor. A lot of energy and research goes into each and every trade. You can't just flip the order and simulate equity curves. Someone else already mentioned serial correlation I think. This fact renders MCS useless for trading system analysis due to the complexity involved in removing serial correlation.
@intradaybill,
imho - that's not correct. MCS stresstests show the inherent (hidden) risk of a trading system/concept. That risk isn't visible with one and only historical test.
Perhaps your "fund managers" laughed to loud some years ago, stumbling clueless in the big "financial crisis 2008"...
bye,
zentrader
I never look at the equity curve for backtests as the eye always sees what it wants to. If you're excited about the system, you'll only see the inexorable march higher in equity, ignoring the nasty looking pullbacks and the occasional 1yr+ drawdown periods.
I've looked at Kelly before, and it usually gives a number about 5 times that which I'm trading. The current accepted usage seems to be to dedicate a portion of your account, e.g. one fifth, to a strategy and then use that Kelly number, i.e. the same thing.
The main metrics for determining a system for me are return/drawdown, followed by Sharpe and Sortino. Win ratio is also important, as below 30% the system becomes difficult to stick with. Obviously each of these numbers is flawed in its own way, but it gives you a good flavour of the risk-adjusted return.
I've looked at Kelly before, and it usually gives a number about 5 times that which I'm trading. The current accepted usage seems to be to dedicate a portion of your account, e.g. one fifth, to a strategy and then use that Kelly number, i.e. the same thing.
The main metrics for determining a system for me are return/drawdown, followed by Sharpe and Sortino. Win ratio is also important, as below 30% the system becomes difficult to stick with. Obviously each of these numbers is flawed in its own way, but it gives you a good flavour of the risk-adjusted return.
True for faster traders. Trend followers can stick with w < 30% if their winners are very large. I agree with you neverthelees. I find it hard to stick with a system if the win rate is less than 50%.
Yes, sure trades are not taken randomly; thereâs usually (always!) a reason (a signal) for taking a trade.
But no manager can always know with certainty in advance the precise outcome of every trade; sometimes it will be a winner, sometimes a loser. This is what introduces something akin to ârandomnessâ...
We can expand further:
- Even if the manager knows the last trade was a winner, this doesnât determine always and with 100% certainty what the next trade outcome will be; other market events occur simultaneously with the potential to effect the outcome of the trade, and the manager cannot always have perfect information about all such events in advance; again, we have not-knowing-outcomes-with-certainty (and certainly not perfect serial correlation, if thatâs the correct term...).
- The equity curve the manager eventually obtains will be the result of all trades taken. Unfortunately though, the manager cannot know always in advance with 100% certainty what this equity curve will be.
So how might potential equity curves be analysed in advance? Doesn't not-knowing-outcomes-with-certainty allow for use of probabilities?
Vanilla MCS, other than as some sort of âfirst orderâ model, may be too crude (on account of the zero serial correlation assumption); but what would be wrong with methods applying probabilities to each trade outcome to generate âfamiliesâ of potential synthetic equity curves (as mentioned above by Steven.Davis)?
IMO the asnwer to this lies in the "many systems" concept. I think the analysis here
http://www.priceactionlab.com/Blog/2011/03/time-to-hire-a-monkey-not-really/
also applies to MCS simulations. If you randomize trade order, you essentially have different systems with uknown logic. Comparing the performance of a specific system to many unknown systems just doesn't make sense to me. I wonder why it makes sense to other people. It is like comparing the performance of your car to a large number of unknown cars. The information has no value.
A better test is removing trades with no replacement. I recall some old timers used this statistical test. Similar to bootstrap tests, I forget the name of it.
Hmm...
... when will you guys learn it???
For a MCS stress test you don't need the single trades sequence and randomize them. It's enough to have the system report results (trade ratio, pay-off ratio etc.) and you can simulate this system still having the same profit factor as before but generating other trades than in the one and only historical system test/back test!!!
That's the point and this alone shows normally e.g. higher risks (DDs) for your system than history. They call it MCS system simulation stress test and such a test is valid for remaining market conditions.
If you'll also simulate hidden risks for changing market conditions you can additionally simulate artificial price histories (based on the original data), do new system tests and new MCS stress tests.
If you a find a system, that don't have problems with both hurdles (MCS system simulation and MCS data simulation) you have a system found, that's very near to the holy grail...
Only my two cents - more information here:
http://www.zentrader.de/html/monte_carlo_simulator1.html
bye,
zentrader
This doesn't sound to me like a MCS. This is some kind of ad-hoc simulation. Can you explain to me how MCS is possible without resampling?
@intradaybill,
concerning the results you want to get from such a stress test (profit and DD expectations) it's the same whether you resample the single trades or you resample average trades, if you do a high number of simulation runs.
If the model is correct, the quantity structure is the important part of the MCS method!!!
example 1 (single trade list):
------------------------------------
win 10
loss 5
win 20
loss 40
loss 10
loss 25
win 80
win 50
...
or
example 2 (system report with average results):
-------------------------------------------------------------
wins: 4
losses: 4 => trade ratio of 0.50
average profit: 160/4 = 40
average loss: 80/4 = 20 => pay-off ratio of 2.00
Imagine you have not 8 trades, but 1,000 trades in your historical system test and simulate this test in 10,000 or more simulation runs (resulting in over 10 millions of trades).
You will get the same MCS results concerning possible profits or (more important) possible DDs - independent if you use example 1 or example 2.
bye,
zentrader
concerning the results you want to get from such a stress test (profit and DD expectations) it's the same whether you resample the single trades or you resample average trades, if you do a high number of simulation runs.
If the model is correct, the quantity structure is the important part of the MCS method!!!
example 1 (single trade list):
------------------------------------
win 10
loss 5
win 20
loss 40
loss 10
loss 25
win 80
win 50
...
or
example 2 (system report with average results):
-------------------------------------------------------------
wins: 4
losses: 4 => trade ratio of 0.50
average profit: 160/4 = 40
average loss: 80/4 = 20 => pay-off ratio of 2.00
Imagine you have not 8 trades, but 1,000 trades in your historical system test and simulate this test in 10,000 or more simulation runs (resulting in over 10 millions of trades).
You will get the same MCS results concerning possible profits or (more important) possible DDs - independent if you use example 1 or example 2.
bye,
zentrader
I think I will not agree this is a MCS. It is more like a bootstrap. MCS tests the hypothesis that trading system returns are paired with market returns in a random fashion. Bootstrap test the hypothesis that the mean return is zero. Bootstrap tests are very optimistic in that sense.
Regardless, I do not see how you can account for commission impact. When commission is included performance is a function of initial capital. I also do not see how you can account for serial dependency and market bias. I think tests that are based on performance measures that do not take into account actual market changes and pair them to trading system returns are useless and misleading because they can cause rejection of otherwise good systems.
Frankly speaking, what you are suggesting is an ad-hoc simulation that has no theoretical basis for it. Simulations make sense only when realistic inputs are used. They cannot be based solely on output data. Actually, I would not call your method even a simulation. It is more of a perturbation test of performance parameters. In that sense it is not only limited but also useless, IMO. In order for me to accept it, you must provide theoretical framework for it. I have seen websites using such ad-hoc "simulations", like collective2 but they do not make any sense to me. This is because, there is not one to one mapping from perturbation of performance parameters to resampled trading system results. Thus, the fundamental assumption of MCS of equal probability is violated.