Playing with monte carlos will not help you make a dime. Your reaction to this thread shows that you have invested way to much in failed analysis. The point that you failed to understand because of your knee jerk reaction that shows panic is that the author say that you need to develop your own systems because when you buy one you do not know how it was developed. If you wait for 30 trades to do your oversimplified monte carlos you may have experienced already a big loss. Then another one, then another one. When you find a good system with monte carlos you will maybe have no more money. besides monte carlos is stupid analysis. Are you one of those monte carlos analysts? Change your attitude and don't panic.
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
Good article about commercially available trading systems
- Thread starter Sergio77
- Start date
Playing with monte carlos will not help you make a dime. Your reaction to this thread shows that you have invested way to much in failed analysis. The point that you failed to understand because of your knee jerk reaction that shows panic is that the author say that you need to develop your own systems because when you buy one you do not know how it was developed. If you wait for 30 trades to do your oversimplified monte carlos you may have experienced already a big loss. Then another one, then another one. When you find a good system with monte carlos you will maybe have no more money. besides monte carlos is stupid analysis. Are you one of those monte carlos analysts? Change your attitude and don't panic.
Playing with monte carlos will not help you make a dime. Your reaction to this thread shows that you have invested way to much in failed analysis. The point that you failed to understand because of your knee jerk reaction that shows panic is that the author say that you need to develop your own systems because when you buy one you do not know how it was developed. If you wait for 30 trades to do your oversimplified monte carlos you may have experienced already a big loss. Then another one, then another one. When you find a good system with monte carlos you will maybe have no more money. besides monte carlos is stupid analysis. Are you one of those monte carlos analysts? Change your attitude and don't panic.
Okay.... The point I was making was 30 samples is not enough for a relevant study.. ....did you even read the entire thread, or are your trying to sell PAL subscriptions?
Last edited:
If 19/30 monte carlos produce returns that are LESS than your backtest, than your result is significant with 95% confidence, using the Binomial Distribution with a probability of success of 50%. All this means is you are 95% sure what is happening is not random......it DOES NOT mean 95% of your trades will be winners.
Probability of success of 50% doesn't mean that reward:risk is 1. What about if reward:risk is 5?
Also, what about if the system does trend-following? Does the MC make any sense?
"Set up a a monte carlo study that results in 15 entries uniformly distributed across the day, with holding period statistics that match your trades."
What does it mean to "set up a monte carlo that results in 15 entries uniformly distributed"?
OK, you are making some statements I'm not familiar with. Please elaborate.
Probability of success of 50% doesn't mean that reward:risk is 1. What about if reward:risk is 5?
Correct, but the risk:reward is irrelevent. All the monte carlo simulation is seeking to determine is if the returns produced by the "system" during backtest are likely or unlikely to have been produced by random chance. A market model isn't being varied, entries and exits are.
Also, what about if the system does trend-following? Does the MC make any sense?
I dont do trend following, but I think you should still get value out of it. If your trend following system is getting lucky, its likely random entries will to. You could even set a trend bias as monte carlo paramter (e.g 80% of entries should be long, +/- 10%...etc).
What does it mean to "set up a monte carlo that results in 15 entries uniformly distributed"?
Lets say your system produces an average of 15 entries per day (normally distributed, +/-3 trades 1 Sigma), and has an a average holding time of 5 minutes (laplacian distributed +/- 1 minute 1 Sigma). A monte Carlo simulation to establish confidence that your backtest returns were not produced by chance could look like this, for each day tested:
1. Set Random Number Generator Seed
2. Based upon system entry statistics, generate 15 (or so) entry times that are uniform-randomly distributed across the entire trading day.
2. Based on holding time statistics, generate holding periods for each trade
3. Backtest the trades with the same market data that the system was tested with.
4. Repeat for a new random seed.
See my response in the other thread about your other test. This test is also irrelevant if it is based on results from a backtested curve-fitted system. Most random systems will rank inferior to the curve-fitted system. Your true null hypothesis here is:
H0: Backtest returns are better than random returns
H1: Backtest returns are worse than random returns
As you can see if the backtest returns were curve-fitted to price series there is a high propability of Type I error.
I think you have to go back to drawing table. Your tests are based on misconceptions.
H0: Backtest returns are better than random returns
H1: Backtest returns are worse than random returns
As you can see if the backtest returns were curve-fitted to price series there is a high propability of Type I error.
I think you have to go back to drawing table. Your tests are based on misconceptions.
See my response in the other thread about your other test. This test is also irrelevant if it is based on results from a backtested curve-fitted system. Most random systems will rank inferior to the curve-fitted system. Your true null hypothesis here is:
H0: Backtest returns are better than random returns
H1: Backtest returns are worse than random returns
As you can see if the backtest returns were curve-fitted to price series there is a high propability of Type I error.
I think you have to go back to drawing table. Your tests are based on misconceptions.
Of course it is prone to curve-fitting, data snooping, and other biases. I guess I didn't explicitly state that the simulation should be done on out sample data, but I figured that was implied.... guess not.
You seem to have a lot of opinions on what doesn't work... I'd love to hear what does?
Of course it is prone to curve-fitting, data snooping, and other biases. I guess I didn't explicitly state that the simulation should be done on out sample data, but I figured that was implied.... guess not.
You seem to have a lot of opinions on what doesn't work... I'd love to hear what does?
Those are not opinions. They are facts based on established statistics. Note that you wrote:
A monte Carlo simulation to establish confidence that your backtest returns were not produced by chance could look like this,
So do I read this right or did you write one thing and you meant another? You talked about tests based on backtest results. All tests on backtest results are useless and tests on real results may be too late as someone else already mentioned. You have limited choices and nobody will tell you what works. You should be grateful that some people are willing to help you correct your mistakes. These mistakes are too basic.
Those are not opinions. They are facts based on established statistics. Note that you wrote:
So do I read this right or did you write one thing and you meant another? You talked about tests based on backtest results. All tests on backtest results are useless and tests on real results may be too late as someone else already mentioned. You have limited choices and nobody will tell you what works. You should be grateful that some people are willing to help you correct your mistakes. These mistakes are too basic.
http://www.elitetrader.com/et/index...ng-study-resources.201826/page-4#post-4139471
Just be careful you may be accused by him for promoting a website or book. I doubt he understands one word of what you said. He is busy flipping coins.
Thanks for the clarifications anyway. Very useful stuff to me at least.
Okay.... The point I was making was 30 samples is not enough for a relevant study.. ....did you even read the entire thread, or are your trying to sell PAL subscriptions?
I'm going to post links to useful websites that protect people from your likes no matter your intimidation efforts.
Here is another one for you and the vendors that design these stupid monte carlos tools:
http://eranraviv.com/multiple-testing/
This should teach you proper statistics and also tell you that the tools you may be promoting do not work.
Those are not opinions. They are facts based on established statistics. Note that you wrote:
So do I read this right or did you write one thing and you meant another? You talked about tests based on backtest results. All tests on backtest results are useless and tests on real results may be too late as someone else already mentioned. You have limited choices and nobody will tell you what works. You should be grateful that some people are willing to help you correct your mistakes. These mistakes are too basic.
Okay, so you are saying there is no value in back testing?