Value of Backtesting and Stops
- Thread starter bluedemon77
- Start date
Anyone who says that testing using historical price data has no value is not thinking clearly.
How does one arrive at any strategy without knowledge of the past? How do you determine when to buy or sell?
What are the alternatives? Guessing?
All we have is what the markets have done so far. Even if we are trading live as a discretionary trader using only our personal experience as a guide for what to do in the future, we are relying on subjective back-testing.
It's not hard to argue that computers can more reliably test ideas using the same data. Backtesting works for the same reasons that smart traders can develop systems after years of experience in trading. They note repeating patterns that offer opportunities for profits. They then trade using ideas designed to capitalize on those opportunities.
You can do the same thing with a computer but with a more rigorous analysis. You will often find that ideas that appeared promising won't work because of something that you did not anticipate. It is much better to find this out using a computer than an actual account.
The reason that some people shy away from backtesting is that there are many ways to screw up backtesting and to find methods that appear to work that won't actually work in real markets. These problems are tractable if you pay attention to avoid the most common pitfall: over optimization.
Just because someone can use a knife to slit their wrists doesn't mean you don't want to use one in the kitchen when cooking. You just need to be careful with sharp instruments.
- Curtis
How does one arrive at any strategy without knowledge of the past? How do you determine when to buy or sell?
What are the alternatives? Guessing?
All we have is what the markets have done so far. Even if we are trading live as a discretionary trader using only our personal experience as a guide for what to do in the future, we are relying on subjective back-testing.
It's not hard to argue that computers can more reliably test ideas using the same data. Backtesting works for the same reasons that smart traders can develop systems after years of experience in trading. They note repeating patterns that offer opportunities for profits. They then trade using ideas designed to capitalize on those opportunities.
You can do the same thing with a computer but with a more rigorous analysis. You will often find that ideas that appeared promising won't work because of something that you did not anticipate. It is much better to find this out using a computer than an actual account.
The reason that some people shy away from backtesting is that there are many ways to screw up backtesting and to find methods that appear to work that won't actually work in real markets. These problems are tractable if you pay attention to avoid the most common pitfall: over optimization.
Just because someone can use a knife to slit their wrists doesn't mean you don't want to use one in the kitchen when cooking. You just need to be careful with sharp instruments.
- Curtis
Well put.
Just because it worked in backtesting doesn't mean it will work going forward. Butit seems to me if you can establish that something didn't work in the past, that's a good reason to avoid it in the future.
Every now and then someone comes along who makes it painfully clear for all the experienced traders to see, he doesnt know a damn thing about backtesting 
Anyone who pooh-poohs backtesting is ignorant of good mechanical systems development methodologies.
If you know proper backtest methodology, you can absolutely backtest a system which WILL work in the future 99% guaranteed. Good edges dont suddenly disappear the VERY DAY you start trading them (rolling eyes).
If they do, its because the edge was never really there and you dont know how to backtest properly.
Noobs assume that if you backtest and it fails live, that this somehow proves that backtesting doesnt work. This is purely ignorance and inexperience. There is a lot more to backtesting than simply running a fricken backtest. Christ.
This is like someone picking up a book on how to play a flute, reading chapter one, picking up the flute, making horrible sounds, and then proclaiming that flutes dont work.
Well guess what? Just because you buy tradestation, learn how to load some data, and backtest your first strategy, doesnt mean you know a damn thing about proper backtesting.
I already know how to drive a car, but Im no Mario Andretti. Just because I cant race a ferrari as fast as Mario can, do I have the right to proclaim that ferraris are not fast? Of course not.
I really should shut up and instead, convince everyone that backtesting is a stupid waste of time. I need more ignorant people pooh poohing on backtesting so the edges in the market remain there.
If you know proper backtest methodology, you can absolutely backtest a system which WILL work in the future 99% guaranteed. Good edges dont suddenly disappear the VERY DAY you start trading them (rolling eyes).
If they do, its because the edge was never really there and you dont know how to backtest properly.
Noobs assume that if you backtest and it fails live, that this somehow proves that backtesting doesnt work. This is purely ignorance and inexperience. There is a lot more to backtesting than simply running a fricken backtest. Christ.
This is like someone picking up a book on how to play a flute, reading chapter one, picking up the flute, making horrible sounds, and then proclaiming that flutes dont work.
Well guess what? Just because you buy tradestation, learn how to load some data, and backtest your first strategy, doesnt mean you know a damn thing about proper backtesting.
I already know how to drive a car, but Im no Mario Andretti. Just because I cant race a ferrari as fast as Mario can, do I have the right to proclaim that ferraris are not fast? Of course not.
I really should shut up and instead, convince everyone that backtesting is a stupid waste of time. I need more ignorant people pooh poohing on backtesting so the edges in the market remain there.
It is no suprise that 95% of the traders eventually blow up if I see the responses in this thread.
There is no substitue to understanding the markets and adjusting your play as the market unfolds. I have said it before and am saying it again: backtesting is based on the wrong premises. Yes with backtesting you may be able to define a mediocre system but that is all what it ever will be, mediocre.
It is like sitting at the poker table: if you do not know who the sucker is then you are the sucker. How are you going to "mechanically backtest" a poker game? 'nough said.
Old Chinese proverb: Those who say it cannot be done ought not to interrupt those who are doing it.
Holmes
There is no substitue to understanding the markets and adjusting your play as the market unfolds. I have said it before and am saying it again: backtesting is based on the wrong premises. Yes with backtesting you may be able to define a mediocre system but that is all what it ever will be, mediocre.
It is like sitting at the poker table: if you do not know who the sucker is then you are the sucker. How are you going to "mechanically backtest" a poker game? 'nough said.
Old Chinese proverb: Those who say it cannot be done ought not to interrupt those who are doing it.
Holmes
Obivously you have failed at backtesting, others havent. I have several kick ass systems, that are 100% mechanical, based on nothing more than backtesting. Mediocre system?? How does a 64% win rate with a 2.3 profit factor sound? How does 50% return a year with 6% drawdown sound?? Mediocre?
The ignorance around here is amazing. In case you didnt know, lots of people backtest poker. In fact, im aware of many people who make money on the poker sites using poker robots that they have "backtested".
Exactly, take this advice. There are highly successful backtesters out there.
Actually I tend to agree with your point about backtesting not being a substitute for market sense. Unfortunately, market sense takes a long time to develop, and most newbies will blow out during the process if they are following the typical advice dispensed in books and courses. Using backtesting to find out what doesn't work and to get a feel for the effect of various stop loss levels is invaluable.