When you back tested it, did you make sure parameters of 2 or 3 from the parameters you are defining work as well? You want to see a "bell curve" of parameters above and below to work as well, perhaps not better but do work whereas if only one set of parameters works, system will break down as not being robust enough.
Just how reliable is backtesting?
- Thread starter nxt7
- Start date
Backtesting is not that reliable -- i don't want to make a cliche statement...but that's kind of like driving using your rear-view mirror. 
\
Trading is about...having vision and foresight
\Trading is about...having vision and foresight
Excuse the conentious tone, but not just you at all: some of us don't consider a single MA-based system worth backtesting to start with, because we wouldn't be willing to make trading decisions on its basis, going forward, however apparently impressive the results. It's like a pharmaceutical company commissioning a series of clinical trials: there has to have been some logical, reasonable reason for believing that the thing might be of some therapeutic value in the first place, otherwise your results won't be worth anything, and the very best they can ever show is correlation rather than causation.
Garbage In = Garbage Out
Computers are really dumb machines, as all they understand is 0 or 1!
If you introduce TIME into your studies, you might just discover some things that the majority fail to see, why, well the answer is simple, as in, they never bother to look!
TIME = MONEY
J_S
I somewhat disagree. Backtesting can be valuable if you have a large enough sample size...say 10,000+ trades.Backtesting is not that reliable -- i don't want to make a cliche statement...but that's kind of like driving using your rear-view mirror.
Trading is about...having vision and foresight
Also, having successful backtest results with multiple symbols provides even more assurity that it will work in the future.
I've seen many wanna-be's make the mistake of trying to backtest a system with a large time-in-trade value....like 5-20 days. In that case, with a single symbol, it's practically impossible to obtain a large enough set of trades to properly evaluate....using Sharpe, Sortinoe, or K ratios.
Backtesting is not that reliable -- i don't want to make a cliche statement...but that's kind of like driving using your rear-view mirror.
Trading is about...having vision and foresight
If you are driving backwards, you have to use your rear view mirror, and even turn your whole head round at times.
J_S
Then what do you use
There is no doubt that analyzing data is important to identify consistency, as trading decisions are best made in relation to easily identifiable repeatable events.
A repeatable event can occur with any instrument!
J_S
Reliability depends of the quality and honesty of your backtest. So it can be very reliable or not reliable at all.
A simple sample: signal when you cross the 200MA. That's very reliable, at least if you take every signal that is generated. The rule is simple and easy to check. You see the value of the 200MA, and the quote where you are now. No subjective interpretation is possible. Just compare two figures.
But if you want to trade on head and shoulder it is less clear. When is it exactly a head and shoulder, and what if it is almost a head and shoulder?
S/R is dificult to test objectively too. Lots of room for interpretation.
That's why I like statistics. Calculations always give a clear outcome. 1 is always 1 and can never be interpreted as something else.
If you have very clear and strict rules, reliability is high.
I did backtests and the results were similar to the results of realtime trading. But I took losses in backtesting because the rules told me to take the signal, although I saw already that it would be a loss as I saw the following periods already. Stay honest and don't cheat.
You using an automated trading system ?
If not, there's your answer because if you're not automated and your backtesting shows profitability but your real trading results shows a losing system...
That means there's obviously a psychological component that you can not backtest and it then shows up in your real trading. For example, maybe you have discipline problems, late entries, early entries, late exits or whatever.
Thus, backtesting is good in that it will reveal what you truly lack in your real trading. If it becomes a consistent problem...you may want to then turn your system into an automated system to remove the psychological variable.