Source(s) for 'most people do it wrong'?
has anyone ever backtested a method and then traded it profitably?
- Thread starter 1a2b3cppp
- Start date
Source(s) for 'most people do it wrong'?
I probably represent the opposite position (not for the first time): I've never traded any method or system without having backtested it extensively first.
But I also agree unreservedly with Kevin's post above (#8).
Those may look at first glance like two conflicting statements, but they're not.
It's by no means certain that systems/methods backtested with apparently successful results will produce profits when traded in real time with a funded account, but I still find backtesting supremely useful for excluding all the (great majority of) things that clearly don't have any genuine edge at all. Even if you look at backtesting purely as a way of removing all the things that don't work from potential further study/attention, it can be pretty valuable. My own opinion, though, is that it can also, for some people, be more useful than just that.
I did that on purpose, for 3 reasons:
1) There is no one right way to backtest. Lots of material out there from successful traders to review and possibly incorporate, though.
2) People need to think about the backtesting process more, instead of just slapping some rules together and optimizing. That is what most trading software directs people to do, unfortunately.
3) Ultimately, the method of backtesting has to lead to profit. The old garbage in, garbage out idea.
Personal observation, from e-mails I get and backtest results I see on various trading forums.
My personal favorite is still the guy who could not understand why his real time results looked nothing like backtest. He had a 3 month backtest. 51 trades. 47 optimized variables. He saw nothing wrong with this.
Shorter message this time...
I truly believe that those that say "I backtested and then it didn't work" are implying their backtested method had positive expectancy and it just didn't work in real money trading.
I strongly suspect many of them did not properly backtest. For example, you will see someone say I checked the historical charts going back 30 days, 60 days or briefly reviewed charts to see that it worked and the worst ones will say their backtesting involved using someone's else inadequate backtesting data that contains no information about the actual trade method...the latter means that real money trading is applied different in comparison to the backtested results...
The reason why I suspect this is because the ones that say the above...do you notice most of them do not present any data results. Just a few do. Then you have the few that backtested on one thing but then go into real money trading on something else. Then there's those that do not understand the drawdown stats in their backtest results. For example, pretend you have 10 years of backtested results with an average of 7 trade signals per day that produce positive expectancy you're comfortable with.
Yet, as soon as you get a few loses in real money trading especially by those with a small trading account (not properly capitalize)...loses that are within the expected drawdown stats...
The trader than "tweaks" (makes changes) to the trade method during real money trading for whatever reason in an effort to "optimize" the method to try to improve the performance to only see a worst performance in comparison to the backtest results. Trader then panics because he/she is losing real dollars and/or trading in a personal situation in which they can not afford to lose money...continues tweaking (making changes) during real money trading until the loses become unbearable or account has declined so low that they are not able to trade again unless they refund the account...
Resulting in the trader announcing the trade method does not work and/or announcing trading is too difficult along with developing a distrust for backtesting.
Simply, I think backtesting is misused and then some eventually just don't trust it.
wrbtrader
Last edited:
True and not true.
I am a discretionary trader and did/do a lot of backtesting. The backtesting was done manually, just following strict rules.
In real trading I cannot take my emotional state out of the equation as I am a discretionary trader. I have to click each time the mouse button and analyze the market to take the final decisions.
The backtesting had 2 goals:
- test the rules of the system
- built confidence so that my emotional state would not interfere, or at least as little as possible, in strictly following the rules.
Exactly. That's why I had to have excellent results in backtesting, as a system with a low winning rate, combined with a bad avg win/avg loss ratio would never give me enough confidence to switch off my emotions efficiently enough.
1 advantage for me was that this forced me to built a system with excellent entries, high % of winning trades and avg profit is a multiple of avg loss. Took me years to achieve that.
[has anyone ever backtested a method and then traded it profitably?]
Simple answer to the thread question - yes, I have.
It is my experience that most people new to trading think of backtesting in terms of automated optimization of fixed parameter indicators that results in them thinking they have found the holy grail but in reality they have found a curve fit clusterf@#$%k.
Backtesting trading systems as a component of the scientific process - Observation, Question, Hypothesis Experiment/Backtesting, Conclusion, is certainly beneficial. Why wouldn't it be?
Simple answer to the thread question - yes, I have.
It is my experience that most people new to trading think of backtesting in terms of automated optimization of fixed parameter indicators that results in them thinking they have found the holy grail but in reality they have found a curve fit clusterf@#$%k.
Backtesting trading systems as a component of the scientific process - Observation, Question, Hypothesis Experiment/Backtesting, Conclusion, is certainly beneficial. Why wouldn't it be?
IMHO the problem starts already in the way the trading system was built. Before starting to built a trading system you should first make an analysis of the market and of the way you want to built that trading system.
Most people start immediately with software, a number of indicators and start to optimize all the parameters. That will never work consistently. And it will give unreliable results from backtesting as in most cases the system is optimized for the backtesting period. Change the period and the results are different...
Most people start immediately with software, a number of indicators and start to optimize all the parameters. That will never work consistently. And it will give unreliable results from backtesting as in most cases the system is optimized for the backtesting period. Change the period and the results are different...