Just how reliable is backtesting?

[Sergio77]

Literally just using 1 parameter and thats a moving average. I don't know how much simpler I can get

Not even close. One major issue with backtesting results is not the number of parameters or curve-fitted vs. over-fitted but is related to how many variations of systems you tried before the one you finally chose. Read this article that I think explains the issue well and in graphical form.

 

[Sergio77]

First you cant make a profit off a single MA unless you have serious price based logic going on aswell.

True. This is a very important paper on this subject. A must read.

 

[userque]

First you cant make a profit off a single MA unless you have serious price based logic going on aswell.

Likely covered, but your back testing with closed Candles, not tick data, it gives you a cheaty advantage, then it fails live.

I wasted 6 months chasing EAs and will never return and Im actually a programmer.

For shi+s and giggles, I did this using logic that dynamically adjusted the MA parameter based upon the recent price action. The EA determined the optimal factors associated with the formula to alter the parameter.

This simple, single-dynamic-MA-crossing-the-underlying system was profitable.

 

[kut2k2]

A good system works always in all market circumstances. The only thing that can influence the results is how big the trends are. If very trending, profits will be bigger and win rate will be higher, and in non trending markets profits will be smaller and win rate will be lower.
But the system on itself will always work ...

I don't believe there's any single system that works in all market conditions. A combination of systems would be required, one for trading trends and another for trading in choppy conditions. And a regime switch of some kind to determine when to use which system.

Something like

http://www.elitetrader.com/et/index.php?posts/2043606/

 

[benwm]

I don't believe there's any single system that works in all market conditions. A combination of systems would be required, one for trading trends and another for trading in choppy conditions. And a regime switch of some kind to determine when to use which system.

Something like

http://www.elitetrader.com/et/index.php?posts/2043606/

Did you trial CATSCAN (Choppy And Trendy Scanner)? Any experience with it, or did you build your own version along these lines?

 

[i am nobody]

I don't believe there's any single system that works in all market conditions. A combination of systems would be required, one for trading trends and another for trading in choppy conditions. And a regime switch of some kind to determine when to use which system.

A good system consists of subsystems that each are optimized for specific market behavior. And there is a switch between the subsystems depending on the trend. I wrote:Your system should be self-learning or self-adapting.
One system can have 1000 subsystems which make all together one system.

I trade one system that is selfregulating, which means that it adapts itself to the strenght of the trend. In a choppy market it takes small profits and when the trend is strong it can stay in for hours. I am daytrader.

 

[kut2k2]

Did you trial CATSCAN (Choppy And Trendy Scanner)? Any experience with it, or did you build your own version along these lines?

I have no experience with CATSCAN.

 

[Humpy]

When back testing on MT4 I notice that on some forex pairs they seem to work well but on others they makes losses. I suppose this indicates weaknesses of the EA and should be avoided.

 

[d08]

I don't believe there's any single system that works in all market conditions. A combination of systems would be required, one for trading trends and another for trading in choppy conditions. And a regime switch of some kind to determine when to use which system.

It's not a requirement but it can be beneficial. I run everything all the time, every system needs to at least produce a flat curve in adverse conditions. I've yet to find a reliable method to distinguish between trending and ranging - one without lag.

 

[userque]

For shi+s and giggles, I did this using logic that dynamically adjusted the MA parameter based upon the recent price action. The EA determined the optimal factors associated with the formula to alter the parameter.

This simple, single-dynamic-MA-crossing-the-underlying system was profitable.

Clarification:

I've assumed that people using "EA," were referring to "Evolutionary Algorithm." I now see that many here are using MetaTrader and EA likely means "Expert Advisor."

I don't use MT and my EA above actually means evolutionary algorithm.