Just how reliable is backtesting?

[kut2k2]

It's not a requirement but it can be beneficial. I run everything all the time, every system needs to at least produce a flat curve in adverse conditions. I've yet to find a reliable method to distinguish between trending and ranging - one without lag.

Are you saying that even a little bit of lag in one of your systems kills its profitability?

 

[d08]

Are you saying that even a little bit of lag in one of your systems kills its profitability?

It obviously lowers profitability considerably but it doesn't completely kill it. I'm saying that when conditions for a system are not favorable, it will lose very little or flatline the equity but that's alright because it outperforms when the conditions are right.

 

[i am nobody]

It obviously lowers profitability considerably but it doesn't completely kill it. I'm saying that when conditions for a system are not favorable, it will lose very little or flatline the equity but that's alright because it outperforms when the conditions are right.

That's why trying to know when markets are trending or not is the most important, but also the most difficult, part of building a system. If you know when markets are trending, or not, you have already achieved the biggest part of the job. That part will be responsable for the major part of the performance. Building the rest will be much easier as it is only timing of entries and exits.

 

[syswizard]

You do that? I would consider that system useless. The system should be adaptive and the only need for substantial modification would happen not more than 3-4 times per year.
Not much changes in 1 day when it comes to the structure of the markets, therefore daily optimization would be unnecessary.

I totally agree....especially this:"The system should be adaptive". For example, it needs to adjust for volatility. Small range bound conditions can really prove to be challenging.

 

[userque]

I totally agree....especially this:"The system should be adaptive". For example, it needs to adjust for volatility. Small range bound conditions can really prove to be challenging.

Yes, the system runs optimizations (adapts) internally. I don't manually optimize it daily. As that seems to matter to some here.

To be simpler: As many have pointed out, there is chop, there is trend, there is flat. A system that optimizes daily can adjust on a dime to such market conditions.

Also, it is based on an extremely customized kNN type algorithm that also utilizes machine learning ensemble methods.

 

[Buy1Sell2]

Backtesting lends itself to curve fitting. This is a weakness that most traders have.

 

[Buy1Sell2]

Best bet is to get in and trade. Your losses are called tuition. If not able top turn a profit in 6 months, it's best to try your hand ta something else.

 

[Buy1Sell2]

Forward testing is the only thing of any value in the learning process.

 

[Buy1Sell2]

Lose real money in the real market. That beats backtesting every trip of the train.

 

[i am nobody]

Backtesting lends itself to curve fitting.

And if it works in foresight too, you have an edge.