How to use Backtesting the best?

[Ironplates]

Am finding it challenging to build a successful automated trading system in some instances. Especially if the back test results are very compelling.

I typically build systems to get as much [INSIDE THE BARS] as possible. I attempt to accomplish this by designing automated systems in such a way that it can deliver positive backtests using where short periodicity, such as Renko 3 brick sizes, 1 minute, 241 ticks, or 1 Range bar.

My reasoning is that if the system can work inside such a short periodicity it may increase the likelihood of the system working in longer periodicity ie. 12 to 24 brick sizes, 15-60 minutes, 4500 ticks, of ranges of 5 or larger.

In contrast, when running the system on replay or live, the same conditions that the back test reports where undesirable or led to poor results, the replay appears to find desirable or profitable for the short duration I test.

I would like to get a better understanding of the value a back test contributes to ultimate live results and the use or benefit of "walk forward" optimization.

Thank you in advance for your feedback and insight.

Have Backtests proven useful in the making of your automated trading systems?

 

[Sun Light]

in the back testing...you will find what you are looking for.
trick is not to try too hard and see what is reasonably obvious. But then that would have already factored in the market. No?

 

[Ironplates]

when you say "find what I am looking for" are you referring to "datamining" issues?

 

[Sun Light]

backtesting is datamining, isn't it?
You start with some hypothesis, code it, backtest/optimize it. You can curve fit anything in back data.
But Causation and Correlation are different things.

 

[traitor786]

Back testing can be dangerous if you do not understand the complex math.

What if you came up with a strategy that looked ok and at the same time I came up with one that looked amazing ?

If none were optimized, whos would you want to trade ?

The best one in my opinion is the one with the least optimized.

Say none were optimized after the first back test.

Then would you use mine over yours?

its all numbers right ?

What if I then told you mine was based on the stars and yours was based on twitter feeds from key people and key phrases were looked for giving an out put in the form of a number.

Would you switch back to your system ?

If you are not certain about the math, as I am not. I would attempt to be safe, minimize or eliminate the programming. Test on different sets of data. Once you have your results, do not go back and optimize. just accept the results. Also make sure you get similar results in one uptrend and another.

You should be aware of when you are messing with the math.

Let me know if I am rambling..

 

[traitor786]

Also, I am not sure if backtesting is needed. but, if it is needed, and used data is available, even a system that was not based on back testing should back test well. The results of such a system should blow your mind with no optimization.

 

[NetTecture]

Using NInjaTrader? Here is something for you: The platform SUCKS.

A lto fo your disappointment is working under false pretense. Renko backtest: does not work. Renko cuts of prices high and low, backtest in ninja does not see that - there you go. IN market reokay the price is outside the bars. Trade no 1 tick or use a better renko that has real high/low and uses open/close only as per renko rules - so high and low are real.

Slippage in Ninja? Standard fill type will never get you a price outside high/low of the bar, REGARDLESS of slippage setting. A better fill type needs to be coded, no that hard, but then.

The list goes on.

OBVIOUSLY when you test and optimize a system under such conditions - all other elements aside - it will fail under real life conditions, as you simply did not even look at the real market conditions during development.

 

[abattia]

Quote from NetTecture:
... Renko backtest: does not work. Renko cuts of prices high and low, backtest in ninja does not see that - there you go. IN market reokay the price is outside the bars. Trade no 1 tick or use a better renko that has real high/low and uses open/close only as per renko rules - so high and low are real ...

Add a secondary time series (e.g. ticks, or 1-second bars, etc) that's not Renko; your signal is calculated off the Renkos, and your fills off the secondary time-series. Works fine.

 

[TheMagican]

whats backhurting?

 

[traitor786]

Quote from abattia:

Add a secondary time series (e.g. ticks, or 1-second bars, etc) that's not Renko; your signal is calculated off the Renkos, and your fills off the secondary time-series. Works fine.

Really ?

I had this issue with Heinkin Ashi charts. Without understanding them I saw it was clear, flip positions every time a candle of opposite colour is closed. Was so happy it was a breeze!!! didn't have to learn to back test, was all manual.

Then I learnt that the candle was not real. This meant that all of a sudden I may get stopped out ! well that was not fun, I assumed since the whole system was based on fake candles it was garbage.

But I couldn't help but wonder if a buying pattern could be made so that the heinkin ashi chart is real !

The heinkin ashi chart I think is related to averages both open and close, possibly high and low. Instead of buying and getting prices that do not satisfy the candle. Maybe I could buy every 10 seconds leaving me with and average of the last 5 min bar. This may be equal to the heinkin Ashi candle.

Fees, entry time and a bunch of problems came to mind so it was not entertained and thrown away. I always wondered if they actually had any value in a more conventional way