Limitations in backtesting

[TriPack]

oddiduro,

The statement "Any system can be made to work with money management, some work better in real time than with backtesting," has caught my attention.

Rather than asking for elaboration on this subject in the form of commentary and reasoning, I am wondering if you would put this to the test with a simple system of your choosing, such as a MA crossover or a breakout system, and show how a simple indicator loser can be turned into a winner with MM. I would be eternally grateful if you would be willing to do this in this or another thread, and I'm certain many other system traders would find it fascinating.

If you want I can provide a simple TradeStation system that loses money if this will get things started. If you have any minimum criteria for the system, let me know.

TriPack

Quote from oddiduro:



What I am speaking of specifically is that there are systems that backtest poorly and do well in realtime. Any system can be made to work with money management, some work better in real time than with backtesting.

There may be people out there throwing away good systems because of a marginal backtest.

I now advocate testing all systems in real time to falsify the loss as well as the gain.

Of course, this cost some money. But the results may be well worth it.

Regards
Oddi

 

[oddiduro]

Quote from TriPack:

oddiduro,

The statement "Any system can be made to work with money management, some work better in real time than with backtesting," has caught my attention.

Rather than asking for elaboration on this subject in the form of commentary and reasoning, I am wondering if you would put this to the test with a simple system of your choosing, such as a MA crossover or a breakout system, and show how a simple indicator loser can be turned into a winner with MM. I would be eternally grateful if you would be willing to do this in this or another thread, and I'm certain many other system traders would find it fascinating.

If you want I can provide a simple TradeStation system that loses money if this will get things started. If you have any minimum criteria for the system, let me know.

TriPack

I'm game

Provide your system. This should be an interesting learning experience.

Regards
Oddi

 

[WDGann]

Here's one:

Variable: MA(0), ATR(0);

MA = Average(Close, 10);
ATR = AvgTrueRange(10);

If MA > MA[1] then Buy at High[1] Stop;
If MA < MA[1] then Sell at Low[1] Stop;

ExitLong at EntryPrice - ATR Stop;
ExitShort at EntryPrice + ATR Stop;

Testing period has to be at least 3 years under an 1 min. bar basis. This is because you can take any short profitable (trending) period of a system and say you turned it around. And also it does a lot of trades to show enough samples.

Because it's a MM rule to be added, the entry/exit is hard set. No criteria is to be added or changed, for example adding another set of entry, stoploss exit or profit-taking exit.

Oddi, let's get it on!

 

[oddiduro]

Quote from WDGann:

Here's one:

Variable: MA(0), ATR(0);

MA = Average(Close, 10);
ATR = AvgTrueRange(10);

If MA > MA[1] then Buy at High[1] Stop;
If MA < MA[1] then Sell at Low[1] Stop;

ExitLong at EntryPrice - ATR Stop;
ExitShort at EntryPrice + ATR Stop;

Testing period has to be at least 3 years under an 1 min. bar basis. This is because you can take any short profitable (trending) period of a system and say you turned it around. And also it does a lot of trades to show enough samples.

Because it's a MM rule to be added, the entry/exit is hard set. No criteria is to be added or changed, for example adding another set of entry, stoploss exit or profit-taking exit.

Oddi, let's get it on!

Okay, wait........

If no stoploss or profit taking parameters are allowed, then how are we supposed to manage the money?

Perhaps we need to define what "money management" is. I may have a misunderstanding

Regards
Oddi

 

[TriPack]

Yes I thought stoploss and profit target were money management.

Oddiduro,

If you want to use Gann's system that's fine with me, though if Gann won't let you add MM then there's probably no point.

 

[TriPack]

As a backup system here are the simple rules: Buy at the highest high of the previous two bars + some constant on a stop. Do the reverse to go short. The system is always in (SAR). Feel free to add whatever MM rules you wish.

Apply to a 15 minute chart: (or you could pick another timeframe)

{ 
    feel free to optimize this input or make it a 
    variable and base it on volatility or whatever works
}
Inputs: StpAmt(1); 

Buy at Highest(High, 2)[1] + StpAmt Stop;
Sell at Lowest(Low,2)[1] - StpAmt Stop;

 

[rickty]

Quote from TriPack:

Yes I thought stoploss and profit target were money management.

Money Management (MM) means different things to different people. I primarily think of MM as "position sizing". That is, managing the number of contracts (futures) or shares (stocks). Others, obviously refer to MM as stoploss and profit targets.

Richard

 

[TriPack]

Quote from rickty:



Money Management (MM) means different things to different people. I primarily think of MM as "position sizing". That is, managing the number of contracts (futures) or shares (stocks). Others, obviously refer to MM as stoploss and profit targets.

Richard

I guess I should have said "were components of money management."

I don't claim to be any kind of MM expert, so maybe I'm off base and it is primarily position sizing and not stop loss and profit target. I'm just here to learn. Thanks for your comments.

 

[WDGann]

Quote from oddiduro:



Okay, wait........

If no stoploss or profit taking parameters are allowed, then how are we supposed to manage the money?

Perhaps we need to define what "money management" is. I may have a misunderstanding

Regards
Oddi

LOL...

OK... take only the:

MA = Average(Close, 10)

If MA > MA[1] then Buy at High[1] Stop;
If MA < MA[1] then Sell at Low[1] Stop;

3+ year data... 1 min. bar...

Please add other "MM" rules in your terms.

 

[oddiduro]

Quote from WDGann:



LOL...

OK... take only the:

MA = Average(Close, 10)

If MA > MA[1] then Buy at High[1] Stop;
If MA < MA[1] then Sell at Low[1] Stop;

3+ year data... 1 min. bar...

Please add other "MM" rules in your terms.

okay, now we can have a discussion

Your system is a beautiful example of a limitation in backtesting.

The backtest will show entries and exits at the end of the bar, but at the 1 minute level, we are scalping, from the looks of the system. Price is so sensitive that entries and exits intrabar will affect the viability of the system. If had to trade this, I would put a breakeven stop, and stop loss below the low of the second bar.

Your turn