How reliable is the backtesting result?
- Thread starter TAM
- Start date
supply vs demand.
Discipline, money management and a little common sense.
and you can trade anything.
keep it vanilla.
Discipline, money management and a little common sense.
and you can trade anything.
keep it vanilla.
Agree. Thanks
"Keeping it SIMPLE" is always my goal as well. This strategy only uses one major indicator (already performs well by itself) with a few minor controls to filter bad trades and minimize draw down.
My question is, from which part of the result that you can tell this strategy may not work in real life? This is the answer I want to hear most by posting the thread. Thanks
if you are considering realistic comms and slippage in the backtest (I heard you did), there are at least 1000 trade data points across several non correlated pairs and the three initial months trading live have a similar track record, I'd say you might have an edge there.
I don't like the multiple day holding period systems because they're more prone to crash when the market changes but your results seem quite interesting though, so i'd definitely keep it in my radar.
good luck
This looks interesting; I do my own work in Ninjatrader so I'm familiar with some of what you're doing.
Some questions -
What's your fill algorithm? Do you have it set to the "strict" method (that is, if you use limit orders, do you require trade-through prices to fill your limit?)
Slippage isn't calculated like you think in Ninja...for example, for futures you might specify 2 ticks of slippage. However, if NinjaTrader detects that you "could have" sold on the bid, it will not take into account any of the slippage you programmed in. If you want to test this, do something crazy like add +50 points of slippage, and watch your backtest results come out the same.
How precise is the data you're using for backtesting? That is, do you have not only "last" trades, but "bid" and "ask"? If Ninja doesn't detect bid or ask prices in your historical data, it will just go off of last. That doesn't help at all with limit orders, or even market orders.
I once used NeoTicker to develop systems. I had put a lot of time in to a specific system that appeared to work well across every instrument, with (literally) no parameter adjustment. I was totally stoked, and I actually put it to use trading live money.
What I discovered was that NeoTicker was using a "one touch" fill mechanism - that is, a price simply needed to have traded and you were filled. You can imagine, on a tick-by-tick basis, how this sort of algorithm can replicate a money-making machine.
In reality, getting filled is harder than you think. If your system uses time-in-force entry orders, then the fill algorithm for backtesting needs to be very strict to simulate your likelihood of getting filled (especially if you're not sitting in the queue for a while).
Just things to keep in mind.
Some questions -
What's your fill algorithm? Do you have it set to the "strict" method (that is, if you use limit orders, do you require trade-through prices to fill your limit?)
Slippage isn't calculated like you think in Ninja...for example, for futures you might specify 2 ticks of slippage. However, if NinjaTrader detects that you "could have" sold on the bid, it will not take into account any of the slippage you programmed in. If you want to test this, do something crazy like add +50 points of slippage, and watch your backtest results come out the same.
How precise is the data you're using for backtesting? That is, do you have not only "last" trades, but "bid" and "ask"? If Ninja doesn't detect bid or ask prices in your historical data, it will just go off of last. That doesn't help at all with limit orders, or even market orders.
I once used NeoTicker to develop systems. I had put a lot of time in to a specific system that appeared to work well across every instrument, with (literally) no parameter adjustment. I was totally stoked, and I actually put it to use trading live money.
What I discovered was that NeoTicker was using a "one touch" fill mechanism - that is, a price simply needed to have traded and you were filled. You can imagine, on a tick-by-tick basis, how this sort of algorithm can replicate a money-making machine.
In reality, getting filled is harder than you think. If your system uses time-in-force entry orders, then the fill algorithm for backtesting needs to be very strict to simulate your likelihood of getting filled (especially if you're not sitting in the queue for a while).
Just things to keep in mind.
The slippage I mentioned was not assumed, it was observed by comparing actually trades with back tested trades during the same period. It should be more meaningful since this is actual slippage.
The other thing is that I always use market order, not limit order. This may be why the slippage is only 3 pips on average.
I am still testing various versions of the similar strategy now. One thing I observed is that the simpler the strategy, the less discrepancy between real time and back testing. For example,one version of my strategy has returned over 21.43% in the past 45 days with 3.87% maximum drawdown so far. But I will definitely keep watching its real time performance for longer time horizon. I have my PC turned on 7/24 now
The other thing is that I always use market order, not limit order. This may be why the slippage is only 3 pips on average.
I am still testing various versions of the similar strategy now. One thing I observed is that the simpler the strategy, the less discrepancy between real time and back testing. For example,one version of my strategy has returned over 21.43% in the past 45 days with 3.87% maximum drawdown so far. But I will definitely keep watching its real time performance for longer time horizon. I have my PC turned on 7/24 now