I noticed that there are a lot of bars misprinted while I backtested my 5-min chart (maybe due to bad data feed?) and their highs or lows go way way beyond what the normal range would be. this can really make my backtesting results inaccurate. Does anyone have the same problem when they backtest? How do I delete outlier bars from the test?
What to do with outlier bars when backtesting?
- Thread starter anh_trader
- Start date
You should back test using bid/ask tick data (BBO and Level II).
Last trade data is often out outside of the top of the order book BBO data. Also depending on the instruments you trade last trade prints may only account for less than 10% of the bid/ask ticks making your results a fantasy. ie. 6E can generates 12000 - 16000 price ticks with offers exceeding the highs and lows for a trading session. Last trades are reported only on a fraction of these price changes.. maybe 1000 - 2000.
Here are some suggestions:
1. Back test against BBO data.
2. Require price to go through and level II to show sufficient size for a limit order fill.
3. For Market orders use Level II data to calculate fills and use MIT to trigger release.
Many paper trading platforms use last trade to simulate fills which often lead to unrealistic results. Using last trade the spread is not taken into account resulting in fills that would never really happen.
Last trade data is often out outside of the top of the order book BBO data. Also depending on the instruments you trade last trade prints may only account for less than 10% of the bid/ask ticks making your results a fantasy. ie. 6E can generates 12000 - 16000 price ticks with offers exceeding the highs and lows for a trading session. Last trades are reported only on a fraction of these price changes.. maybe 1000 - 2000.
Here are some suggestions:
1. Back test against BBO data.
2. Require price to go through and level II to show sufficient size for a limit order fill.
3. For Market orders use Level II data to calculate fills and use MIT to trigger release.
Many paper trading platforms use last trade to simulate fills which often lead to unrealistic results. Using last trade the spread is not taken into account resulting in fills that would never really happen.
I use ninjatrader 7 to backtest and it has only 3 options for prices: ask, bid, last. If I use "last" option I would get my backtest results but with those outliers taken into consideration. The other 2 options, ask and bid, did not generate any results at all. Can you recommend any trustworthy backtesting platform?
Any meaningful backtest/tuning facility must have an efficient filtering system for the bid/ask series. Dealing with unfiltered data, makes results of no value.
And not only, also the spread should be carefully checked.
[Similarly, automated system must deal with highly filtered data feed (wrt to bid/ask/spread), to avoid being fooled by spikes or abnormal values, etc.]
There can be various types of filtering. Against outliers, against spikes, etc. (add!) All these issues become pretty obvious dealing with data tick by tick.
Tom
Thanks for the advice Tom, can you recommend any good backtesting platforms?
SIERRA
+
Excellent data feed, more important would be data though
Not really, being extremely demanding I tend to rebuild all the tools I need. [In the years, I have created my own complete trading/backtesting/tuning system, which also does all the possible filtering tricks imaginable and I can change it at any time, if new ideas or suggestions arise. I prefer to work (a lot) more at the beginning, to have, then, complete control and flexibility.]
I am sure here many people will recommend several good commercial platform. Let me know which one works best for you.
Tom
Hi friend,
as you can guess from my previous post, I have also realized a program to capture realtime data feed from my broker.
So I store it, to use later for testing purposes.
The data is fine for my purposes, and it gets anyway highly filtered by the trading engine (which in my case is also coincident with the backtesting/tuning engine).
Having "imperfect" data, is not a big problem, anyway. I see it as an opportunity. I think that spykes and gaps in the data may even "help" bulding more robust systems, as eventually, the reality will always find ways to surprise you
)As said, i prefer to rebuild all the tools, in order to maximize my control on each detail (and have the capability to adjust it at will, as my competence increases and new problems appear at the horizon).
Tom
Having "imperfect" data, is not a big problem, anyway. I see it as an opportunity. I think that spykes and gaps in the data may even "help" bulding more robust systems, as eventually, the reality will always find ways to surprise you
100% agree. A "real" system must learn to adapt to the imperfect world.