You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
How do you backtest HFT strategies?
- Thread starter mizhael
- Start date
More or less, you can't. You pretty much need to come up with a reasonable idea in theory, and see how it acts once its on the order books using small sizes and throttling.
You need data (full book, resting orders, both stops and limits, as well as all market orders) and then you need to build an order execution engine that mirrors how the market acts. Then .you need to run your data through your order engine and compare the executions your order engine spits out to the actual executions that occurred. Assuming accurate data and an accurate execution engine, you're ready to test your system.
At that point you'd be able to introduce "new" orders into the system (these would be your orders) and assess your theory.
Not easy, not cheap.
At that point you'd be able to introduce "new" orders into the system (these would be your orders) and assess your theory.
Not easy, not cheap.
Quote from mizhael:
Is there a way to capture the market impact and do backtest realistically?
I'm not going to beg for you to reply to my PMs but I have sent you several and haven't heard back.
As other have said this is very complicated and not very easy. That said, you have open offers from people in NYC willing to help. Also, as for IP and giving out your trade, chances are I'm already running the same/similar strategy and wouldn't be interested in it - so in terms of offering help, its out there, on the table and free/no strings attached.
FYI, I moved offices, I'm in Midtown now (IIRC you are as well).
That is because mizhael is a posting-bot, not a real human. He exists solely to pump up the post count by starting countless stupid question threads. He is only a second generation posting-bot, so PM functiontionality has not been implemented yet.Quote from WinstonTJ:
I'm not going to beg for you to reply to my PMs but I have sent you several and haven't heard back.
Winston:
Do you mind sharing a few hints? I am starting to work with tick data and have the same issue.
The standard approach to count trades that go through your price is a start, but it does not account for fills that you might get at your price.
Trades marked as TFR should probably be ignored since many are internalized broker trades or dark pools that you could never get filled on.
Modeling each exchanges order book with recorded quote data is one thing, if you can get and process the data, but that still does not account for the HF firm that will jump your bid by a penny.
I guess we could use hidden orders to minimize market impact, but we will miss fills at other exchanges since our order is hidden and cant become the NBBO and has lower priority. Same happens if we trade less than a 100 share lot. It will not move the NBBO or even be reported as a trade.
The best I could do is estimate that I would get filled on 18% or so of the trades at my price or better. I got this number from comparing model trades to live trades.
Do you mind sharing a few hints? I am starting to work with tick data and have the same issue.
The standard approach to count trades that go through your price is a start, but it does not account for fills that you might get at your price.
Trades marked as TFR should probably be ignored since many are internalized broker trades or dark pools that you could never get filled on.
Modeling each exchanges order book with recorded quote data is one thing, if you can get and process the data, but that still does not account for the HF firm that will jump your bid by a penny.
I guess we could use hidden orders to minimize market impact, but we will miss fills at other exchanges since our order is hidden and cant become the NBBO and has lower priority. Same happens if we trade less than a 100 share lot. It will not move the NBBO or even be reported as a trade.
The best I could do is estimate that I would get filled on 18% or so of the trades at my price or better. I got this number from comparing model trades to live trades.
It really is hard. You have to set things VERY pessimistic and then burn some money by trading live and backtesting the same day. Get to know your backtest engine and get to know your tick data. You need to know each strategy and what is a realistic setting for entries or exits.
For example, different data sets are better/worse for trading - the better data sets for trading usually produce better backtest results while the poor data for trading (IB/Sterling, etc.) publish 'snapshot' quotes every 100ms and they tend to produce worse results in backtest - which are usually more accurate. It takes a long time to compare the different tick datasets, learn the differences and know how pessimistic to set your system to know what will be a realistic approach.
I know that's a bit vague however its hard to explain - reality is you need to have some drawdown money - $5-$10k per strategy. Turn the thing on, see what it does, do some backtesting, compare backtest to actual, do some optimization, tweak code, debug, turn thing back on, draw down more $$, backtest, compare backtest to 2nd round of actual, optimize, tweak code, debug.....
Rinse & repeat.
You have to understand what type of data you have, how it is published, how it behaves in backtest vs. what happens in actual real life, etc.
Hope that helps - I'll be happy to answer more questions if I missed things.
EDIT: Market orders - never use them. Limit orders - good but either provide liquidity or prepare to not get filled. If you plan on sending limits at the bid/offer you'll be at the back of the book and get crushed. This isn't very hard to simulate its just that most people don't fully understand the markets so they don't take into account fees, spreads, fills, etc. so they come up with something that is too good to be true.
I don't know that I feel comfortable discussing my entry/exit points in backtest or real strategies but its not very hard with good tick data.
For example, different data sets are better/worse for trading - the better data sets for trading usually produce better backtest results while the poor data for trading (IB/Sterling, etc.) publish 'snapshot' quotes every 100ms and they tend to produce worse results in backtest - which are usually more accurate. It takes a long time to compare the different tick datasets, learn the differences and know how pessimistic to set your system to know what will be a realistic approach.
I know that's a bit vague however its hard to explain - reality is you need to have some drawdown money - $5-$10k per strategy. Turn the thing on, see what it does, do some backtesting, compare backtest to actual, do some optimization, tweak code, debug, turn thing back on, draw down more $$, backtest, compare backtest to 2nd round of actual, optimize, tweak code, debug.....
Rinse & repeat.
You have to understand what type of data you have, how it is published, how it behaves in backtest vs. what happens in actual real life, etc.
Hope that helps - I'll be happy to answer more questions if I missed things.
EDIT: Market orders - never use them. Limit orders - good but either provide liquidity or prepare to not get filled. If you plan on sending limits at the bid/offer you'll be at the back of the book and get crushed. This isn't very hard to simulate its just that most people don't fully understand the markets so they don't take into account fees, spreads, fills, etc. so they come up with something that is too good to be true.
I don't know that I feel comfortable discussing my entry/exit points in backtest or real strategies but its not very hard with good tick data.
Quote from dloyer:
Winston:
Do you mind sharing a few hints? I am starting to work with tick data and have the same issue.
The standard approach to count trades that go through your price is a start, but it does not account for fills that you might get at your price.
Trades marked as TFR should probably be ignored since many are internalized broker trades or dark pools that you could never get filled on.
Modeling each exchanges order book with recorded quote data is one thing, if you can get and process the data, but that still does not account for the HF firm that will jump your bid by a penny.
I guess we could use hidden orders to minimize market impact, but we will miss fills at other exchanges since our order is hidden and cant become the NBBO and has lower priority. Same happens if we trade less than a 100 share lot. It will not move the NBBO or even be reported as a trade.
The best I could do is estimate that I would get filled on 18% or so of the trades at my price or better. I got this number from comparing model trades to live trades.
Thanks. Yes, it is a interesting problem.
Surprised that you find the snap shot data to be more useful than tick by tick data. I currently download tick data each night from IQFeed. It is cheap and better than I had before and avoids issues with data drops, etc.
Lots of trial and error...
Surprised that you find the snap shot data to be more useful than tick by tick data. I currently download tick data each night from IQFeed. It is cheap and better than I had before and avoids issues with data drops, etc.
Lots of trial and error...
Just saw your edit. Yes. Modeling a limit order fill is very frustrating.
Dealing with this much data is not easy either. I ended up building my own backtest code that runs on a map/reduce cluster.
The nice thing about is is that I can backtest any number of systems with one pass through the data rather than a pass per system.
Dealing with this much data is not easy either. I ended up building my own backtest code that runs on a map/reduce cluster.
The nice thing about is is that I can backtest any number of systems with one pass through the data rather than a pass per system.
Quote from dloyer:
Surprised that you find the snap shot data to be more useful than tick by tick data.
Its only because snapshot data is really bad and therefore presents a much more realistic worst-case backtest scenario. Spreads are wider, bid/ask sizes are smaller, prints are less frequent, etc.
Its similar to how everyone can identify penny/sub-penny arb situations on paper and become a millionaire but in practice they go so quickly it is not a tangable edge. Really good scrubbed tick data is very similar - if you were to simulate based on narrower spreads, more liquidity, more frequent and larger prints, etc. then your edge would seem better than real-life testing.
The biggest thing that snapshot data taught me is that it is garbage and if you are using a snapshot data provider its time to get a real data provider and just fire orders through any execution API. If you compare the two and use them properly you can increase your edge a bunch to the point that you are getting results much closer to the clean/good tick data vs. snapshot results.