Trading Simulator - what to do with non-fillable orders?

N

NetTecture

I am writing the backend of a back tester these days. Tick by tick etc. blablabla ;) Anyhow, my problem is:

How would you deal with non fillable orders?

Basically, when an order is triggered I an generate fills according to the book (bids, asks).

My problem comes along when the book is not available and / or too small to accommodate the values. This can be because I do not have order book (only best bid/ask) in a tight market, or in general because the market IS tight (overnight, crazy market situation). Actually - it is a lot more relevant during night trading etc.

I do not want to keep those orders all around, and for a market order that would not be valid. Those are particularly very fancy times anyway (flash crash) and I want to actually make this somehow visible in a possibly run.

What would be your suggest way of handling those orders? Cancel them after filling what can be filled (and sending a cancel to the strategy, obviously), plus a logging message that can go into the backtest results?

I also consider a log message / info attachment to an execution to state how much of the order book was used to create the data. This normally should not be more than single digit percetnage, but a number higher than 100 would indicate the order was larger than the book, prompting an error in the backtest?

Asking for opinions here ;)

Also, how would you handle limit order queueing? My idea is that I track the queue location for every order, and decrease that
* On every trade at the price, by trade volume.
* When the book moves away in my favour, I assume I got all fills there.

The main problem is tracking queue position when out of the book - so I assume I just end up at the end always ;)?
 
Quote from NetTecture:

I am writing the backend of a back tester these days. Tick by tick etc. blablabla ;) Anyhow, my problem is:

Asking for opinions here ;)
More...

These questions seem to be pertaining to the order matching engine that you are using. Essentially, you program your OME with the rules of the exchange/exchanges you want to simulate as best you can.

You may need more than one OME running at the same time. All of these OME's are sitting behind your dummy/backtest Feed/Broker facades. So, as far as your strategy is concerned they look just like real feeds/brokers in the backtest.

A big but separate question is whether you want your backtest to be deterministic or non-deterministic i.e. will it have exactly the same results for identical backtests or will you introduce some randomness into your OME for filling orders (when you can't determine from the data you have how an order might be filled) and hence have a non-deterministic backtest which produces different results every time.

Of course, your OMS, Strategies etc. have to be able to handle partial fills/non-fills in live trading so it should behave the same way in back-testing. How you decide what to do with partial fills is up to your strategy - nobody can tell you that, but typically the strategy will be aware of partial fills via some kind of position manager and/or callback with execution reports. Obviously your strategy will be monitoring the book and incoming ticks and will know when best to cancel/replace an order that has not yet been filled as determined by the business rules you have given the strategy to execute.

Maintaining logs/annotations with each order and execution is a good idea in any case - partial fill or otherwise.

Good luck. Accurate back testing with depth of book replay and limit orders is not that easy.
 
Quote from Random.Capital:

Do whatever your exchange's order matching engine does.
More...

Non-answer. May I suggest you try thinking firt?

Remember that as a simulator I only have a specific book size, while the exchchnge has more. Also on real trades the market makers will react. When a stop is triggered then new counter orders normally come in. Bad fill maybe, but they DO come.

In the simulator, the fills are totally simulated - so I do not have the market makers "going in" when I simulate trades, and that is the whole problem.

if no counter order is available (extreme tight market conditions - like during the flash crash) I have no idea how the exchange will react because guess what - these conditions are not easily reproducable.

So, thanks for an answer contributing northing except a +1" to your posting count.
 
I've given a lot of thought to simulators and ways to get around the fundamental problem they have, where in theory you're able to fill infinite simulated shares against a real 100 share order...

I've come across 2 paths that seem to work best.

1. only 1 fill per position per price... if you want to buy 10,000 shares you hit the ask... if there's > 10,000 you get the full size... if there's less then you get that... the problem with this approach is that it gets complicated keeping track of what price levels you've already executed.

2. use only market orders and simulate sweeping the book... if you buy 10,000 shares and the stock is dry you would go through several price levels... getting you slippage and affecting your avg price. I like this approach since it is simpler, doesn't require keeping track of many levels and such... and it gets you a price that is worse than you will get in reality (since in a real exec you may get hidden liquidity)... I rather have a simulation throw "worse than reality" executions than the other way around...
 
Quote from eusdaiki:

I've given a lot of thought to simulators and ways to get around the fundamental problem they have, where in theory you're able to fill infinite simulated shares against a real 100 share order...

I've come across 2 paths that seem to work best.

1. only 1 fill per position per price... if you want to buy 10,000 shares you hit the ask... if there's > 10,000 you get the full size... if there's less then you get that... the problem with this approach is that it gets complicated keeping track of what price levels you've already executed.

2. use only market orders and simulate sweeping the book... if you buy 10,000 shares and the stock is dry you would go through several price levels... getting you slippage and affecting your avg price. I like this approach since it is simpler, doesn't require keeping track of many levels and such... and it gets you a price that is worse than you will get in reality (since in a real exec you may get hidden liquidity)... I rather have a simulation throw "worse than reality" executions than the other way around...
More...

I full agree with that - also I think complexity is not somethign to avoid.

The problem is that if you want to buy 10.000 shares and because it is a bad time the whole DOM has only 8000 shares. So, what do I do with the alst 2000? ;) In "the real world" the market makers would possibly throw in some additional orders as the order book disappears, but I have in the simulator - best case - the 10 level deep order book. Worst case I have only best bid / ask and if I go through that.... - can only guess.
 
Quote from NetTecture:

I full agree with that - also I think complexity is not somethign to avoid.

The problem is that if you want to buy 10.000 shares and because it is a bad time the whole DOM has only 8000 shares. So, what do I do with the alst 2000? ;) In "the real world" the market makers would possibly throw in some additional orders as the order book disappears, but I have in the simulator - best case - the 10 level deep order book. Worst case I have only best bid / ask and if I go through that.... - can only guess.
More...
Yeah, the headache is in the corner cases... :)
 
Quote from eusdaiki:

Yeah, the headache is in the corner cases... :)
More...

Now guess why I came here and asked ;)

The rest is a beginner programmer training in programming, but this stupid edge case...

Now, it should not happen (one should not trade those markets), but when a strategy gets trapped in a flash crash event, the simulator should somehow... well, provide sensible data.
 
Quote from NetTecture:

Now guess why I came here and asked ;)

The rest is a beginner programmer training in programming, but this stupid edge case...

Now, it should not happen (one should not trade those markets), but when a strategy gets trapped in a flash crash event, the simulator should somehow... well, provide sensible data.
More...
You could do what many HFT's did during the flash crash...


if liquidity dries up and things are way too weird then stop trading and ask humans for help :)
 
Quote from eusdaiki:

You could do what many HFT's did during the flash crash...


if liquidity dries up and things are way too weird then stop trading and ask humans for help :)
More...

No an option if:

* YOu have a position in the market
* You backtest.

And guess what - this is exaxctly what I talk about here. I dont want things popping up during backtests.
 
You must log in or register to reply here.
Back
Top