What techniques to market makers use to manage operational and 'massive' adverse selection risks?

[cjbuckley4]

Most exchanges do cancel on disconnect. There's some safety functionality when using the mass quote message on futures options. Not intuitive how to configure but if you read the docs closely you can figure it out. I wish they had it for futures.
CME

Thanks, @garachen. I found the docs for COD. Obviously, this isn't the only operational risk implicit in grabbing Q, but it's perhaps the most salient as helplessly watching a disconnect scenario unfold and run through your limits could quickly become a several 100K misadventure.

Any thoughts on how to mitigate the multi price level adverse selection risk? Or is that just the nature of the beast when it comes to MM? I couldn't think of anyway to prevent it from happening other than observing VPIN and stuff like that. I've read a lot of stuff questioning how well you can actually predict toxicity though. I suppose it's somewhat on me to implement it myself and "learn by doing" as I can test this with historical data. Even with a prediction of toxicity, to some extent, asymmetric information is just asymmetric information. How could observing the state of the book and time & sales tell you if some really large player instantaneously becomes willing to tolerate some market impact because they know something you don't--it really just can't. Fortunately, looking at my data, this scenario appears rather rare depending on the magnitude of market impact you're considering, but it could be really painful if you have size in the first two price levels.

Also, it's not something I've ever seen publicly discussed, so no worries if you don't want to break ground here, but there's gotta be some strategy to grabbing Q. For one, you the have the aforementioned adverse selection risk where either a big order sweeps several levels or you have sustained flow that functions like a big order because you can't cancel fast enough. Secondly, you have the Efficient Messaging Ratios to uphold (well I'm not sure if you do--one option market maker I interviewed with didn't seem to know what I was talking about, my guess because of some DMM agreement). So these risks make me feel intuitively like you're gonna want to space your quotes out by a couple price levels or at least have a couple hundred contracts behind you in the queue before you join at the next level. That being said, maybe there's a flip side of that coin. Maybe you have some kind of DMM mandate that makes you need to quote in a certain fashion (let's ignore that for discussion's sake unless it's actually a big deal) or maybe your strategy relies on some short term forecast that loses information content over time, so you want to squeeze many contracts in short term (without showing your hand) because your signal is active. I can think of other possible reasons, but I'll spare you my babbling.

By the way, I've kinda shelved the straight up quoting-both-sides MM approach in favor of more of a scalping type of deal that still relies on queue position because:
a) I can't see how I'll ever have the speed to get good queue position on the freshly created price level when a change in midprice occurs. I can do okay on my layered orders on levels 4-10 on a lot of products, but I'm screwed in "if you ain't first, you're last" scenarios like this.
b) I'm too slow to cancel if it appears that I'm one of the last guys offering on a side that's about to get taken out. Obviously, I'd only want to cancel if I hadn't been filled on the other side yet.
c) biggest issue: using a limit on both sides invalidates my backtest in a lot of ways (since I can't really afford to test things live, this matters). For example, suppose I know my max queue position for obvious reasons, I.e. FIFO. The max queue position worst case scenario only guarantees my limit is filled if the market trades through my queue position. If everyone cancels in front of me (we've talked about this privately), I could get filled without even knowing in a backtest and prices could move away from me. I can only guarantee a profit if price trades through my order on both sides, else there's uncertainty or at least assumptions made about the distribution of cancellations. With the scalping approach, I only attempt to exit for a profit with a resting limit (enter with marketable limit) so my worst case is really my worst case.

I know I always bombard you with long, not-so-lucid emails and ET replies. Thanks as always for your time and expertise, as well as the accidental mentorship your old posts provide. If this is any indication that I've been studying your old content, I'm sure you killed it on CL vol recently. Congrats. Let me know if I'm unclear on anything I've discussed above. I often find reading about HFT strategies and multiple price levels and stuff like that a bit confusing the first time through, and I'm sure I'm not always the best at explaining my ideas. Sincere thanks.

 

[winnertakesall]

I have been playing with automated market-making strategies in sim for some time and I am also looking to brainstorm for fresh ideas. Here is what I done so far.

1. I wrote a bot that submit limit order on both sides with a variable spread and inventory control. It seems to make stable profit for a few hours, then would lose many times that when one big adverse price movement comes later in the day.

2. I tried many versions of Stop-And-Reverse (the bot would stop providing liquidity and would start taking liquidity) to chase the momentum but that doesn't seem to work because of slippage.

3. If my simple market-making bot is such a consistent daily loser, why don't I just write a bot that takes exact opposite action to my inventory controlled market maker. This then just becomes a momentum scalper with position sizing. Well, sim shows that doesn't work either. If I need to chase momentum, I need to submit market order which introduces slippage. This accumulated slippage eats up most of the momentum scalper profit.

So how does a real professional market maker bot makes money?

1. They get paid by the exchange to stay in the market 80% of the time. That pay is their profit.

2. Traditionally, people thought market maker needs to be in front of the queue to make money. My observation suggests otherwise. HFT market maker literally cancels or revises order 30 times a second. Every time they do that, they become the last in line. And I think that's their edge in 2 ways. First, they get paid by the exchange for providing liquidity 80% of the time. So if they can do that while incurring minimum inventory, then the rebate becomes pure profit. Second cancelling or revising orders 30 times a second, I postulate, gives hft mm some advantage in minimizing the damage from adverse selection. I think I read somewhere the longer your limit order sits in the queue, the less expected profit if traded.

3. How can mm firms like Virtu have one losing day out of six whole years? Virtu says its diversification is key. I suppose it's possible. If some markets are toxic but most are not on one day, diversification will smooth out earning. But there are many days in which nearly all securities and futures headed down. How can mm make money on days like that. You can't. The best thing for a pure market maker is to just stay out on those days. But because mm are so close to so many markets at the same time, mm can decide fairly quickly when the entire market becomes toxic. Compare to if your market making bot only look at one market, then you might still be providing liquidity when dow, sp500, and nasdaq are already in sharp decline and you may thus incur large adverse inventory minutes later in your securities.

Let me know if you got ideas that i can try. And that's just my 2 cents.

 

[cjbuckley4]

I have been playing with automated market-making strategies in sim for some time and I am also looking to brainstorm for fresh ideas. Here is what I done so far.

1. I wrote a bot that submit limit order on both sides with a variable spread and inventory control. It seems to make stable profit for a few hours, then would lose many times that when one big adverse price movement comes later in the day.

2. I tried many versions of Stop-And-Reverse (the bot would stop providing liquidity and would start taking liquidity) to chase the momentum but that doesn't seem to work because of slippage.

3. If my simple market-making bot is such a consistent daily loser, why don't I just write a bot that takes exact opposite action to my inventory controlled market maker. This then just becomes a momentum scalper with position sizing. Well, sim shows that doesn't work either. If I need to chase momentum, I need to submit market order which introduces slippage. This accumulated slippage eats up most of the momentum scalper profit.

So how does a real professional market maker bot makes money?

1. They get paid by the exchange to stay in the market 80% of the time. That pay is their profit.

2. Traditionally, people thought market maker needs to be in front of the queue to make money. My observation suggests otherwise. HFT market maker literally cancels or revises order 30 times a second. Every time they do that, they become the last in line. And I think that's their edge in 2 ways. First, they get paid by the exchange for providing liquidity 80% of the time. So if they can do that while incurring minimum inventory, then the rebate becomes pure profit. Second cancelling or revising orders 30 times a second, I postulate, gives hft mm some advantage in minimizing the damage from adverse selection. I think I read somewhere the longer your limit order sits in the queue, the less expected profit if traded.

3. How can mm firms like Virtu have one losing day out of six whole years? Virtu says its diversification is key. I suppose it's possible. If some markets are toxic but most are not on one day, diversification will smooth out earning. But there are many days in which nearly all securities and futures headed down. How can mm make money on days like that. You can't. The best thing for a pure market maker is to just stay out on those days. But because mm are so close to so many markets at the same time, mm can decide fairly quickly when the entire market becomes toxic. Compare to if your market making bot only look at one market, then you might still be providing liquidity when dow, sp500, and nasdaq are already in sharp decline and you may thus incur large adverse inventory minutes later in your securities.

Let me know if you got ideas that i can try. And that's just my 2 cents.

Good to hear from someone with similar interests. I'll share a few thoughts.

1. I'm guessing the huge loses in directional markets is a function of a couple factors:
a) Adverse selection as a result of a comparatively poor forecast of intraday volatility.
b) Are you doing anything to quantify orderflow? Sustained flow is usually your queue to GTFO given the strategy you've detailed above.
c) The standard quote-both-sides and manage your inventory trade is really difficult to pull of at a retail level in my opinion. There's a myriad of reasons, but basically quoting both sides is just easy money if done conservatively and that leads to competition.
d) biggest factor as a retail trader that's probably gonna cause you issues here--ignoring the fact we just can't afford to properly hedge by quoting several related instruments to hedge asymmetric information--is poor queue position. Queue position is really where the intrinsic value of your limit order comes from. Quoting both sides is a bet that the queue behind you on side A holds up long enough for you to get filled on side B after you get filled on side A. If you're toward the end of the queues, the likelihood of this happening decreases. Furthermore, if you're the last or one of the last guys offering at a given level before it gets taken out, you start your trade one tick in the hole. You can see how this spirals out of control when mean reversion breaks down.
2. This is closer to my current approach. I'd be pretty surprised if I'm the first guy to have this idea, but as retail traders, we may actually be better prepared to compete for this trade than for straight up MM opportunities. PM me if interested. There's at least one glaring mistake in what you've described off the bat.
3. See part 2 and PM for details. Scalping is a dangerous game to attempt with retail commissions, but I think it might stack up to be a bit more in retail traders' favor than perfectly adhering to the O'Hara literature.

W/r/t part 2:
1. DMMs do well in some markets and don't exist in others. Since there's so much literature on when DMMs lose money, I've often thought they might present interesting opportunities to those of us without quoting restrictions.

2. As someone partially initiated into the pro market making industry (my job this summer), I must respectfully disagree. Queue position is a big deal. Several cancellations and modifications is not evidence to the contrary to this. Possible reasons for cancellations/modifications that have nothing to do with attempting to be at the back of the queue:
a) you got filled on a highly related instrument and don't want to double your exposure. Say you got filled on CLF16 offer, you're gonna probably want to pull your G16 offer quotes.
b) your signal is no longer active. Pro market makers don't just passively try to absorb whatever flow comes their ways, they have overlays to their strategies that dictate their quoting patterns.
c) your a sophisticated spread trader and you work limit entries. Your fill on leg 1 of this contrived two leg spread was incomplete, you need to modify your order on leg 2 to get the appropriate hedge ratio.
d) perhaps the single biggest reason: you predict you no longer have a good chance of completing the trade, namely queue on side 1 holding up long enough to get filled on side 2. Could be adverse selection risk in any number of forms, could be that your queue positions on each side are inconsistent with orderflow, could be that you have a leg up on how the cash market is trading and don't want to be some other sophisticated traders' counterparty. Could be something entirely different.

Cancellations alone can't be considered evidence of the back of the queue being the place to be.

Further evidence of this you can observe from the limit Order book: the shape of the book. You see how on most liquid products you have n contracts offered at level 0 (the BBO), significantly more offered at levels 1-2 and then levels 3-9 tail off to some fixed size? All those fixed size guys on levels 3-9 are just GTCs trying to hold queue position before the 1-2 guys pile on when their respective 3-9 PriceLevel becomes the BBO.

The only real advantage I can think of to being last in the queue is that you can easily ascertain your queue position. It becomes harder the closer you get to the front if more people pile on behind you who are constantly canceling and modifying.

3. Diversification is a part of the picture, but I interviewed at an options market making firm that does well and doesn't touch a thing besides ES and SPX options and their underliers.

 

[garachen]

Thanks, @garachen. I found the docs for COD. Obviously, this isn't the only operational risk implicit in grabbing Q, but it's perhaps the most salient as helplessly watching a disconnect scenario unfold and run through your limits could quickly become a several 100K misadventure.

Any thoughts on how to mitigate the multi price level adverse selection risk? Or is that just the nature of the beast when it comes to MM? I couldn't think of anyway to prevent it from happening other than observing VPIN and stuff like that. I've read a lot of stuff questioning how well you can actually predict toxicity though. I suppose it's somewhat on me to implement it myself and "learn by doing" as I can test this with historical data. Even with a prediction of toxicity, to some extent, asymmetric information is just asymmetric information. How could observing the state of the book and time & sales tell you if some really large player instantaneously becomes willing to tolerate some market impact because they know something you don't--it really just can't. Fortunately, looking at my data, this scenario appears rather rare depending on the magnitude of market impact you're considering, but it could be really painful if you have size in the first two price levels.

Also, it's not something I've ever seen publicly discussed, so no worries if you don't want to break ground here, but there's gotta be some strategy to grabbing Q. For one, you the have the aforementioned adverse selection risk where either a big order sweeps several levels or you have sustained flow that functions like a big order because you can't cancel fast enough. Secondly, you have the Efficient Messaging Ratios to uphold (well I'm not sure if you do--one option market maker I interviewed with didn't seem to know what I was talking about, my guess because of some DMM agreement). So these risks make me feel intuitively like you're gonna want to space your quotes out by a couple price levels or at least have a couple hundred contracts behind you in the queue before you join at the next level. That being said, maybe there's a flip side of that coin. Maybe you have some kind of DMM mandate that makes you need to quote in a certain fashion (let's ignore that for discussion's sake unless it's actually a big deal) or maybe your strategy relies on some short term forecast that loses information content over time, so you want to squeeze many contracts in short term (without showing your hand) because your signal is active. I can think of other possible reasons, but I'll spare you my babbling.

By the way, I've kinda shelved the straight up quoting-both-sides MM approach in favor of more of a scalping type of deal that still relies on queue position because:
a) I can't see how I'll ever have the speed to get good queue position on the freshly created price level when a change in midprice occurs. I can do okay on my layered orders on levels 4-10 on a lot of products, but I'm screwed in "if you ain't first, you're last" scenarios like this.
b) I'm too slow to cancel if it appears that I'm one of the last guys offering on a side that's about to get taken out. Obviously, I'd only want to cancel if I hadn't been filled on the other side yet.
c) biggest issue: using a limit on both sides invalidates my backtest in a lot of ways (since I can't really afford to test things live, this matters). For example, suppose I know my max queue position for obvious reasons, I.e. FIFO. The max queue position worst case scenario only guarantees my limit is filled if the market trades through my queue position. If everyone cancels in front of me (we've talked about this privately), I could get filled without even knowing in a backtest and prices could move away from me. I can only guarantee a profit if price trades through my order on both sides, else there's uncertainty or at least assumptions made about the distribution of cancellations. With the scalping approach, I only attempt to exit for a profit with a resting limit (enter with marketable limit) so my worst case is really my worst case.

I know I always bombard you with long, not-so-lucid emails and ET replies. Thanks as always for your time and expertise, as well as the accidental mentorship your old posts provide. If this is any indication that I've been studying your old content, I'm sure you killed it on CL vol recently. Congrats. Let me know if I'm unclear on anything I've discussed above. I often find reading about HFT strategies and multiple price levels and stuff like that a bit confusing the first time through, and I'm sure I'm not always the best at explaining my ideas. Sincere thanks.

There's the normal scenario and the extreme scenario. During the normal scenario there are many techniques available to apply to product microstructure to identify toxic flow. Things like adapting VPIN to be more reactive and relevant.

In the extreme scenarios you have to know where you can lay off risk the fastest. The techniques for doing this are heavily dependent on the exchange/matching engine/product and require a good inderstanding of... everything. There's no publicly available data that can help here. You really need accurate time stamps on the (possibly) multiple exchanges feeds plus your TCP sessions with them. And then do a lot of guessing and testing until you get it right. And this stuff changes frequently.

It might be possible to market make without dealing with the extreme cases. Some products more than others. But I'm not sure.

 

[cjbuckley4]

There's the normal scenario and the extreme scenario. During the normal scenario there are many techniques available to apply to product microstructure to identify toxic flow. Things like adapting VPIN to be more reactive and relevant.

In the extreme scenarios you have to know where you can lay off risk the fastest. The techniques for doing this are heavily dependent on the exchange/matching engine/product and require a good inderstanding of... everything. There's no publicly available data that can help here. You really need accurate time stamps on the (possibly) multiple exchanges feeds plus your TCP sessions with them. And then do a lot of guessing and testing until you get it right. And this stuff changes frequently.

It might be possible to market make without dealing with the extreme cases. Some products more than others. But I'm not sure.

Thanks for the clarification and opining a bit on the different scenarios and how they're handled. Appreciate the help as always. I'll keep your comment on improving metrics for toxicity in mind.

 

[Gambit]

Thank you both for the fascinating discussion. I just bought Maureen O'hara's book on market microstructure for an understanding of the underlying theory.

 

[cjbuckley4]

Thank you both for the fascinating discussion. I just bought Maureen O'hara's book on market microstructure for an understanding of the underlying theory.

No problem.

 

[cjbuckley4]

You really need accurate time stamps on the (possibly) multiple exchanges feeds plus your TCP sessions with them.

I have an ancillary question that follows from this statement (I'm shamelessly abusing your generosity at this point).

I have been using Nanex data for a while now, but there have been some minor frustrations such as the lack of tag 346 (Number of Orders for Incremental Refresh). I want to bring my data as close to what you'll see coming over a TCP session as possible for obvious reasons. I spoke to CME about getting some MDP 3.0 market depth files and am thinking of pulling the trigger. With the FIX/FAST format, you obviously had to timestamp server side to end up with suitably granular timestamps for this sort of stuff, but with the new format there's nanoseconds since EPOCH with guaranteed microsecond precision for Incremental Refreshes, so my question is what differences will this data have vs. a recorded TCP session like what you look at if any? From everything I've read, it seems the same. Maybe the devil is in the details and a big issue is you being able to compare the CME timestamp to your internal timestamp to assess your whole network latency for each update, whereas mine is just a guess. Since I'm so far from bleeding edge, I'm prepared to ignore this. Aside from that though, is there something I'm missing or will we be looking at basically the same thing?

 

[garachen]

I have an ancillary question that follows from this statement (I'm shamelessly abusing your generosity at this point).

I have been using Nanex data for a while now, but there have been some minor frustrations such as the lack of tag 346 (Number of Orders for Incremental Refresh). I want to bring my data as close to what you'll see coming over a TCP session as possible for obvious reasons. I spoke to CME about getting some MDP 3.0 market depth files and am thinking of pulling the trigger. With the FIX/FAST format, you obviously had to timestamp server side to end up with suitably granular timestamps for this sort of stuff, but with the new format there's nanoseconds since EPOCH with guaranteed microsecond precision for Incremental Refreshes, so my question is what differences will this data have vs. a recorded TCP session like what you look at if any? From everything I've read, it seems the same. Maybe the devil is in the details and a big issue is you being able to compare the CME timestamp to your internal timestamp to assess your whole network latency for each update, whereas mine is just a guess. Since I'm so far from bleeding edge, I'm prepared to ignore this. Aside from that though, is there something I'm missing or will we be looking at basically the same thing?

Market data is a multicast feed. TCP is how you send your orders and get your fills.
It's nice of the exchange to provide better time stamps now but they aren't useful. You need to be able to know what time the multicast data actually hits your server and correlate that with when fills are hitting your TCP session to be able to start to do any real anaylsis. This only matters when big trades come through and you are trying to lay off your risk quickly. Unfortunately, that's precisely when the exchange marked timestamps will be useless.

 

[cjbuckley4]

Market data is a multicast feed. TCP is how you send your orders and get your fills.
It's nice of the exchange to provide better time stamps now but they aren't useful. You need to be able to know what time the multicast data actually hits your server and correlate that with when fills are hitting your TCP session to be able to start to do any real anaylsis. This only matters when big trades come through and you are trying to lay off your risk quickly. Unfortunately, that's precisely when the exchange marked timestamps will be useless.

I can see how not having both pieces of information would be very limiting in this particular context as I have nothing more than a guess as to how long it would take these updates to actually reach me, which is far from good enough for the extreme scenario. This is really just for my edification at this point, because it's becoming increasingly clear that being able to lay off a lot of risk quickly is one of the biggest weaknesses for someone attempting to make markets who isn't fast. Accepting it is easy, getting rid of it is hard. That's probably why you told me "It might be possible to market make without dealing with the extreme cases. Some products more than others. But I'm not sure." in retrospect. The lower you try to drill down and the more comprehensive you try to make your backtest, the more complicated and expensive things become...I doubt this is news to you though.

I think where I really would expect to see the higher res timestamps add value is for analyzing updates that I'm not involved in. Seeing how the market reacts to different incoming updates under different scenarios and how long they take...things like that. I'm hoping this will serve a pedagogical purpose in preparing me for the industry. Investing the next week in studying for finals might be the better career move though.

There's no reason for me to explicitly want to chase these bleeding edge opportunities. It's probably better for me to find places where speed is less of an issue, that's why I kinda shelved straight up MM as I mentioned earlier. I think I have a decent idea as to how to mitigate some of the speed/first-ish in the Q or GTFO/general adverse selection concerns. I'm sure you and anyone remotely interested in microstructure has already thought of it and I've rambled enough. I'll shoot you an email if it starts to come together. Ideas are really easy in this game, execution is painstakingly difficult (at least for me). Thanks as always for your help. Enjoy your weekend!