The observable fact is that IB is not interested in serving clients who want to direct route. There is a class of strategies that require it, and because of these fees they simply cannot be traded at IB. So those traders go elsewhere.
The reasoning for IB to leave part of a market that could be profitable for them, we can only speculate. Which is all I was doing with my Timber Hill theory.
Another theory - and this seems to match pretty well what you are saying - is that IB just doesn't have their technology/infrastructure up to the level that would be required. The per-order processing costs are too high on their antiquated systems to be profitable with the kinds of order-to-execution ratios those clients would generate.
You are free to speculate but your speculation above is way off. With equity capital of $8 billion we could easily compete in the HFT space. I can't fault the decision to focus solely on the brokerage given the success we've had. You most certainly have no idea how much we invest in infrastructure and certainly don't have a full understanding of why we stopped market making. I will not provide details but I do know we have the resources and do keep our infrastructure state of the art. I think a good example which shows the quality of our technology and infrastructure is this past March when trades doubled to nearly 2 million per day in March. Also don't forget to include the number of orders, cancellation and modifications while significant portions of the workforce were forced to WFH before stating we have antiquated systems. IMO these numbers alone tell you a thing or two about the quality of our infrastructure and team of engineers.
Don't forget there are not only direct exchange and bandwidth costs but there are huge regulatory, surveillance and other items to consider. We certainly can spend additional dollars to support any type of business we put our minds to but we do aim to be profitable and thus do not support every type of activity a trader desires. If you can make a business case, please do so. I suspect API direct routing has been hashed out a number of times over the years and the decision to not charge direct routing via TWS but add a surcharge to those doing it via the API has been well considered. We have great confidence in our SMART router and for most (and I understand not all), it is a win/win decision.
Just to give you some idea on costs - I'll leave with some details in HK which I'm very much on top of as an example. Say you have a strategy that requires submission of 100 orders every 10 seconds and it is key for you to have those orders reach the exchange w/o delay (I.e. you blast them in all at once via an automated system).
Exchange cost: each API/Throttle for stocks on SEHK delivers a whopping 2 transactions a second. So if we you are to place your orders w/o delay, a firm would need 50 throttles to service you (not including other clients).
Cost per throttle - 50,000 HKD one time up front, plus 960 HKD per month or HKD2.5mm (~$322K USD) one time fee plus 48K HKD ($6.2K USD) per month.
Do the math, what volumes do we need to accommodate your business? What if other clients join in and ramp up the orders to 200 per second or update every tick on the order book for a basket of stocks. One could easily purchase another 200 throttles to accommodate the orders but you'd be a fool to do it if the bulk of those orders weren't marketable and didn't cover the costs.
That's just a small piece of the puzzle but as usual, there are many pieces of the puzzle to consider.