reqExecutions (API)

[d08]

For anyone using reqExecutions and trading multiple timezones, IB states that only executions after midnight will be available, the daily reset wipes executions.
This is fine if you're using US servers to trade the US market only but things get complicated as the US server reset happens during Asian market hours.

Would I be able to retrieve Asian morning session executions at say 14:00 JST (1AM EST) or would they already be reset?
If anyone with experience could chime in, it would be appreciated.

 

[GRULSTMRNN]

Yes, crappy API, this has been an issue forever. I am not sure on which api version and tws/gateway version you are on. But a dirty workaround is to use tws as api hook not the gateway, then choose "show trades for last 7 days" in the trade log. Not a solution, I know but it might still work with your version.

For a more solid solution I need to know more details what exactly you are trying to solve or prevent. Your live execution details should always be persisted so when your machine crashes or disconnects you still know your fills. Of course you need to reconcile your pre-issue pnl and positions with the position details you get from IB when you reconnect.

For anyone using reqExecutions and trading multiple timezones, IB states that only executions after midnight will be available, the daily reset wipes executions.
This is fine if you're using US servers to trade the US market only but things get complicated as the US server reset happens during Asian market hours.

Would I be able to retrieve Asian morning session executions at say 14:00 JST (1AM EST) or would they already be reset?
If anyone with experience could chime in, it would be appreciated.

 

[d08]

Yes, crappy API, this has been an issue forever. I am not sure on which api version and tws/gateway version you are on. But a dirty workaround is to use tws as api hook not the gateway, then choose "show trades for last 7 days" in the trade log. Not a solution, I know but it might still work with your version.

For a more solid solution I need to know more details what exactly you are trying to solve or prevent. Your live execution details should always be persisted so when your machine crashes or disconnects you still know your fills. Of course you need to reconcile your pre-issue pnl and positions with the position details you get from IB when you reconnect.

I'm looking for my morning (say 9 to 11 am Japan Standard Time) executions in the afternoon, 3 hours later. This is not an issue when trading using the Asian servers but using the US servers, the reset period is at about 11-12 noon Japanese time - right at the Japanese lunch hour.

My overall goal is to find a robust way to separate positions based on strategy.
Looking for a different approach, I might just see if using reqPositionsMulti and modelCode might do what I need.

 

[GRULSTMRNN]

Still confused. If systematic then as said you should constantly persist incoming fills and positions to disk. You should not have a single missed fill/execution. Unfortunately the IB api does not have a parameter for the reqExecutions method to request executions from a certain timestamp onwards.

If discretionary then just check your trade log and choose multiple days. But I assume you asked for the API approach.

I'm looking for my morning (say 9 to 11 am Japan Standard Time) executions in the afternoon, 3 hours later. This is not an issue when trading using the Asian servers but using the US servers, the reset period is at about 11-12 noon Japanese time - right at the Japanese lunch hour.

My overall goal is to find a robust way to separate positions based on strategy.
Looking for a different approach, I might just see if using reqPositionsMulti and modelCode might do what I need.

 

[nooby_mcnoob]

You should not have a single missed fill/execution.

This happens all the time for a variety of reasons (mostly to do with losing connectivity for random reasons). But why check fills and executions, why not just check positions and compare to your desired positions?

 

[d08]

Still confused. If systematic then as said you should constantly persist incoming fills and positions to disk. You should not have a single missed fill/execution. Unfortunately the IB api does not have a parameter for the reqExecutions method to request executions from a certain timestamp onwards.

If discretionary then just check your trade log and choose multiple days. But I assume you asked for the API approach.

Completely automated.

Sure, constant logging is one way but I've usually built things with connection problems in mind. So any gap in data wouldn't be an issue as the IB servers will always sync trades even if I lose connectivity.

reqPositions is global for all clientIds. So separating anything with it is useless. reqExecutions is clientId based, that's why I prefer it. Previously I had no instrument overlap so this was not a problem.

 

[d08]

This happens all the time for a variety of reasons (mostly to do with losing connectivity for random reasons). But why check fills and executions, why not just check positions and compare to your desired positions?

As noted, it works fine for a single strategy/instrument but not with overlap. So I might be long TSLA for days but intraday I'd take a short position in it for a couple of hours, reqPositions cannot distinguish between the trades. Using different clientIds will help reqExecutions make a distinction.

 

[GRULSTMRNN]

Well, I also wished for an api that was perfect. It's not. Hopefully IB will continue delivering on improvements. I have a sense that there is someone behind the wheels these days who pushes the dev teams more than in past years.

I strongly recommend you to accept imperfection and work around the issues. What I do is I persist every execution to disk, those fills come in as the fills happen. My architecture then immediately reconciles those with the OMS on my side. Once verified the fills actually don't matter anymore. All that matters is the position. And after your outage or server reset or whatever it may be you simply request a position update and make sure positions match. That is all you need to be absolutely in sync with IB servers. If there is a discrepancy I doubt any architecture is currently able to figure out the issue on its own and usually human eyes are anyway required to check what is going on. When this happens my engine immediately stops submitting any new orders and alerts me to this issue.

Hope this helps

Completely automated.

Sure, constant logging is one way but I've usually built things with connection problems in mind. So any gap in data wouldn't be an issue as the IB servers will always sync trades even if I lose connectivity.

reqPositions is global for all clientIds. So separating anything with it is useless. reqExecutions is clientId based, that's why I prefer it. Previously I had no instrument overlap so this was not a problem.

 

[nooby_mcnoob]

As noted, it works fine for a single strategy/instrument but not with overlap. So I might be long TSLA for days but intraday I'd take a short position in it for a couple of hours, reqPositions cannot distinguish between the trades. Using different clientIds will help reqExecutions make a distinction.

Could you use a combination of:

- reqOpenOrders
- reqCompletedOrders
- orderRef field of Order

I would rather you find a somewhat idempotent way to do this.

 

[d08]

Well, I also wished for an api that was perfect. It's not. Hopefully IB will continue delivering on improvements. I have a sense that there is someone behind the wheels these days who pushes the dev teams more than in past years.

I strongly recommend you to accept imperfection and work around the issues. What I do is I persist every execution to disk, those fills come in as the fills happen. My architecture then immediately reconciles those with the OMS on my side. Once verified the fills actually don't matter anymore. All that matters is the position. And after your outage or server reset or whatever it may be you simply request a position update and make sure positions match. That is all you need to be absolutely in sync with IB servers. If there is a discrepancy I doubt any architecture is currently able to figure out the issue on its own and usually human eyes are anyway required to check what is going on. When this happens my engine immediately stops submitting any new orders and alerts me to this issue.

Hope this helps

Thanks for the ideas.

I'm not complaining about the imperfection at all. I've accepted it. I just thought of asking because maybe someone has an approach to it that I didn't think of. There's more than a few ways to do this.

With the solution you're using, wouldn't an overlap be an issue. So one system going long, other short in the same instrument. Positions wouldn't reflect this and would assume there is some problem. When partial fills and disconnections are added, I can't see this being reliable.
With executions you'll have three critical pieces of info - instrument, position, clientId. Assuming you use one clientId per system, this gives a better picture than reqPositions.

But currently in production, I use the same method as you. For the future I need something with a bit more separation, that's all.