IB API server connection lost

[blink18]

Today I noticed between 15:24 CET and 16:50 CET that connection between IB Gateway and IB API server was lost. Usually this happens on Saturdays or early in the morning (IB system reset times). My logs show that this can also happen on market hours (very rarely). IB API client connection was working fine, Account number and Net liquidation value was retrieved, however portfolio positions were not retrieved!

What if an automated system wants to send an order and this happens? Does an order go to pending state or something else? Does it even work, since you cannot retrieve portfolio positions?

@globalarbtrader do you have any tips/tricks on this?

 

[ValeryN]

Not having open positions info returned is a possibility. I've designed my automation to act on a latest version either local or broker's. That helped to mitigate that.

Unfortunately outages can come in different shapes and forms.

IB in particular has data outages few times a year, that's pretty much should be expected. The portfolio info outage happens once a year or so based on my observations.

One thing where IB has been very reliable at is order execution. So if you manage to construct a version of reality based on local and their version that should give a reasonable failt tolerance.

 

[HobbyTrading]

What if an automated system wants to send an order and this happens? Does an order go to pending state or something else? Does it even work, since you cannot retrieve portfolio positions?

I have the impression that IB uses multiple servers. For example: the server which provides positions is a different one than the server which provides market data. The server which processes trades is again a different server. The fact that you can't get your position overview does not necessarily block you from sending trades to the server.
That is why it is important that you process the order and trade confirmation messages in your software.

 

[globalarbtrader]

Today I noticed between 15:24 CET and 16:50 CET that connection between IB Gateway and IB API server was lost. Usually this happens on Saturdays or early in the morning (IB system reset times). My logs show that this can also happen on market hours (very rarely). IB API client connection was working fine, Account number and Net liquidation value was retrieved, however portfolio positions were not retrieved!

What if an automated system wants to send an order and this happens? Does an order go to pending state or something else? Does it even work, since you cannot retrieve portfolio positions?

@globalarbtrader do you have any tips/tricks on this?

The solution is to assume that you'll frequently lose IB connection, and then when it does you can live with it. My primary failsafe is a process that runs throughout the day and compares my position state with IB position state. If this fails (which could be a lost connection, or could instrument specific eg because I've had a position auto liquidated near expiry) then I won't try and submit orders for that particular instrument (which could of course be the entire book if it's a connection failure). The same process will clear locks once the mismatch is resolved (connection returns, or I manually enter the auto liquidation order into my database).

This won't affect orders that have already been submitted, so there is a bit of an edge case there, but there's not a lot you can do if you submit an order and then you lose the connection... you just have to wait until the IB server comes back and then see what state the order is in.

GAT

 

[blink18]

The solution is to assume that you'll frequently lose IB connection, and then when it does you can live with it. My primary failsafe is a process that runs throughout the day and compares my position state with IB position state. If this fails (which could be a lost connection, or could instrument specific eg because I've had a position auto liquidated near expiry) then I won't try and submit orders for that particular instrument (which could of course be the entire book if it's a connection failure). The same process will clear locks once the mismatch is resolved (connection returns, or I manually enter the auto liquidation order into my database).

This won't affect orders that have already been submitted, so there is a bit of an edge case there, but there's not a lot you can do if you submit an order and then you lose the connection... you just have to wait until the IB server comes back and then see what state the order is in.

GAT

What if the system reads state of position from algorithm and daily quote data (inposition/outofposition) and compares it every 30 minutes with positions on IB server. If it doesn't match - sell it or buy it. Do you think this is a good solution or do you see any flaws?

 

[globalarbtrader]

What if the system reads state of position from algorithm and daily quote data (inposition/outofposition) and compares it every 30 minutes with positions on IB server. If it doesn't match - sell it or buy it. Do you think this is a good solution or do you see any flaws?

No, don't do this.

Suppose your system wants to be long 2 contracts, and that is your actual position. There is a system glitch, and your position on IB goes to zero. So you buy 2 contracts. Then the glitch is cleared, and now you're long 4 contracts. So you sell 2 contracts. You get the idea...

What I do is compare the optimal state of the system with the positions I have stored locally, and then generate trades, unless my second comparision (local database of positions vs IB) has a break in which case I wait until the break is resolved (see earlier post) - because until it's resolved I don't know for sure what my positions are.

GAT