IB Gateway US FUTURE market data farm remains OFF (yellow light) after reconnection

I see a total failure at least once a day of the "usfuture" data farm (historical data farm is no problem).

DataFarmOFF.gif


There is no way from the client application to make the gateway restart the data flow, apart manually restarting the Gateway (secure device login).

The IB gateway has become of no use to me.

Also, a very very bad idea adding an automatic daily re-login in the "latest" version (you guess what version it is).

If you wish to insist on this nonsense, please, at least make it optional and not mandatory.
 
Last edited:
I have not seen you launch a post/thread on the tws api user group : https://groups.io/g/twsapi/topics





fullautotrading said:
I see a total failure at least once a day of the "usfuture" data farm (historical data farm is no problem).

View attachment 202163

There is no way from the client application to make the gateway restart the data flow, apart manually restarting the Gateway (secure device login).

The IB gateway has become of no use to me.

Also, a very very bad idea adding an automatic daily re-login in the "latest" version (you guess what version it is).

If you wish to insist on this nonsense, please, at least make it optional and not mandatory.
More...
 
GRULSTMRNN said:
I have not seen you launch a post/thread on the tws api user group : https://groups.io/g/twsapi/topics
More...

There is already a thread there. I do not see the point to repeat over there, since it has not been solved in more than1 year.
Diversification may help. Also, I like this specific forum ("IB Technical Issues") better.

Anyway, in case they may be of some help, here are the messages that the Gateway continue spitting out, and remains unable to resume the data:

Code: 
2019-05-07 12:57:21.125 [CG] INFO  [JTS-EServerSocket-3060] - Start API message, ClientID=1
2019-05-07 12:57:21.436 [CG] INFO  [JTS-QuickMktDataWrapper-81] - Farm usfuture/DISCONNECTED: Adding message to queue 923
2019-05-07 12:57:21.581 [CG] INFO  [NIA-Output-Queue-1] - |jtsNIA|nia.ar@7b72dfc0 | a()#853 | subscribe VCF | 97589608 | REGULAR | 20200514 |
2019-05-07 12:57:21.581 [CG] INFO  [JTS-QuickMktDataWrapper-81] - Farm usfuture/DISCONNECTED: Adding message to queue 924
2019-05-07 12:57:21.581 [CG] INFO  [JTS-Async-27] - |jtsNIA|s:97589608/REGULAR | subscribe()#120 | TWS subscribes |
2019-05-07 12:57:21.581 [CG] INFO  [NIA-Output-Queue-1] - |jtsNIA|nia.ar@7b72dfc0 | a()#871 | subscribe VSF | 97589608 | REGULAR |
2019-05-07 12:57:21.581 [CG] INFO  [JTS-Async-27] - subscribing for 589 with params expiry=20200514
2019-05-07 12:57:21.581 [CG] INFO  [NIA-Output-Queue-1] - |jtsNIA|nia.y@255e0093 | a()#93 | Switch to volsf | 97589608 | REGULAR | 1 |
2019-05-07 12:57:21.581 [CG] INFO  [NIA-Output-Queue-1] - |jtsNIA|t:97589608/REGULAR | subscribe()#120 | TWS subscribes |
2019-05-07 12:57:21.587 [CG] INFO  [JTS-QuickMktDataWrapper-81] - Farm usfuture/DISCONNECTED: Adding message to queue 925
2019-05-07 12:57:22.961 [CG] INFO  [JTS-Cleaner-31] - market data wrapper, cleaning maps
2019-05-07 12:57:24.001 [CG] INFO  [NIA-Output-Queue-1] - |UNKN|MDLNAV1 | onDividendChanged()#1119 | Dividend Arrive | 17340715DividendSchedule[] | AUTO |
2019-05-07 12:57:26.081 [CG] INFO  [JTS-EServerSocket-3060] - [1:142:142:1:0:0:0:ERR] Socket connection for client{1} has closed.
2019-05-07 12:57:26.081 [CG] INFO  [JTS-EServerSocket-3060] - Connection terminated
2019-05-07 12:57:26.081 [CG] INFO  [JTS-EServerSocketNotifier-3061] - Terminating async queue thread
2019-05-07 12:57:26.081 [CG] INFO  [JTS-EServerSocket-3060] - Cleaning up [serverId: 1]...
2019-05-07 12:57:26.082 [CG] INFO  [JTS-EServerSocket-3060] - Cleaning up [serverId: 1]...
2019-05-07 12:57:26.082 [CG] INFO  [JTS-EServerSocket-3060] - Cleaning up [serverId: 1]...
2019-05-07 12:57:26.082 [CG] INFO  [JTS-EServerSocket-3060] - ArEServer: Removing 668702198 with id 1
2019-05-07 12:57:26.082 [CG] INFO  [JTS-EServerSocket-3060] - eServersChanged: 0
2019-05-07 12:57:26.083 [CG] INFO  [NIA-Output-Queue-1] - |jtsNIA|nia.ar@7b72dfc0 | a()#877 | unsubscribe VSF | 97589608 | REGULAR |
2019-05-07 12:57:26.083 [CG] INFO  [JTS-EServerSocket-3060] - There are not API orders being processed.
2019-05-07 12:57:26.083 [CG] INFO  [JTS-EServerSocket-3060] - SecDefRequestTimeOut: There are not timed out SecDef requests.
2019-05-07 12:57:26.083 [CG] INFO  [JTS-EWriter289-3062] - [1:142:142:1:0:0:0:SYS] Writer thread terminated for socket client{1}.
2019-05-07 12:57:26.083 [CG] INFO  [NIA-Output-Queue-1] - |jtsNIA|nia.y@255e0093 | b()#129 | Switch to volcrv | 97589608 | REGULAR | 1 |
2019-05-07 12:57:26.083 [CG] INFO  [JTS-EServerSocket-3060] - [1:142:142:1:0:0:0:SYS] Ending conversation with client{1} at 127.0.0.1
2019-05-07 12:57:26.083 [CG] INFO  [JTS-QuickMktDataWrapper-81] - Farm usfuture/DISCONNECTED: Adding message to queue 926
2019-05-07 12:57:26.083 [CG] INFO  [NIA-Output-Queue-1] - |jtsNIA|s:97589608/REGULAR | subscribe()#120 | TWS subscribes |
2019-05-07 12:57:26.083 [CG] INFO  [NIA-Output-Queue-1] - subscribing for 589 with params expiry=20200514
2019-05-07 12:57:26.084 [CG] INFO  [NIA-Output-Queue-1] - |jtsNIA|nia.ar@7b72dfc0 | a()#863 | unsubscribe VCF | 97589608 | REGULAR | 20200514 |
2019-05-07 12:57:26.090 [CG] INFO  [JTS-QuickMktDataWrapper-81] - Farm usfuture/DISCONNECTED: Adding message to queue 927
2019-05-07 12:57:26.091 [CG] INFO  [JTS-QuickMktDataWrapper-81] - remove volsffn gentick; remaining=2 desubscribedFromTop=true desubscribedFromFrozenTop=false
2019-05-07 12:57:26.115 [CG] INFO  [JTS-SocketListener-46] - State: HEADER, IsAPI: UNKNOWN
2019-05-07 12:57:26.115 [CG] INFO  [JTS-SocketListener-46] - State: STOP, IsAPI: YES
2019-05-07 12:57:26.115 [CG] INFO  [JTS-SocketListener-46] - ArEServer: Adding 1825496766 with id 2147483647
2019-05-07 12:57:26.115 [CG] INFO  [JTS-SocketListener-46] - eServersChanged: 1
2019-05-07 12:57:26.116 [CG] INFO  [JTS-EServerSocket-3064] - [2147483647:142:142:1:0:0:0:SYS] Starting new conversation with client on 127.0.0.1
2019-05-07 12:57:26.116 [CG] INFO  [JTS-EServerSocketNotifier-3065] - Starting async queue thread
2019-05-07 12:57:26.116 [CG] INFO  [JTS-EServerSocket-3064] - [2147483647:142:142:1:0:0:0:SYS] Server version is 142
2019-05-07 12:57:26.116 [CG] INFO  [JTS-EServerSocket-3064] - [2147483647:142:142:1:0:0:0:SYS] Client version is 142
2019-05-07 12:57:26.116 [CG] INFO  [JTS-EServerSocket-3064] - [2147483647:142:142:1:0:0:0:SYS] is 3rdParty true
2019-05-07 12:57:26.117 [CG] INFO  [JTS-EServerSocket-3064] - Start API message, ClientID=1
2019-05-07 12:57:26.433 [CG] INFO  [JTS-QuickMktDataWrapper-81] - Farm usfuture/DISCONNECTED: Adding message to queue 928
2019-05-07 12:57:26.579 [CG] INFO  [NIA-Output-Queue-1] - |jtsNIA|nia.ar@7b72dfc0 | a()#853 | subscribe VCF | 97589608 | REGULAR | 20200514 |
2019-05-07 12:57:26.579 [CG] INFO  [JTS-QuickMktDataWrapper-81] - Farm usfuture/DISCONNECTED: Adding message to queue 929
2019-05-07 12:57:26.579 [CG] INFO  [JTS-Async-27] - |jtsNIA|s:97589608/REGULAR | subscribe()#120 | TWS subscribes |
2019-05-07 12:57:26.579 [CG] INFO  [NIA-Output-Queue-1] - |jtsNIA|nia.ar@7b72dfc0 | a()#871 | subscribe VSF | 97589608 | REGULAR |
2019-05-07 12:57:26.579 [CG] INFO  [JTS-Async-27] - subscribing for 589 with params expiry=20200514
2019-05-07 12:57:26.579 [CG] INFO  [NIA-Output-Queue-1] - |jtsNIA|nia.y@255e0093 | a()#93 | Switch to volsf | 97589608 | REGULAR | 1 |
2019-05-07 12:57:26.579 [CG] INFO  [NIA-Output-Queue-1] - |jtsNIA|t:97589608/REGULAR | subscribe()#120 | TWS subscribes |
2019-05-07 12:57:26.585 [CG] INFO  [JTS-QuickMktDataWrapper-81] - Farm usfuture/DISCONNECTED: Adding message to queue 930
2019-05-07 12:57:27.081 [CG] INFO  [NIA-Output-Queue-1] - |UNKN|MDLNAV1 | onDividendChanged()#1119 | Dividend Arrive | 17340715DividendSchedule[] | AUTO |
 
Last edited:
HobbyTrading said:
Have you also sent this log file and screen shot to IB? What do they say about this?
More...

I just did.

We will see if they fix it, and also fix the daily re-login/autorestart nonsense.
I will keep this thread posted if a see solutions.

As another member (cruisecontrol) rightly pointed out, all this to a programmer's eyes tells only one thing...

If not solved in due time, personally, I think I will switch technology. (I do not have time to work for the broker too. Writing algos to fight market making algos is challenging enough.)
 
Any other broker suggestions for automated trading with full access to world markets? Am curious because I can't think of any.

fullautotrading said:
I just did.

We will see if they fix it, and also fix the daily re-login/autorestart nonsense.
I will keep this thread posted if a see solutions.

As another member (cruisecontrol) rightly pointed out, all this to a programmer's eyes tells only one thing...

If not solved in due time, personally, I think I will switch technology. (I do not have time to work for the broker too. Writing algos to fight market making algos is challenging enough.)
More...
 
GRULSTMRNN said:
Any other broker suggestions for automated trading with full access to world markets? Am curious because I can't think of any.
More...
Ah, yes, you are right: "in the land of the blind, the one-eyed man is king".
Not my philosophy though.

What automated trading? I have been doing this for about 20 years, and I am looking now at a gateway which needs manual re-login every day (to log with a device). And, not satisfied with that, an automatic daily relogin/autorestart has also been added to the "latest" version?

The system worked way better several years ago than now. (Although to have the idiotic blocking re-login prompt from the gateway I almost had to fight over the phone with an IB director (see previous thread), and now this.)

What "automated" trading?
 
Well, all cynicism aside, you answered my question. Unless you trade a very narrowly defined set of assets it's hard to find another broker out there that offers what IB does. Hence, it pays to keep bringing up the issues to IB support and also consider alternative ways such as FIX or the like.

fullautotrading said:
Ah, yes, you are right: "in the land of the blind, the one-eyed man is king".
Not my philosophy though.

What automated trading? I have been doing this for about 20 years, and I am looking now at a gateway which needs manual re-login every day (to log with a device). And, not satisfied with that, an automatic daily relogin/autorestart has also been added to the "latest" version?

The system worked way better several years ago than now. (Although to have the idiotic blocking re-login prompt from the gateway I almost had to fight over the phone with an IB director (see previous thread), and now this.)

What "automated" trading?
More...
 
Did you see the post in the tws/in group?

"Can confirm still an issue on my side.

I've just installed TWS 976, and you can set up an automatic daily restart in Global Configuration - then all you need is your app to disconnect and reconnect around the same time. Slightly easier than IBC."

fullautotrading said:
Ah, yes, you are right: "in the land of the blind, the one-eyed man is king".
Not my philosophy though.

What automated trading? I have been doing this for about 20 years, and I am looking now at a gateway which needs manual re-login every day (to log with a device). And, not satisfied with that, an automatic daily relogin/autorestart has also been added to the "latest" version?

The system worked way better several years ago than now. (Although to have the idiotic blocking re-login prompt from the gateway I almost had to fight over the phone with an IB director (see previous thread), and now this.)

What "automated" trading?
More...
 
GRULSTMRNN said:
Slightly easier than IBC.
More...

From the point of view of an automated trader, most of the time, it makes no sense that the Gateway takes the initiative to shut down at a given time.
There should be no point of doing that on a daily basis if it were programmed properly. It only makes people laugh about their coding skills.

Anyway. Now, I will tell you the proper way to implement a relogin or autorestart feature.

Talking of API automation, in general, the king rule is that the gateway is a tool for the client to communicate with IB and absolutely all communications and actions must happen through that channel.

So if you add a relogin autostart feature, there has to be a command via API to trigger that, so that the client application can decide when and if to do that, in case it is necessary.

In any case, the relogin or autostart process must be seamless, require no security device to relogin to the same account and absolutely raise no prompts.
The blocking window warning about paper trading account should also be removed, or else nobody will be able to even start trying automation.

Under no circumstance, a blocking window must ever appear. Use API to tell things!

And the type of account should be communicated via API (we should not be forced to make arbitrary inference based on letters of account code). It is ridiculous that along so much unnecessary pieces of information transmitted, that one is not.

In brief.

- Absolutely no prompts at any time (repeat no prompts! Do not open warning windows! Use the API channel to tell things: there is a client listening, not humans!).

- Everything must happen via API, including possible re-login and/or auto restart commands. There must be proper API commands, to do so.

- Re-login to same account (triggered by client application) must be possible without requiring any (security) device and without any warning or prompt, outside of API communications, no matter what it is the type of account (paper/real).
 
Last edited:
You must log in or register to reply here.
Back
Top