Data feed overload in live testing!?

R

rols

I am developing an automated system using TWS and VB. In simulated trading the system works fine. As soon as I go live however and get portfolio feedback from TWS using the updateportfolio and reqAcctDetails() method I get jumbled data and unpredicatable results.

It is almost as if the system can't keep up. This especially happens during the frenzy of the open when prices are moving very fast.


Help!!
 
I doubt that it due to TWS. I can easily monitor all NAS 100 stocks + ES, NQ, YM etc without the machine raising a sweat. (Java application).

If by simulated trading, you mean IB's demo system, be aware that it has message rates that would put a snail to shame.

Have you checked the CPU utilization, both of your app and the system as a whole ? Do you have any other processes consuming excessive CPU ? Are you doing excessive logging in your code , writing every tick to disk file or such like ?

If your app is using excessive CPU, you'll need to profile it to find the offending code fragments.

Of course it may also be just bugs and not a performance issue at all.
 
Thanks for your replies, I have sometimes in the past hit the 50 message/sec limit but not in this case I am however logging to disk every second and I reckon that this may well be contributing.

The trouble is it seems to happen most often when trades are being completed very rapidly and I need to make a record of this.

Currently I am storing each completed trade as a text file so perhaps if I just keep a database connection open and record the data to this it will be able to keep up?
 
Quote from rols:

Thanks for your replies, I have sometimes in the past hit the 50 message/sec limit but not in this case I am however logging to disk every second and I reckon that this may well be contributing.

The trouble is it seems to happen most often when trades are being completed very rapidly and I need to make a record of this.

Currently I am storing each completed trade as a text file so perhaps if I just keep a database connection open and record the data to this it will be able to keep up?
More...

Record to the database and record the whole text file after hours or something like that. Textfiles can be a problem in themselves, they can slow as they get bigger because software will read the whole file before it can append. Tradestation is like that.
 
I have been observing more closely and it seems that after an order has been put into TWS if there is a delay in the order being filled there is also a delay in activity coming back to VB. I don't know if it momentarily freezes the whole system or not. Perhaps a few doevents will cure it. Has anybody else found similar things happening?
 
Quote from rols:

I have been observing more closely and it seems that after an order has been put into TWS if there is a delay in the order being filled there is also a delay in activity coming back to VB. I don't know if it momentarily freezes the whole system or not. Perhaps a few doevents will cure it. Has anybody else found similar things happening?
More...

No, I've always found that order status events are sent promptly by TWS eg 'submitted' when order reaches the exchange, 'pending' if order is a simulated order type such as trailing stop. Don't know about the VB stuff though. I never use it.
 
I've used VB for the past 10 months with no problems whatsoever. I'll be switching to a .net based version soon but either way I don't think VB is the issue....

SSB
 
Quote from rols:

I have been observing more closely and it seems that after an order has been put into TWS if there is a delay in the order being filled there is also a delay in activity coming back to VB. I don't know if it momentarily freezes the whole system or not. Perhaps a few doevents will cure it. Has anybody else found similar things happening?
More...

Some of us have been experiencing latency with Market orders, maybe you could try a marketable limit as a workaround.
 
You must log in or register to reply here.
Back
Top