Creating logic in Python for auto-generating a tuple of futures spreads to query

[tommcginnis]

This is from an IB QuoteMonitor page, off-loaded to a .csv, and imported into an OpenOffice spreadsheet. Takes me about 3 minutes (maybe) from data-to-hard-copy; I'm shopping for ways (INCLUDING PYTHON) to get it done in real-time.

My own demands are for two different databases, being two different configurations of SPX expirys running about 700 option fields (rows) each, 1400 total. On a 75Mbps connection, it takes about 2 seconds to print the file to disk, and another 8 seconds or more to populate the (sizable) OpenOffice spreadsheet. So, if I'm not careful, my "Real-time data scraper" project will sludge up the works some....

(So, my own current battle is shopping for that configuration that will produce the most stable, longest-lived, platform-agnostic set-up. So, I wrestle between IB's TWS client and the data Gateway, between Java and Python, and between OpenOffice Calc as the final UI, or just re-writing the whole business in Python and chopping OpenOffice out the the loop entirely.)

Python will speed development and deployment over Java, but will be 100x slower. But if I can carve OpenOffice Calc out of the equation by just going to Python, wouldn't that speed things along by more than Python>Java would slow things down? Hmmmmm.

 

[O(1)]

This is from an IB QuoteMonitor page, off-loaded to a .csv, and imported into an OpenOffice spreadsheet. Takes me about 3 minutes (maybe) from data-to-hard-copy; I'm shopping for ways (INCLUDING PYTHON) to get it done in real-time.

My own demands are for two different databases, being two different configurations of SPX expirys running about 700 option fields (rows) each, 1400 total. On a 75Mbps connection, it takes about 2 seconds to print the file to disk, and another 8 seconds or more to populate the (sizable) OpenOffice spreadsheet. So, if I'm not careful, my "Real-time data scraper" project will sludge up the works some....

(So, my own current battle is shopping for that configuration that will produce the most stable, longest-lived, platform-agnostic set-up. So, I wrestle between IB's TWS client and the data Gateway, between Java and Python, and between OpenOffice Calc as the final UI, or just re-writing the whole business in Python and chopping OpenOffice out the the loop entirely.)

Python will speed development and deployment over Java, but will be 100x slower. But if I can carve OpenOffice Calc out of the equation by just going to Python, wouldn't that speed things along by more than Python>Java would slow things down? Hmmmmm.

sounds interesting..
imho
if I had to choose to do something real-time like that, given what you've mentioned.. I might try to break up the task of updating the real-time data into threads. Then another to be trying to find a trade to make. I'd rather have the data in memory than at the hard drive. I'd choose java over python.

personally, i don't need to make continual data requests.. so i throw some sleep in there to slow things down