I have many years of runway plus multiple sources of passive income but I'm still going to give myself a limited amount of time to show results.
Developing "Spartan"
- Thread starter nooby_mcnoob
- Start date
I have many years of runway plus multiple sources of passive income but I'm still going to give myself a limited amount of time to show results.
OK this thread isn't for you bub. DM me if you want information unrelated to this thread.
Thanks for posting all these updates, it's all very interesting (and inspiring!).
I am working on a long term personal project to build such a system and was wondering how you have architected the crossover between back-testing and live running (to ensure as much overlap as possible, code-wise).
Also, does you system you check the market every x milliseconds or do you check on each tick event/other timeframe? Or do you have some other trigger?
These are the two areas where I struggle (and see the most complexity) with: the event driven nature of a system and how to build a single system to handle both back-testing and live running.
Keep the updates coming please!
I am working on a long term personal project to build such a system and was wondering how you have architected the crossover between back-testing and live running (to ensure as much overlap as possible, code-wise).
Also, does you system you check the market every x milliseconds or do you check on each tick event/other timeframe? Or do you have some other trigger?
These are the two areas where I struggle (and see the most complexity) with: the event driven nature of a system and how to build a single system to handle both back-testing and live running.
Keep the updates coming please!
My system is event driven which (theoretically) works fine in both back and forward testing. The problem you're having is trying to design up front for every possibility. Solve one problem at a time.
IB tick volume (trades where last price changed - latter two blocks) vs histdata (midpoint price changes - first three blocks). Glad I'm caught up for backtesting but will be using IB ticks from now on. The volume is normalized in my algorithm so it isn't that much of a problem though obviously takes some time to adjust.
Volume is now in the database going back to 2005. The database takes up ~ 205 MB so hardly anything special.
Volume is now in the database going back to 2005. The database takes up ~ 205 MB so hardly anything special.
Volume
Phew... What a week. Volume has been insane to deal with. First, reqHistoricalData for tick volume is exceptionally slow due to IB throttling once you need more than 1 hour and 1 currency. In the end, I decided to create a quote table in my database with the intention of filling in ticks and (perhaps) collapsing the data daily, or backing it up elsewhere.
The next problem with reqTickByTickData is that I am apparently limited in the number of tick-by-tick requests I can make. This means 2 of the currencies I need would not have tick data stored. This is pretty frustrating/annoying.
I wonder if I get the ability to subscribe to more tick data if I use IB Gateway.
Or maybe I should just switch to IQFeed...
Phew... What a week. Volume has been insane to deal with. First, reqHistoricalData for tick volume is exceptionally slow due to IB throttling once you need more than 1 hour and 1 currency. In the end, I decided to create a quote table in my database with the intention of filling in ticks and (perhaps) collapsing the data daily, or backing it up elsewhere.
The next problem with reqTickByTickData is that I am apparently limited in the number of tick-by-tick requests I can make. This means 2 of the currencies I need would not have tick data stored. This is pretty frustrating/annoying.
I wonder if I get the ability to subscribe to more tick data if I use IB Gateway.
Or maybe I should just switch to IQFeed...
Oh shit I just realized it's Thursday. Sweet. I can let the ticks record over the next day and see how it goes.
The limitation is not related to TWS or Gateway. It is limited to your account size. It is possible to buy additional blocks (caveat: I don't know the costs involved). See here at section Limitations: https://interactivebrokers.github.io/tws-api/market_depth.html#limitations
https://interactivebrokers.github.io/tws-api/market_data.html#market_lines
https://interactivebrokers.github.io/tws-api/tick_data.html
Well I'm an idiot. I guess it's because I had TWS set to my lowest non-paper account? After changing the account to the main one and resetting the data (CTRL+ALT+F in TWS), I had no problems. Thank you sir.