Nice. Would be keen to see your backtested results when you are done
Fully automated futures trading
- Thread starter globalarbtrader
- Start date
Hey folks, here's another update from me. First was https://elitetrader.com/et/threads/fully-automated-futures-trading.289589/page-209#post-5055797 and second was https://elitetrader.com/et/threads/fully-automated-futures-trading.289589/page-217#post-5127521.
TL;DR - in a nasty drawdown, same as a few other folks I've seen here (hang in there Kernfusion!)
Here's the money shot since inception (comparing to S&P 500 because that's what I got data for, in future, I'll probably benchmark against 60/40 as that's what I would put my money in if I wasn't doing this. I know it's not apples to apples, but it's what makes sense for me):
Here's the nasty bit:
Was it not for extreme volatility and luck at the beginning, I'd be down, and not up 7% (I am still down a few percent in cash terms because I didn't deposit full amount all at once when I opened my account, but over the course of 2 months).
Since the last time, I've focused a bit on refactoring of the system architecture and a bit on research. Few words on each.
The system is 80% (I got a few more weeks to get to 100%) event-driven, meaning that different components send events between each other to communicate. For example, instrument component is in charge of fetching new prices, and when it gets a new price, it emits NewPrice event, which forecasting component is registered to receive. Forecasting is executed and that emits ForecastUpdated, which position component receives and so on. This makes is very easy to test each component in isolation as well as end-to-end test everything together. I also implemented event logging so in one place I can see exactly what's happening. I didn't want to complicate things so all this event passing is done in-memory, withing a single process (system works with daily prices so I got all the time in the world). Some day I might move to a proper messaging platform like PubSub on Google Cloud or roll out my own (I had good results with Redis before).
Here's a snapshot of what that looks like:
Besided that, I spent a bunch of time making my own backtesting plumbing that also has reporting. I tried PyFolio, but it seems not to be maintained anymore and I didn't need half of what they have. 2 screenshots above are from that work, it's all in Jupyter. I've made it modular so I can just pass in trades and it spits out all the stats. That means that I get the same stats for realized performance (I just pass in my executed trades) as for backtest (I hook up ScheduleTrade event to execute a trade in the backtest plumbing). Here's one more screenshot, this time from a backtest.
I'm adding more stuff into this, but I've realized there's an infinite number of stats and graph one can add, so this seems like "less-is-more" type of a thing.
For research, I've settled on the notebook-per-topic approach, so I have notebooks like ForecastAnalysis (analyses average forecasts for rules), DiversificationMultipliers, RealizedPerformance and so on. I haven't yet solved the problem of versioning Jupyter notebooks in git, all the methods out there seem like hacks to me, but I think this might be one of those things where there's no ideal solution.
I'm planing to add 9 more markets (currently trading 12) few weeks after the elections, I don't have any solid data on this but I thought I didn't want to double my account and add markets and exposure just before US elections, maybe that's just being too conservative, but I'll sleep better.
I am a bit concerned with the current drawdown but I'm still not touching anything. In the backtest, the system did have one nasty drawdown of 55% around 2007 (mostly due to ESTX50, see below) so it's not like this is atypical.
I do have one question for folks here, do you have markets that are not profitable (or are even losers) in the long term backtest in your portfolio? ESTX50 is that for me, over the 20 year backtest, that just consistently loses money. I'm not pulling it out of portfolio yet because I don't want to cherrypick (I realize that what happened in the past is not the best indication of future) but 20 years does seem like a good amount of data, and it includes 2 bigger crisis periods as well. Do you pull markets like that out or keep them in?
TL;DR - in a nasty drawdown, same as a few other folks I've seen here (hang in there Kernfusion!
)Here's the money shot since inception (comparing to S&P 500 because that's what I got data for, in future, I'll probably benchmark against 60/40 as that's what I would put my money in if I wasn't doing this. I know it's not apples to apples, but it's what makes sense for me):
Here's the nasty bit:
Was it not for extreme volatility and luck at the beginning, I'd be down, and not up 7% (I am still down a few percent in cash terms because I didn't deposit full amount all at once when I opened my account, but over the course of 2 months).
Since the last time, I've focused a bit on refactoring of the system architecture and a bit on research. Few words on each.
The system is 80% (I got a few more weeks to get to 100%) event-driven, meaning that different components send events between each other to communicate. For example, instrument component is in charge of fetching new prices, and when it gets a new price, it emits NewPrice event, which forecasting component is registered to receive. Forecasting is executed and that emits ForecastUpdated, which position component receives and so on. This makes is very easy to test each component in isolation as well as end-to-end test everything together. I also implemented event logging so in one place I can see exactly what's happening. I didn't want to complicate things so all this event passing is done in-memory, withing a single process (system works with daily prices so I got all the time in the world). Some day I might move to a proper messaging platform like PubSub on Google Cloud or roll out my own (I had good results with Redis before).
Here's a snapshot of what that looks like:
Code:
2020-10-27 19:56:51,169 INFO PROD messagebus Handling message PositionUpdated(timestamp=datetime.datetime(2007, 9, 5, 23, 0), symbol='GBP', position=Decimal('2.553'), avg_position=Decimal('4.331'))
2020-10-27 19:56:51,246 INFO PROD messagebus Handling message ScheduleTrade(timestamp=datetime.datetime(2007, 9, 5, 23, 0), symbol='GBP', contract_ym=202012, quantity=1)
2020-10-27 19:56:51,248 INFO PROD messagebus Handling message ForecastUpdated(timestamp=datetime.datetime(2007, 9, 5, 23, 0), symbol='PL', forecast=Decimal('-2.507), instrument_risk=Decimal('0.136))Besided that, I spent a bunch of time making my own backtesting plumbing that also has reporting. I tried PyFolio, but it seems not to be maintained anymore and I didn't need half of what they have. 2 screenshots above are from that work, it's all in Jupyter. I've made it modular so I can just pass in trades and it spits out all the stats. That means that I get the same stats for realized performance (I just pass in my executed trades) as for backtest (I hook up ScheduleTrade event to execute a trade in the backtest plumbing). Here's one more screenshot, this time from a backtest.
I'm adding more stuff into this, but I've realized there's an infinite number of stats and graph one can add, so this seems like "less-is-more" type of a thing.
For research, I've settled on the notebook-per-topic approach, so I have notebooks like ForecastAnalysis (analyses average forecasts for rules), DiversificationMultipliers, RealizedPerformance and so on. I haven't yet solved the problem of versioning Jupyter notebooks in git, all the methods out there seem like hacks to me, but I think this might be one of those things where there's no ideal solution.
I'm planing to add 9 more markets (currently trading 12) few weeks after the elections, I don't have any solid data on this but I thought I didn't want to double my account and add markets and exposure just before US elections, maybe that's just being too conservative, but I'll sleep better.
I am a bit concerned with the current drawdown but I'm still not touching anything. In the backtest, the system did have one nasty drawdown of 55% around 2007 (mostly due to ESTX50, see below) so it's not like this is atypical.
I do have one question for folks here, do you have markets that are not profitable (or are even losers) in the long term backtest in your portfolio? ESTX50 is that for me, over the 20 year backtest, that just consistently loses money. I'm not pulling it out of portfolio yet because I don't want to cherrypick (I realize that what happened in the past is not the best indication of future) but 20 years does seem like a good amount of data, and it includes 2 bigger crisis periods as well. Do you pull markets like that out or keep them in?
Well, past that's actually happened is essentially just one Monte Carlo run of a random process
... I personally wouldn't know, but people who researched it (one of whom we all know rather well ) say there's no statistical evidence that some instruments "trend more" or "have a more reliable carry performance" than others... There are however folk-sayings like "Eurodollar is a king of trends" and "Equities seem to trend less in the recent years", but I personally only choose instruments by what I can afford to trade and try to balance it with as wide as possible coverage of uncorrelated markets (really want some metal (not this one \m/ ) in my portfolio right now, but can't afford any just yet..). Also regret that I couldn't add any long-term bonds, this is also a real gap in my PROD portfolio, and just as luck would have it, BTP and US 20y are the best performing markets in my PAPER system (which is currently trading at 500k and 36 markets and is still up since launch..).
Last edited:
but I personally only choose instruments by what I can afford to trade and try to balance it with as wide as possible coverage of uncorrelated markets (really want some metal (not this one \m/) in my portfolio right now, but can't afford any just yet..). Also regret that I couldn't add any long-term bonds, this is also a real gap in my PROD portfolio,
I appreciate that with your current portfolio constraints you may not be able to trade large bond contracts such as the Buxl, Ultra US, or big JGB’s, but even with a small portfolio you should certainly be able to afford 1-lot allocations to 10yr Australian and Korean bonds as well as the mini-JGB on SGX
As for metals, take a look at the Tokyo markets …
https://www.jpx.co.jp/english/derivatives/products/precious-metals/index.html
… these are all much smaller contracts than their US market counterparts, plus they’re all priced in JPY which helps to diversify further away from USD based commodities
In addition you could also look at the Micro Gold on CME …
https://www.cmegroup.com/trading/metals/precious/e-micro-gold_contract_specifications.html
… and Iron Ore on SGX …
https://www.sgx.com/derivatives/products/iron-ore#Contract Specifications
All of these are suitable for trading within a smaller portfolio with limited risk constraints
Hey folks, here's another update from me. First was https://elitetrader.com/et/threads/fully-automated-futures-trading.289589/page-209#post-5055797 and second was https://elitetrader.com/et/threads/fully-automated-futures-trading.289589/page-217#post-5127521.
TL;DR - in a nasty drawdown, same as a few other folks I've seen here (hang in there Kernfusion!
Here's the money shot since inception (comparing to S&P 500 because that's what I got data for, in future, I'll probably benchmark against 60/40 as that's what I would put my money in if I wasn't doing this. I know it's not apples to apples, but it's what makes sense for me):
View attachment 242944
Here's the nasty bit:
View attachment 242945
Was it not for extreme volatility and luck at the beginning, I'd be down, and not up 7% (I am still down a few percent in cash terms because I didn't deposit full amount all at once when I opened my account, but over the course of 2 months).
Since the last time, I've focused a bit on refactoring of the system architecture and a bit on research. Few words on each.
The system is 80% (I got a few more weeks to get to 100%) event-driven, meaning that different components send events between each other to communicate. For example, instrument component is in charge of fetching new prices, and when it gets a new price, it emits NewPrice event, which forecasting component is registered to receive. Forecasting is executed and that emits ForecastUpdated, which position component receives and so on. This makes is very easy to test each component in isolation as well as end-to-end test everything together. I also implemented event logging so in one place I can see exactly what's happening. I didn't want to complicate things so all this event passing is done in-memory, withing a single process (system works with daily prices so I got all the time in the world). Some day I might move to a proper messaging platform like PubSub on Google Cloud or roll out my own (I had good results with Redis before).
Here's a snapshot of what that looks like:
Code:2020-10-27 19:56:51,169 INFO PROD messagebus Handling message PositionUpdated(timestamp=datetime.datetime(2007, 9, 5, 23, 0), symbol='GBP', position=Decimal('2.553'), avg_position=Decimal('4.331')) 2020-10-27 19:56:51,246 INFO PROD messagebus Handling message ScheduleTrade(timestamp=datetime.datetime(2007, 9, 5, 23, 0), symbol='GBP', contract_ym=202012, quantity=1) 2020-10-27 19:56:51,248 INFO PROD messagebus Handling message ForecastUpdated(timestamp=datetime.datetime(2007, 9, 5, 23, 0), symbol='PL', forecast=Decimal('-2.507), instrument_risk=Decimal('0.136))
Besided that, I spent a bunch of time making my own backtesting plumbing that also has reporting. I tried PyFolio, but it seems not to be maintained anymore and I didn't need half of what they have. 2 screenshots above are from that work, it's all in Jupyter. I've made it modular so I can just pass in trades and it spits out all the stats. That means that I get the same stats for realized performance (I just pass in my executed trades) as for backtest (I hook up ScheduleTrade event to execute a trade in the backtest plumbing). Here's one more screenshot, this time from a backtest.
View attachment 242946
I'm adding more stuff into this, but I've realized there's an infinite number of stats and graph one can add, so this seems like "less-is-more" type of a thing.
For research, I've settled on the notebook-per-topic approach, so I have notebooks like ForecastAnalysis (analyses average forecasts for rules), DiversificationMultipliers, RealizedPerformance and so on. I haven't yet solved the problem of versioning Jupyter notebooks in git, all the methods out there seem like hacks to me, but I think this might be one of those things where there's no ideal solution.
I'm planing to add 9 more markets (currently trading 12) few weeks after the elections, I don't have any solid data on this but I thought I didn't want to double my account and add markets and exposure just before US elections, maybe that's just being too conservative, but I'll sleep better.
I am a bit concerned with the current drawdown but I'm still not touching anything. In the backtest, the system did have one nasty drawdown of 55% around 2007 (mostly due to ESTX50, see below) so it's not like this is atypical.
I do have one question for folks here, do you have markets that are not profitable (or are even losers) in the long term backtest in your portfolio? ESTX50 is that for me, over the 20 year backtest, that just consistently loses money. I'm not pulling it out of portfolio yet because I don't want to cherrypick (I realize that what happened in the past is not the best indication of future) but 20 years does seem like a good amount of data, and it includes 2 bigger crisis periods as well. Do you pull markets like that out or keep them in?
Thanks for the update. I'm also in the process of doing a lot of refactoring; having got all the functionality I need in pysystemtrade to run my system I now need to back track and clean some stuff up before I can add code to support new strategies (if you're followed my earlier posts, this is somewhere between the end of stage 4 and stage 5).
I wasn't brave enough to use the event driven approach, broadly speaking I have some stuff that just runs daily, and some other stuff that periodically checks for updates (the frequency of checks depends on the latency needs, so for example during order execution the checks are continous). Sort of a listener pattern, except there are no formal subscriptions or messaging platform, just database documentts that hold state (actually for the order execution, it's an IB insync object that holds the state, but let's not get pedantic).
What we have in common though, by the sounds of it, is that things are broken into components rather than monoliths (the previous version of my trading system was much more monolithic) although I don't think I'd describe what I have as 'microservices' or even 'miniservices'
Current drawdown is around 12.5% from HWM in March, where it's basically been since May with the odd up or down.
"I do have one question for folks here, do you have markets that are not profitable (or are even losers) in the long term backtest in your portfolio?"
Yes. And I'd expect to. Back of the envelope, if you have 20 futures markets with an average SR of 0.4 (sounds low, but that you would easily give you a portfolio SR of 0.9) and 20 years of data for each one, then the 95% confidence interval will be from -0.05 to 0.85. So you'd expect one or two losing markets even if the 'true' SR of every market is 0.4. And because of the positive skew nature of trend following, it's likely there will be more losing markets (and a few bigger winners than you'd expect).
(For owners of 'leveraged trading' you may recall that figure 1 has 8 losing markets, albeit that is for a single EWMAC rule on which the average SR for an instrument is 0.24)
My allocation process might downweight such a market (see https://qoppac.blogspot.com/2019/12/new-and-improved-sharpe-ratio.html) gradually over the backtest as more evidence appeared, but I'd certainly not do an in sample fit and cherry pick it out.
Good luck everyone, especially on Tuesday evening!
GAT
Interesting. I also have an execution component, and that's the one I'm having biggest issues with. Ultimate root cause of the problem for me is the fact that IB does not expose an API to fetch cancelled or fully filled orders, that's quite unfortunate. So the way I see it, we're left with three options:
a) have an always online process that will listen to order status updates and update our own storage
b) periodically throughout the day fetch trades (Trade object in ib-insync) and do your own math on fills to see whether you were fully filled
c) keep the process online only during execution
I am currently doing c), which is a problem for long-running trades I want to do, for example rolls. I'd like to implement those in a way that when I enter a roll window, roll trades are sent on my side of the spread so I can fully capture the spread. For these, I'm ok waiting for hours or days (depends on the market and how wide roll window is), and I don't want to have a process waiting for that.
However, what I really don't want to do is my own math on fills, for some reason I'm not a fan of that solution. I'd really like my broker to tell me I'm filled.
So I think I will transition to a variation of a) where the process that's online is a thin wrapper around IB, which just listens for updates from the broker and updated my persistent store. Besides the execution logic (I use your "world's simplest execution algo"
) it will have nothing else. Another nice part about that is that if I ever want to switch brokers, this will make it really easy, as I have a standard interface that the rest of my system uses, and I will only have to change this online component.Interesting. I also have an execution component, and that's the one I'm having biggest issues with. Ultimate root cause of the problem for me is the fact that IB does not expose an API to fetch cancelled or fully filled orders, that's quite unfortunate. So the way I see it, we're left with three options:
a) have an always online process that will listen to order status updates and update our own storage
b) periodically throughout the day fetch trades (Trade object in ib-insync) and do your own math on fills to see whether you were fully filled
c) keep the process online only during execution
I am currently doing c), which is a problem for long-running trades I want to do, for example rolls. I'd like to implement those in a way that when I enter a roll window, roll trades are sent on my side of the spread so I can fully capture the spread. For these, I'm ok waiting for hours or days (depends on the market and how wide roll window is), and I don't want to have a process waiting for that.
However, what I really don't want to do is my own math on fills, for some reason I'm not a fan of that solution. I'd really like my broker to tell me I'm filled.
So I think I will transition to a variation of a) where the process that's online is a thin wrapper around IB, which just listens for updates from the broker and updated my persistent store. Besides the execution logic (I use your "world's simplest execution algo"
Yes a) is the approach I'm using. It runs when an order is executing, but I can also call it periodically which would make it suitable for orders that take a while to execute (my main execution algo has written now is blocking, to ensure it has minimum latency, because there is no possibility of database locks or thrashing slowing it down). For example, I call it as part of my end of day process.
As a cross check, I also periodically check my state of positions and compare to IB API version of the same. If their is a mismatch I lock the instrument so it can't trade until the mismatch is cleared. I'm not super bothered about missing a fill, but having the wrong position means I could do uneccessary trades (or even go crazy, a la Knight Capital).
GAT
This is awesome, thanks! I quickly did an initial scan and I see that you ran into the same issue as me - "how many data points to use" to calculate the stdev of correlation estimate (after Fisher transformation). In my research I found that only values around 4-6 came pretty close to the candidate matrix method proposed values.
I will give a thorough read now.
Later I will give a couple of comments about issues I found in candidate weights interpolation method and an issue in SR adjustment code.