Fully automated futures trading

[globalarbtrader]

Originally I hadn't really thought about doing a year end post for once (as you will see, I haven't really done much of interest), but I've been dragged back to the thread by @Kernfusion interesting post, so whilst I'm here...

As everyone knows it's not been a great year for the CTA industry, SG index is down 3% as of a couple of days ago, with some good performance in February, 'that March event' where I lost about 7.5%, a good run up over the summer coming just shy of my HWM in early October, then some losses although things have now stabilised. Overall then perhaps my return of about 3% is adequate, with the UK tax year so far a bit better at 7%. Let's not forget we can't assume risk free is zero and my performance includes interest of around 2.5%, so the Sharpe Ratios aren't great.Current d/d is 7.8%.

I did make a good discretionary call though:

https://twitter.com/x/status/1720078981045903507

In terms of achievements from last year, well I got my book out in April and published all the associated code and spreadsheets. It has sold very well for a very thick and expensive book which is aimed at a narrow audience; although of course it is galling that in the UK Amazon category 'professional investment in futures' where it was briefly #1 shortly after release, it is currently outsold by 24 other books half of which I wouldn't wipe my arse with, and of the remaining good ones aren't really 'professional investment in futures' books.

I also managed to finally clean up the arctic dependency on pysystemtrade, although if I am honest that was rather forced on me when my previous laptop died, and I had to start with a new build. I also got a very small amount of refactoring done.

Research has been pretty minimal, if I am honest. My list of research ideas to look at got longer over the year (and @Kernfusion has just added another one, thanks for that), as I keep coming up with new ones and not having the time to look at them, never mind implement those I have already researched (eg MR and RV). I did some semi serious looking at trading faster earlier in the year which mostly confirmed I am doing the right thing, otherwise just some random more academic things on my blog and a single post about a practical trading strategy based on vol levels which I will probably implement at some point.

I have been very busy with some non profit stuff I do, unrelated to finance. It's involved a fair bit of programming which I find I enjoy more and more. This is a committment that will keep me very busy into Q3 next year, and then will start to lighten.

Until then, it will be all about:

- teaching, again
- usual round of conferences (I have got into a bit of a pattern here now; most of the talks I did last year were at events I've done before)
- consulting (still on retainer for a crypto hedge fund, may god forgive me)
- exercise (increasingly doing more running than cycling, and actually did some semi serious training although christmas colds have put a temporary stop to it)

The only trading related project I really want to do is to add the new instruments I'm now sampling to my trading strategy; I'll do that just before the end of the tax year. I now have a long enough price history to be able to accurately calculate costs for them. That will bring me up to 252 instruments (of course many of these I don't / can't actually hold positions in).

Once I hit Q4 I'm hoping to be able to look at:

- that huge pile of research ideas *
- refactoring and generally improving pysystemtrade
- the next book... current list is (notice the recombination of earlier ideas):

1- a book on systematic trading and investing for dummies; covering 'delta one' instruments stocks, ETF and crypto (sorry). Basically 'Leveraged Trading' but without the leverage. This would be the gateway book into all my other books, so I'd have to dumb it down as much as possible. It would also cover factor investing for dummies. so in part would be a simpler (and shorter) version of my 2nd book, 'Smart Portfolios'.
2 - a book on backtesting and uncertainty in financial markets; but not with python as I've discussed before. This would also effectively be a textbook for parts of my university course.
3 - a popular 'secrets of the quants' book

* I've recently become enamoured with a highly abstract idea which you can eithier think of as a overarching strategy and all other strategies are just special cases of it, or something that can be used in a more practical way to trade highly multi dimensional portfolios. I do feel that with 252 instruments across all the various asset classes and my experience I ought to be able to do something a bit 'fancier' and higher SR, even given my limited capital. This is just a vague idea at this stage... may be nothing, or one day another strategy or another book.

So - it's a cliche but - Happy christmas and a merry new year

Rob

 

[Kernfusion]

By the way in theory you might not want *cheap* instruments

Hi Rob, Sorry, I actually meant "smallest" in this sentence: "We want the "smallest" instruments to be able to trade as many of them as we can to get the max diversification in MR with a small capital.", for some reason I mixed up small with cheap in my head , you're right, cheapness doesn't play a big role here, it's all about fitting as many different instruments as possible into a small capital..

Regarding the implementation, it might be easier for me because my system has been event-driven from the beginning, i.e. I do all-rules evaluation on every tick while the system is running (~ 19h per day, rest is maintenance time), at first I even did the full DO on every tick from every instrument, but later introduced an artificial 10-minute throttle..

The tricky part for me is the small capital, and because the systems will be somewhat overlapping, maybe I can get away with sharing the capital between them? (this however might smell like overleveraging and taking too much risk..) I.e. first I'm not sure how much capital I would need for a single instrument in the fast-MR, and what's the minimum number of instruments I'd need to trade to make it worth doing.
E.g. if I'm currently trading 300k$ in the slow system, I can now for example allocate 100k to the MR trading 20 smallest instruments and reduce the base capital of the slow system to 250k without adding any more actual cash, sort of assuming that the 50k will be "shared"..
There's also a possibility to do the "Horrible path dependence" option for capital sharing among the instruments in the MR (you tried that on the main system in one of the posts preceding DO, unsuccessfully, where we monitor many MR instruments, but the first one which has a signal takes the available risk-budget, blocking others from taking it..)

Overall, yeah, I might try to experiment with it - add one MR instrument (e.g. micro-gold (current margin:1,278$) ) into my Paper system, which I always run in parallel with Prod and see how it behaves..

Happy holidays everyone!

 

[wopr]

Thanks for the summary Rob!

I see fast MR system mentioned, so wanted to see if anybody has success replicating that. I took a go a few months ago, and couldn't get results that would be similar to those in AFTS.

I dug into the code, and I think there might be a bug - specifically, lookahead bias in the equilibrium calculation. If I understood correctly, equilibrium should be lagged by 1 day since we're computing it using closing prices. If we don't lag it, we're using today's closing price to trade intraday, which we can't know. I've checked the code that comes with the book, and equilibrium is not lagged.

Also, when I don't lag it, I get quite similar results to Robs, with SR 1.54 (I have less instruments). When I do lag it, it looks quite bad.

Here's without lag:

Here's with lag:

I'm 90% sure I'm missing something obvious. The implementation I did was safer fast MR, chapter 27.

 

[wopr]

Also, regarding DO, I think there might be a minor bug in the book, in the example that walks through an iteration of the greedy algorithm. Page 396, when we calculate the first standard deviation of the tracking error portfolio, I get 2.89%, instead of 2.67%.

Here's the entire code to replicate, using covariance matrix and weights from the book:

import math
import numpy as np

covs = np.array([
    [0.002704, 0.003838, -0.000889],
    [0.003838, 0.006724, -0.001402],
    [-0.000889, -0.001402, 0.029241]
])
tracking_errors = np.array([-0.088, -0.216, -0.124])

math.sqrt(tracking_errors.dot(covs).dot(tracking_errors))

 

[wopr]

Also, regarding DO, I think there might be a minor bug in the book, in the example that walks through an iteration of the greedy algorithm. Page 396, when we calculate the first standard deviation of the tracking error portfolio, I get 2.89%, instead of 2.67%.

Here's the entire code to replicate, using covariance matrix and weights from the book:

import math
import numpy as np

covs = np.array([
    [0.002704, 0.003838, -0.000889],
    [0.003838, 0.006724, -0.001402],
    [-0.000889, -0.001402, 0.029241]
])
tracking_errors = np.array([-0.088, -0.216, -0.124])

math.sqrt(tracking_errors.dot(covs).dot(tracking_errors))

(Edit time has passed for the above post, so making a new reply).

I ran the entire optimization, and I get optimal positions as (0, 1, 3) for 5 year, 10 year and Micro S&P500, respectively, so one more contract in S&P 500 than Rob. This could make sense as the unrounded optimal position we started with is 3.1, however, Rob has a good explanation in the book why 2 contracts could be a good value as well.

Here's the entire algorithm output, with tracking error and weights:

Initial STD(tracking_error) is 2.891% for weights [0, 0, 0]

STD(tracking_error) is 2.336% for weights [0.22, 0, 0]
STD(tracking_error) is 2.115% for weights [0, 0.24, 0]
STD(tracking_error) is 2.496% for weights [0, 0, 0.04]
New best weights are [0, 0.24, 0], with error 2.115%

STD(tracking_error) is 2.371% for weights [0.22, 0.24, 0]
STD(tracking_error) is 2.887% for weights [0, 0.48, 0]
STD(tracking_error) is 1.44% for weights [0, 0.24, 0.04]
New best weights are [0, 0.24, 0.04], with error 1.44%

STD(tracking_error) is 1.752% for weights [0.22, 0.24, 0.04]
STD(tracking_error) is 2.38% for weights [0, 0.48, 0.04]
STD(tracking_error) is 0.783% for weights [0, 0.24, 0.08]
New best weights are [0, 0.24, 0.08], with error 0.783%

STD(tracking_error) is 1.205% for weights [0.22, 0.24, 0.08]
STD(tracking_error) is 1.984% for weights [0, 0.48, 0.08]
STD(tracking_error) is 0.295% for weights [0, 0.24, 0.12]
New best weights are [0, 0.24, 0.12], with error 0.295%

STD(tracking_error) is 0.877% for weights [0.22, 0.24, 0.12]
STD(tracking_error) is 1.772% for weights [0, 0.48, 0.12]
STD(tracking_error) is 0.705% for weights [0, 0.24, 0.16]
Done with optimization.

Optimal positions: {'ZF': 0.0, 'ZN': 1.0, 'MES': 3.0}

 

[Pavlov]

Also, regarding DO, I think there might be a minor bug in the book, in the example that walks through an iteration of the greedy algorithm. Page 396, when we calculate the first standard deviation of the tracking error portfolio, I get 2.89%, instead of 2.67%.

Here's the entire code to replicate, using covariance matrix and weights from the book:

import math
import numpy as np

covs = np.array([
    [0.002704, 0.003838, -0.000889],
    [0.003838, 0.006724, -0.001402],
    [-0.000889, -0.001402, 0.029241]
])
tracking_errors = np.array([-0.088, -0.216, -0.124])

math.sqrt(tracking_errors.dot(covs).dot(tracking_errors))

I got the same results as you did. If you flip the covariance matrix or the weights you get the results from the book.

math.sqrt(tracking_errors.dot(np.flip(covs)).dot(tracking_errors))

 

[jiikoo]

Just wanted to say BIG thanks to Rob, and all others involved, for the work you have done for all of us. My year 2024 started with significant leap forward in the world of quantitative investing as I'm finally running pst fully automated in production. I started ten years ago with stocks and ETF's with no prior knowledge and after countless mistakes and books read I'm now running fully automated DIY hedge fund! I find it pretty amazing what kind of tools and knowledge are publicly available to people outside of the industry. So thank you so much Rob and hopefully you will find the time and motivation to continue developing pst and write more books, the effort is greatly appreciated!

 

[jiikoo]

Rob, would it be possible to share backtest statistics of your current DO system? The last one I found in the blog was Dec 2021. Would be interesting to see with all the instruments you now have data for.

 

[globalarbtrader]

Rob, would it be possible to share backtest statistics of your current DO system? The last one I found in the blog was Dec 2021. Would be interesting to see with all the instruments you now have data for.

Why? I wouldn't find it interesting at all.

Rob

 

[jiikoo]

I'm curios what would be the expected performance of a DO system with 500k capital, those 40 rules and 250 instruments with as much data history as possible. And how it compares AFTS ch25 statistics with 100 instruments.