Fully automated futures trading

[AvantGarde]

Hi GAT,
I had a question. I vaguely recall at one point you mentioned you backadjust futures for signal inputs but not for P&L, is this correct? One reason I ask as I am recreating a carry type rule and when I run this on my (ratio) backadjusted data my equity curve goes in the complete opposite direction to what I would expect, whereas when I run this on non-backadjusted data (after removing spikes) the equity curve goes in the other direction (in line with my expectations). Do you have any thoughts on this?
Thanks!

 

[test_user]

Hi GAT,
I had a question. I vaguely recall at one point you mentioned you backadjust futures for signal inputs but not for P&L, is this correct? One reason I ask as I am recreating a carry type rule and when I run this on my (ratio) backadjusted data my equity curve goes in the complete opposite direction to what I would expect, whereas when I run this on non-backadjusted data (after removing spikes) the equity curve goes in the other direction (in line with my expectations). Do you have any thoughts on this?
Thanks!

Perhaps I'm misunderstanding something, but how can you calculate carry using backadjusted data? Calculating carry requires comparison of different month contracts (or a contract vs spot price).

 

[AvantGarde]

Perhaps I'm misunderstanding something, but how can you calculate carry using backadjusted data? Calculating carry requires comparison of different month contracts (or a contract vs spot price).

It may not be correct, which is why I am asking, but you could back-adjust the first contract series and back-adjust the second contract series to calculate carry

 

[test_user]

It may not be correct, which is why I am asking, but you could back-adjust the first contract series and back-adjust the second contract series to calculate carry

You don't have to back adjust anything when calculating carry. For example nearest month in ES is December 2021. Next contract month is March 2022. So you take these two and get the price difference between them for a number of days. Then take into consideration that the contracts are 3 months apart. This is it basically.

 

[Kernfusion]

I was finally able to run a backtest of my system with dynamic optimization, I haven't implemented any cost correction yet, and yeah, I can see a drop in PnL and Sharpe and 4.5x increase in the number of trades:

- With the same number of 20 instruments as in the old system but added dynamic-optimization on 230k capital, the 30year PnL dropped from 870k to 639k, and the number of trades increased from 6,364 to 16,847
- when I increased number of instruments to 40 with the same amount of capital and dynamic optimization, PnL increased slightly to 670k and the number of trades jumped even more to 27,880, so this still didn't beat the original no-optimization backtest.

My backtest is not vectorized, i.e. it's the same PROD system which processes prices one at a time, just when backtesting, I'm pumping historical EOD-prices through it instead of real ones. So it calculates the number of trades precisely and it assumes that it gives away the whole bid-ask spread on each execution + pays 5$ commission per contract(in reality this should be slightly better because of the execution algo).
Also, my backtest is very crude, it assumes constant fx-rates, it uses the same instrument and forecast weights for the entire history and it might have some bugs and incorrect historical prices, so I wouldn't trust these results all that much. But it should at least show realistic number of trades.
I will now try to incorporate cost-penalty into the optimization to bring the number of trades to the same number as without optimization..

 

[globalarbtrader]

Hi GAT,
I had a question. I vaguely recall at one point you mentioned you backadjust futures for signal inputs but not for P&L, is this correct? One reason I ask as I am recreating a carry type rule and when I run this on my (ratio) backadjusted data my equity curve goes in the complete opposite direction to what I would expect, whereas when I run this on non-backadjusted data (after removing spikes) the equity curve goes in the other direction (in line with my expectations). Do you have any thoughts on this?
Thanks!

I use backadjusted futures for signals and p&l in sim; in production I calculate p&l on individual contract prices.

As others have said I wouldn't calculate carry as the difference between two backadjusted prices, but actually that isn't a dumb thing to do as it will result in a smoother signal, although it's a lot of work to calculate multiple backadjusted prices for a single instrument and in many cases not even possible.

GAT

 

[Kernfusion]

I was finally able to run a backtest of my system with dynamic optimization, I haven't implemented any cost correction yet, and yeah, I can see a drop in PnL and Sharpe and 4.5x increase in the number of trades:

- With the same number of 20 instruments as in the old system but added dynamic-optimization on 230k capital, the 30year PnL dropped from 870k to 639k, and the number of trades increased from 6,364 to 16,847
- when I increased number of instruments to 40 with the same amount of capital and dynamic optimization, PnL increased slightly to 670k and the number of trades jumped even more to 27,880, so this still didn't beat the original no-optimization backtest.

My backtest is not vectorized, i.e. it's the same PROD system which processes prices one at a time, just when backtesting, I'm pumping historical EOD-prices through it instead of real ones. So it calculates the number of trades precisely and it assumes that it gives away the whole bid-ask spread on each execution + pays 5$ commission per contract(in reality this should be slightly better because of the execution algo).
Also, my backtest is very crude, it assumes constant fx-rates, it uses the same instrument and forecast weights for the entire history and it might have some bugs and incorrect historical prices, so I wouldn't trust these results all that much. But it should at least show realistic number of trades.
I will now try to incorporate cost-penalty into the optimization to bring the number of trades to the same number as without optimization..

I finally implemented cost penalty and reran the backtest: with shadow-cost parameter adjusted to the same number of trades as the original system, the results look very good: PnL=1,037,130, Sharpe=0.93 (~40 instruments) compared to 871,287.69$\0.75 (22 instruments, same capital) without dynamic optimization. So this is clearly worth doing! The next step is to deploy it to the paper system and monitor it for a while before propagating it to prod.

Also, one more thing:

There is one basic problem here: the greedy algo. In particular when the sign of the underlying forecast flips it always close any trade it has on in that instrument.

- regarding this problem, I want to try to add a simple logic into the greedy algo: when calculating "direction" of the instrument as the sign of the new unrounded optimal portfolio add the condition: "if the current position is not zero and it has opposite sign from the new target position, and the absolute difference between the current and new positions is less than 2 contracts -> use the same sign for direction as the current position, not as target". Maybe this will help to prevent frequent position closings from minor forecast fluctuations..

 

[globalarbtrader]

I noe have a buffering type approach which also improves the cost profile. It's checked in and I'm using it but won't be able to write a blog post on the subject for a week or so.

GAT

I finally implemented cost penalty and rerun the backtest: with shadow-cost parameter adjusted to the same number of trades as the original system, the results look very good: PnL=1,037,130, Sharpe=0.93 (~40 instruments) compared to 871,287.69$\0.75 (22 instruments, same capital) without dynamic optimization. So this is clearly worth doing! The next step is to deploy it to the paper system and monitor it for a while before propagating it to prod.

Also, one more thing:

- regarding this problem, I want to try add a simple logic into the greedy algo: when calculating "direction" of the instrument as the sign of the new unrounded optimal portfolio add the condition: "if the current position is not zero and it has opposite sign from the new target position, and the absolute difference between the current and new positions is less than 2 contracts -> use the same sign for direction as the current position, not as target". Maybe this will help to prevent frequent position closings from minor forecast fluctuations..

 

[tradrjoe]

Hi GAT,
Not sure if you have addressed this before, but what proportion of your total portfolio is allocated to your automated strategy? In terms of "forever" horizon asset allocation, what capital split would make sense between automated/active trading, global equities, bonds, and commodities?

 

[globalarbtrader]

Hi GAT,
Not sure if you have addressed this before, but what proportion of your total portfolio is allocated to your automated strategy? In terms of "forever" horizon asset allocation, what capital split would make sense between automated/active trading, global equities, bonds, and commodities?

I can't tell you the exact number, because it would then be a trivial mathematical problem to calculate my total net worth - which I'd rather keep private, if that's okay. However I can tell you that it's somewhere between 5% and 50%

The 'forever' asset allocation is a tricky one as I'm not sure I want to be trading futures - even automatically- when I'm 70, never mind 80. Ignoring that however, I think it's reasonable to allocate something around the 25-40% mark to an active trading strategy, assuming that it has a low correlation to long only assets.

GAT