Fully automated futures trading

[Kernfusion]

So I finally tried backtesting of the short term mean reversion strategy (strategy 27) in my system using my own 1-minute bid-ask data I collected in the last ~4-6 years.
But I used buffering of 4, 6 and 14 forecast-units (with the usual cap of 20, so only 1 step each way initially) to change positions (14 worked slightly better but not by much), i.e. I would only trade if the forecast changed by more than 14 units from the forecast that corresponds to the current position. And I used the same trading algo as for my trend system (no limit orders), but in backtests I simply assume I had to cross the spread on every trade. I tried 9 smallest instruments like US 2Y, MES MCG, etc. and I assumed I have full 300k capital for each of them, I also didn't reduce the base capital in drawdowns.
After fixing many bugs (and I'm sure there's plenty more remaining), the results came out positive, i.e. it's not loosing money, but it's not making a lot either, the average single-instrument sharpe was 0.1 and for the overall 9-instrument-system 0.27. The average number of trades per instrument is 216, and the total number is 1945.
Also the PnL curves look quite sporadic, I was expecting to see a classic negative skew pattern with small frequent gains and rare drops, and some instruments do show that, but not all.
Also, there's no guarantee that my data is clean, the one protection I used (with the benefit of having historical daily prices) is to discard all bid-ask ticks which are outside of the Low-High of the current day (because my prod system does catch weird unrealistic prices during illiquid hours, and including them in such a backtest would inflate results).

View attachment 343018

So I don't know, not a looser, but it requires capital, which would have to be subtracted from a system with the theoretical Sharpe of 1, so doesn't seem to make sense, unless I find some bugs in my backtest and it turns out to be better..

I tried another thing with this: I ran this rule on DAILY prices on a larger portfolio (40+ instruments) and a longer history (the oldest instrument starts from 1989, the same data I use to run my main system's backtest)., with the same conditions: each instrument gets 300k capital independently, position-change step=19(in forecast units), no capital correction in drawdowns, and the result came out positive, although with a very large drawdown between 2013 and 2023:

Sharpe=0.66 Skew=-1.29 trCnt: 1254

This I think is interesting for several reasons: I can trust the quality of my daily data much more than my 'tick' data (still, bugs in the code are quite possible), with such a small trading speed, this strategy becomes compatible with the rest of the forecasting rules of the main system and can probably just be added into the "divergent" bucket together with carry and traded on the same instruments with the same capital (just need to figure out what to do with the large position-change threshold, none of the other rules have quite the same feature..), with some small allocation e.g. 10%.

Really, this modified rule is no longer "fast mean reversion" but more of a "buy short-term trend pullbacks", which I think is expected to work theoretically as well. Also, it's not surprising that it works, because it still trades in the direction of the trend (perhaps it's highly correlated with the other trend rules..)

Would be interesting if anyone else could reproduce similar results to confirm that I'm not dealing with a bug or overfitting..

 

[handelsmeisterei]

I tried another thing with this: I ran this rule on DAILY prices on a larger portfolio (40+ instruments) and a longer history (the oldest instrument starts from 1989, the same data I use to run my main system's backtest)., with the same conditions: each instrument gets 300k capital independently, position-change step=19(in forecast units), no capital correction in drawdowns, and the result came out positive, although with a very large drawdown between 2013 and 2023:
View attachment 343529
Sharpe=0.66 Skew=-1.29 trCnt: 1254

This I think is interesting for several reasons: I can trust the quality of my daily data much more than my 'tick' data (still, bugs in the code are quite possible), with such a small trading speed, this strategy becomes compatible with the rest of the forecasting rules of the main system and can probably just be added into the "divergent" bucket together with carry and traded on the same instruments with the same capital (just need to figure out what to do with the large position-change threshold, none of the other rules have quite the same feature..), with some small allocation e.g. 10%.

Really, this modified rule is no longer "fast mean reversion" but more of a "buy short-term trend pullbacks", which I think is expected to work theoretically as well. Also, it's not surprising that it works, because it still trades in the direction of the trend (perhaps it's highly correlated with the other trend rules..)

Would be interesting if anyone else could reproduce similar results to confirm that I'm not dealing with a bug or overfitting..

How do you calculate the pullback forecast exactly? Happy to test this on my data (I only have daily data)

 

[Kernfusion]

How do you calculate the pullback forecast exactly? Happy to test this on my data (I only have daily data)

It's exactly the strategy 27 'Safer fast mean reversion' from the Rob's last book (I implemented it from the excel file, just make sure to correct the error in it, the columns '16 ma' and '64 ma' on the daily data tab point to wrong MA columns, they should point to 'Z' and 'AC' with the calculated values, but they point to the next ones with the sequential numbers). And I just used daily data instead of hourly and the regular order submission and position sizing logic, same as in the previous e.g. trend strategies.

The only change I added is position-change buffering:
After calculating the current forecast (only strategy 27, scaled, capped to 20 etc.), I take my current actual position and run position-sizing logic in reverse, which gives me forecast value that corresponds to my current position, I then compare it with the new forecast and only if the difference is greater than X I trade to adjust my position. (I tried X = 5, 10, 14, 19, it seems that the higher values work better on tick data, but with daily prices it's the opposite..).

 

[handelsmeisterei]

I did a quick and dirty implementation of the forecast and barely got a positive sharpe after costs starting 1975 on 215 instruments. The performance is especially bad up to 1990. From 1990 onwards I get a sharpe of 0.2.

Slowing down the equilibrium to a span of 10 helps a little bit, but the turnover is still so high that most of the instruments don't trade.

I am not sure if futures are a good candidate for mean reversion on daily data. The effect seems to be very weak. The average autocorrelation on my universe is the following for the first 5 lags:

Lag1 Lag2 Lag3 Lag4 Lag5
0.0091451698 -0.0030619776 -0.0080280450 0.0004358402 -0.0044860885

They exhibit weak negative correlation 2 and 3 days later. The effect for US stocks for example is much greater (over -2% 2009 onwards). Perhaps one needs to not only trade in the direction of the trend but also only instruments that have negative autocorrelation. But that feels like overfitting...

 

[Kernfusion]

I did a quick and dirty implementation of the forecast and barely got a positive sharpe after costs starting 1975 on 215 instruments. The performance is especially bad up to 1990. From 1990 onwards I get a sharpe of 0.2.

Slowing down the equilibrium to a span of 10 helps a little bit, but the turnover is still so high that most of the instruments don't trade.

I am not sure if futures are a good candidate for mean reversion on daily data. The effect seems to be very weak. The average autocorrelation on my universe is the following for the first 5 lags:

Lag1 Lag2 Lag3 Lag4 Lag5
0.0091451698 -0.0030619776 -0.0080280450 0.0004358402 -0.0044860885

They exhibit weak negative correlation 2 and 3 days later. The effect for US stocks for example is much greater (over -2% 2009 onwards). Perhaps one needs to not only trade in the direction of the trend but also only instruments that have negative autocorrelation. But that feels like overfitting...

Did you add the buffering (i.e. only trade if the forecast changed substantially, e.g. by 10+, to require a substantial deviation from the equilibrium to enter)? My turnover was quite low especially if I increase the buffer size to enter only the highest deviations.

 

[handelsmeisterei]

Did a little bit more digging: In my universe only equity futures have significant negative autocorrelation. Most of them on lag1. The mean reversion thing seems to be equity-only.

 

[handelsmeisterei]

Did you add the buffering (i.e. only trade if the forecast changed substantially, e.g. by 10+, to require a substantial deviation from the equilibrium to enter)?

No I did not implement that. Did that change make things much better for you?

 

[Kernfusion]

No I did not implement that. Did that change make things much better for you?

Yes, that's actually the main\only thing that made it profitable at all. At least when I tried it on tick data, without the buffering, when the system entered on even the smallest deviations, it was a money-looser, like others mentioned before.
And the turnover can be slowed down very substantially with that buffering\thresholding, i.e. with the normal forecast capping to +\- 20 if we make the minimal signal step e.g. 19, the system will only enter in the most extreme short-term deviations (and will hold position until the price reverts all the way back or the trend reverses), but still only in the direction of the trend., which basically transforms it into a pullback forecast..

 

[handelsmeisterei]

I could not find a big difference in performance if I force a big move in the forecast. At first I thought I had found something, but as it turned out, I made an error: I had a lookahead bias in the forecast difference (the system know the forecast move of the next day).

 

[Kernfusion]

I could not find a big difference in performance if I force a big move in the forecast. At first I thought I had found something, but as it turned out, I made an error: I had a lookahead bias in the forecast difference (the system know the forecast move of the next day).

I see.. When I run this rule alone on daily prices in my full main system (with instrument weights, DO, capital correction, etc.) It does come out positive for me with sharpe around 0.3, but when I add it into the mix with all the other rules I currently trade with 12% allocation, there's almost no difference in the result.. So I don't know, maybe I'll drop it for now..