Fully automated futures trading

[globalarbtrader]

Just saw your point about doing the calculation at the end of the backtest.

The problem with this is, again, markets moving into the backtest halfway through.,

If you're using a fixed IDM then it will be too high in earlier periods with fewer markets. So if you calculate the IDM based on realised vol over the whole period then you'll get an answer that is too low. By using the final correlation matrix you get an IDM that is correct for live trading, albeit too high in the earlier part of the backtest. The most correct solution is to use a varying IDM, which you recalculate periodically using correlation matrices rather than a rolling window of vol estimation, for the reasons below.

GAT

In your book, you propose to calculate the Instrument Diversification Multiplier (let's call it IDM to prevent RSI (https://en.wikipedia.org/wiki/Repetitive_strain_injury) by measuring or estimating the correlations and calculating 1 / sqrt(W x H x W_t) where W are the weights and H is the correlation matrix.

Don't you assume a gaussian return distribution here?

Could we calculate the IDM also like this:

Backtest the system with IDM = 1 and a given volatility target, like 20%, then calculate the realized volatility for the system and setting IDM = Vol_desired / Vol_realized.

The calculation could be done at the end, that would be the same as your calculation where you are using measured correlations for the whole backtest or on an expanding/rolling window base, only using not-seen data.

I think, this should work, since the IDM is linear in your position calculation and stddev(a * x) = |a| * stddev(x) for a constant a.

The calculation/backtest for the IDM should be done with otherwise unrealistic fractional positions (without rounding to full contracts), I think.

You could do the same for the Forecast Diversification Multiplier (FDM), by backtesting with FDM = 1, calculating combined forecasts and calculating the FDM, so that the mean combined signal over all instruments is 10 on average.

 

[nemo4242]

Yes, I thought of calculating the IDM on a expanding/rolling window base and have it time-varying. When new instruments are added to the system because data for them becomes available, you would have to wait for a certain time until you have enough data to calculate the IDM.

But as you say, this is the same whether you use correlations or realized vol to calculate the IDM.

I think one source of noise in the vol of the whole system is the high granularity of positions, i.e. you can only trade full future contracts and they have quite high impact on the system's vol.

If you backtest with equity going to infinity, let's say billions of $, the granularity becomes less and less (Ignoring the fact that you could not buy 1000s of contracts without impact on the trade price). So I think, you could use fractional trade positions (this is like using infinite equity) to calculate the IDM.

I will try both methods and compare the results, especially the fluctuations of the IDM over time.

 

[globalarbtrader]

I'll do a proper monthly review on 5th January, but in case you're interested here is my 2015:

+24%

Big winners:
Gold 2%
AEX 2.1%
BTP 2.2%
Copper 2.5%
MXP 2.5%
Crude 4%
Platinum 4.3%
Palladium 4.5%
Gas 5.4%

Big Losers:
GBP -2.4%
Leanhogs -2.5%
SMI -3.3%

GAT

 

[gonzatti]

very nice returns for such an year!
waiting to see the full report in a couple days!

 

[globalarbtrader]

Monthly update. Last one was December 5th

Up 7.2% of capital or 29K. I got a new HWM towards the end of last month.

No nice picture this month due to technical issues.

Drawdown 1.7% off new HWM

Actually quite a 'quiet' month in terms of p&l, with about half the gain of last month, despite all the gyrations in the world at large (for information I was up around 10K on yesterdays move). If you look at my risk report below, this reflects the fact that I'm not really exposed to bonds or equities, and the moves in commodities were less pronounced than last month. The financial futures are really struggling to show a new trend, with sideways movement the rule of the day.

I know I keep banging on about it, but this is why diversification is key. Diversification across asset classes has really helped me since the equity / bond rally faded about 9 months ago. Diversification across styles has also helped somewhat, with trend following struggling somewhat carry has come back to the party (most CTA's were flat last year, while I made some money).

I hope everyone makes some money in 2016.

Reports:

P&L

Gainers:

Corn 5K
Crude 6.5K
Wheat 4K

Losers:

JPY -3.5K
Livecow -3K


Positions

       code contractid  positions   Lock WrongContract InFwdNotRoll
14      BTP     201603          2  False         False        False
16   COPPER     201603         -2  False         False        False
0      CORN     201612         -8  False         False        False
9   CRUDE_W     201612         -3  False         False        False
4   EDOLLAR     201906          2  False         False        False
10  EDOLLAR     201903          7  False         False        False
5       EUR     201603         -1  False         False        False
2   EUROSTX     201603        -13  False         False        False  (Hedge)
8    GAS_US     201603         -2  False         False        False
13      GBP     201603         -3  False         False        False
11     KR10     201603          2  False         False        False
20      KR3     201603          8  False         False        False
17  LIVECOW     201610         -2  False         False        False
1       MXP     201603         -4  False         False        False
12      OAT     201603          1  False         False        False
3    PALLAD     201603         -1  False         False        False
6      PLAT     201604         -1  False         False        False
19  SOYBEAN     201611         -5  False         False        False
15      US2     201603          3  False         False        False
18      VIX     201602         -1  False         False        False
7     WHEAT     201612         -5  False         False        False

Risk

       code  multisignal  expected_annual_risk  expected_annual_risk_per_contract  position  expected_annual_risk_rounded_pos

Shorts:

28      MXP        -13.1                  8896                               1964        -4                              7857
33     PLAT        -13.1                  8855                               7987        -1                              7987
17      VIX        -10.9                  7367                               9684        -1                              9684
2   LIVECOW        -11.4                  7721                               5584        -2                             11169
35   GAS_US        -18.4                 12496                               6504        -2                             13008
32   PALLAD        -26.8                 18185                              13786        -1                             13786
0      CORN        -22.8                 15459                               1871        -8                             14965
26      GBP        -23.6                 15992                               5063        -3                             15190
30   COPPER        -24.2                 16394                               8868        -2                             17736
4     WHEAT        -29.4                 19941                               3579        -5                             17897
3   SOYBEAN        -33.5                 22719                               4049        -5                             20247
34  CRUDE_W        -43.8                 29646                               8733        -3                             26200

Longs:
36  EDOLLAR         18.1                 12249                               1394         9                             12542
8       BTP         19.7                 13315                               6594         2                             13188

Trades

         code contractid     filled_datetime  filledtrade  filledprice
7735      AUD     201512 2015-12-08 14:08:25           -1     0.719000
7672     BOBL     201603 2015-12-04 08:39:34           -2   130.670000
7675      BTP     201603 2015-12-04 08:43:09           -1   137.010000
7714      BTP     201603 2015-12-07 08:12:25           -1   136.940000
7855      BTP     201603 2015-12-16 09:21:54           -1   136.690000
7879      BTP     201603 2015-12-21 11:14:25            1   137.650000
8152      BTP     201603 2016-01-04 07:08:48            1   138.290000
7774      CAC     201601 2015-12-11 11:09:49           -1  4563.000000
7861      CAC     201601 2015-12-17 10:20:34            1  4738.500000
7885   COPPER     201603 2015-12-21 16:17:10            1     2.137500
8098     CORN     201612 2015-12-23 15:11:06           -1   390.000000
8122     CORN     201612 2015-12-29 14:30:00           -1   385.750000
8167     CORN     201612 2016-01-04 16:05:07           -1   377.250000
8089  CRUDE_W     201612 2015-12-22 14:25:30           -1    42.220000
8125  CRUDE_W     201612 2015-12-29 14:32:30            1    43.630000
7726  EDOLLAR     201906 2015-12-08 14:45:34            1    97.865000
8173      EUR     201603 2016-01-05 03:18:49           -1     1.083400
7813  EUROSTX     201512 2015-12-14 08:05:09            2  3216.000000
7816  EUROSTX     201603 2015-12-14 08:05:09           -2  3207.000000
7819  EUROSTX     201512 2015-12-14 08:09:55            1  3217.000000
7822  EUROSTX     201603 2015-12-14 08:09:55           -1  3208.000000
7825  EUROSTX     201512 2015-12-14 08:12:26            6  3211.000000
7828  EUROSTX     201603 2015-12-14 08:12:26           -6  3202.000000
7831  EUROSTX     201512 2015-12-14 08:15:08            4  3213.000000
7834  EUROSTX     201603 2015-12-14 08:15:08           -4  3204.000000
7720   GAS_US     201602 2015-12-07 13:34:22           -1     2.192000
8092   GAS_US     201602 2015-12-23 12:02:35            5     1.988000
8095   GAS_US     201603 2015-12-23 12:02:35           -5     2.064000
8113   GAS_US     201603 2015-12-28 14:52:19            1     2.234000
8116   GAS_US     201603 2015-12-29 12:24:32            1     2.340000
8149   GAS_US     201603 2015-12-31 17:24:00            1     2.396000
7723      GBP     201603 2015-12-08 08:53:45           -1     1.502900
7741      GBP     201512 2015-12-09 16:12:13            1     1.517200
7801      GBP     201603 2015-12-14 02:03:04            1     1.520000
7858      GBP     201603 2015-12-16 13:23:53           -1     1.501000
7876      GBP     201603 2015-12-18 13:04:43           -1     1.492700
8137      GBP     201603 2015-12-31 02:06:07           -1     1.482400
7696     GOLD     201602 2015-12-04 16:34:17            1  1086.800000
8164     GOLD     201602 2016-01-04 14:11:22            1  1077.500000
7708      JPY     201512 2015-12-07 03:06:23           -1     0.008110
7738      JPY     201512 2015-12-09 14:48:08            1     0.008190
7744      JPY     201512 2015-12-10 03:28:40            1     0.008224
7747      JPY     201512 2015-12-10 06:24:06            1     0.008254
7750      JPY     201603 2015-12-10 06:24:06           -1     0.008273
7804      JPY     201603 2015-12-14 02:06:37            1     0.008287
7864      JPY     201603 2015-12-17 11:21:38           -1     0.008186
7888      JPY     201603 2015-12-21 17:12:41            1     0.008280
7807    KOSPI     201603 2015-12-14 02:18:21           -1   235.850000
7873    KOSPI     201603 2015-12-18 01:12:07            1   239.950000
8005     KR10     201603 2015-12-22 01:42:35            1   126.040000
8134     KR10     201603 2015-12-30 03:10:36            1   126.280000
7705      KR3     201512 2015-12-07 02:46:39            1   109.170000
7777      KR3     201603 2015-12-14 01:04:52            8   109.390000
7780      KR3     201603 2015-12-14 01:06:57            8   109.380000
7783      KR3     201603 2015-12-14 01:10:51            8   109.390000
7786      KR3     201603 2015-12-14 01:36:56            8   109.370000
7789      KR3     201603 2015-12-14 01:39:25            8   109.380000
7792      KR3     201603 2015-12-14 01:43:28            8   109.370000
7795      KR3     201603 2015-12-14 01:47:49            8   109.370000
7798      KR3     201512 2015-12-14 01:49:05           -8   109.400000
7843      KR3     201603 2015-12-15 01:12:02           -5   109.330000
7846      KR3     201603 2015-12-15 05:56:56          -41   109.340000
8086      KR3     201603 2015-12-22 02:49:15           -1   109.560000
8170      KR3     201603 2016-01-05 03:53:01           -1   109.700000
8146  LEANHOG     201606 2015-12-31 14:32:09            1    77.800000
7711      MXP     201512 2015-12-07 03:12:24           -1     0.059870
7753      MXP     201512 2015-12-10 06:41:09            3     0.058490
7756      MXP     201603 2015-12-10 06:41:09           -3     0.058200
7771      MXP     201603 2015-12-11 03:56:55           -1     0.057800
7678      OAT     201603 2015-12-04 08:43:59           -1   150.000000
8101      OAT     201603 2015-12-28 12:13:31           -1   150.040000
8155      OAT     201603 2016-01-04 09:04:02            1   150.650000
7699     PLAT     201601 2015-12-04 16:40:16            1   883.400000
7837     PLAT     201601 2015-12-14 12:14:13           -1   843.200000
7870     PLAT     201601 2015-12-17 13:47:31            1   857.500000
8119     PLAT     201601 2015-12-29 13:20:32            1   891.800000
8140     PLAT     201601 2015-12-31 13:52:59            1   880.500000
8143     PLAT     201604 2015-12-31 13:52:59           -1   882.500000
7684  SOYBEAN     201611 2015-12-04 12:18:15            1   912.750000
7732  SOYBEAN     201611 2015-12-08 13:42:23           -1   899.000000
7768  SOYBEAN     201611 2015-12-10 12:35:13           -1   893.500000
7840  SOYBEAN     201611 2015-12-14 12:20:57           -1   888.250000
7852  SOYBEAN     201611 2015-12-15 14:42:28           -1   888.000000
7867  SOYBEAN     201611 2015-12-17 12:13:34           -1   873.000000
7882  SOYBEAN     201611 2015-12-21 12:06:03            1   911.500000
8104  SOYBEAN     201611 2015-12-28 12:55:43           -1   885.000000
8131  SOYBEAN     201611 2015-12-29 15:52:07           -1   883.250000
8161  SOYBEAN     201611 2016-01-04 11:30:06           -1   878.000000
7687    SP500     201512 2015-12-04 14:04:18           -1  2051.250000
7690      US5     201603 2015-12-04 14:25:42           -1   118.429688
7717      V2X     201601 2015-12-07 08:31:53           -2    23.050000
7762      VIX     201601 2015-12-10 10:07:09            1    18.750000
7765      VIX     201602 2015-12-10 10:28:08           -1    19.100000
7693    WHEAT     201612 2015-12-04 15:36:35           -1   520.000000
8110    WHEAT     201612 2015-12-28 13:15:41           -1   509.750000

Expected slippage £680, actual £447

 

[globalarbtrader]

 

[gonzatti]

~100k GBP in a "bad year"
thats not bad at all =)
congratulations and keep it coming!

btw: have you coded your "anti slippage" algo into something that is meant to profit, aside from diminushing your execution costs ?

 

[globalarbtrader]

~100k GBP in a "bad year"
thats not bad at all =)
congratulations and keep it coming!

btw: have you coded your "anti slippage" algo into something that is meant to profit, aside from diminushing your execution costs ?

I wouldn't say last year was a bad year; in terms of SR at a fraction under 1.0 it was pretty much in line with backtested expectations. Just not as insanely good as 2014.

(I also note that there are plenty of people on this site for whom making a 'mere' 24% would be a pathetic year)

I've got to make some serious progress on my refactoring project (the public face of which is https://github.com/robcarver17/pysystemtrade); basically to get to the point where I've decoupled my code enough so I can run multiple strategies at once. Since I only expect to spend about 1/5 of my time on this project, that could be a while.

GAT

 

[nemo4242]

You mention on your blog (http://qoppac.blogspot.co.uk/2015/04/futures-trading-performance-year-one.html) that you are using breakout style rules for your trading.

How do you get from breakout signals/events that occur when the instrument makes a new high/low to the continuous forecast needed for your framework?

I tried the following:

For a given look-back period, calculate the AROON oscillator (http://stockcharts.com/school/doku.php?id=chart_school:technical_indicators:aroon_oscillator) and use it directly as forecast.

The AROON basically relates the days since the last high to the days since the last low in the look-back window and scales that to -100 .. 100, so I use a forecast scalar of 0.1.

This seems to work and gives comparable sharpe ratios as with the EWMAC rule described in your book.
I used lookback lengths of 200, 100 and 50 for a first try.
The signal is hard-clipped to -10 .. 10 by design with that rule.

 

[globalarbtrader]

You mention on your blog (http://qoppac.blogspot.co.uk/2015/04/futures-trading-performance-year-one.html) that you are using breakout style rules for your trading.

How do you get from breakout signals/events that occur when the instrument makes a new high/low to the continuous forecast needed for your framework?

I tried the following:

For a given look-back period, calculate the AROON oscillator (http://stockcharts.com/school/doku.php?id=chart_school:technical_indicators:aroon_oscillator) and use it directly as forecast.

The AROON basically relates the days since the last high to the days since the last low in the look-back window and scales that to -100 .. 100, so I use a forecast scalar of 0.1.

This seems to work and gives comparable sharpe ratios as with the EWMAC rule described in your book.
I used lookback lengths of 200, 100 and 50 for a first try.
The signal is hard-clipped to -10 .. 10 by design with that rule.

That sounds good. I use a slightly different approach. I take the price range (min/max) from the last N days (depending on the speed of the system) and then normalise the current price within that range. This gives me a number between -20 (current price is at the bottom of the recent historical range) and +20 (at the top).

So it's perhaps not really a breakout system, but it will show extreme forecasts when a breakout occurs.

Unsurprisingly it's highly correlated (80%) with EWMAC but sufficiently different to make it worth including.

GAT