Fully automated futures trading

globalarbtrader · May 5, 2016

isotope1 said:
I'm up to bootstrapping trading rules on single instruments with rolling windows.

I've been writing my code from scratch to better understand it, only looking at pysystemtrade for clues.

I have one problem. I wrote my accountCurve using a walk-forward backtester. It's extremely slow, so even a simple bootstrap with limited scope takes several hours, despite having multithreaded the optimisations.

I noticed in your code that you use a vectorised implementation based on statistics, but I'm struggling to understand why it works. Would you mind explaining it, as if I were five years old?

Thanks a million!

Can you point to the piece of code that isn't clear?

GAT

isotope1 · May 5, 2016

It should be linked here:
https://github.com/robcarver17/pysystemtrade/blob/master/syscore/accounting.py#L339

But if not, it's lines 339-369 of accounting.py.

Thank you!

globalarbtrader · May 5, 2016

isotope1 said:
It should be linked here:
https://github.com/robcarver17/pysystemtrade/blob/master/syscore/accounting.py#L339

But if not, it's lines 339-369 of accounting.py.

Thank you!

To be clear: this code returns a position; sometimes given only a forecast, but possibly also with other information. Essentially it does all the calculations in chapter 7-10 of the book "in one go" from forecast to position, making assumptions where it needs to. It "works" because we evaluate forecast p&l for bootstrapping making the same assumptions which means we know what the implicit vol target of the little subsystem trading just one market and trading rule is.

Clearer?

GAT

globalarbtrader · May 5, 2016

Monthly review (last one was April 5th)

P&L period: -10.0%
P&L to date: 88.1%
Drawdown: 13.5%

A bit of a bump in the road. Three weeks of bleeding slowly to death, followed by a flat period as I've almost completely derisked. A 13.5% d/d is by no means lethal. It's only slightly higher than the d/d I had a couple of months ago. It's about half the ~20% d/d I was running in the middle of last year. I'm still up, what, 12% for the calendar year? I've made more than 30% in the last 12 months, compared to backtested averages of 18%. It's important to keep all this in perspective.

On interesting thing is I tend to find I check my p&l less when I'm losing money; whilst each new HWM I like to see happening in real time.

Winner:

Soybean +16K

LoserS:

BTP -2.7K
Corn -3.5K
Crude -8.7K
Gas -3.7K
Hedged equities -8.6K

A very interesting month. I can't remember the last time I saw such stark dispersion. Pretty much every trade I had on lost money - a lot of trends reversed. Except soybean. This is an interesting example of typical trend following skewed returns, lots of small losers, small number of big winners, but in the cross section. That's why diversification is so important. In a month when only a few things trend you need have enough things in your portfolio to have a chance of including the stuff that works.

Positions

Code:

       code contractid  positions   Lock WrongContract InFwdNotRoll
6      BOBL     201606          1  False         False        False
3      CORN     201612         -2  False         False        False
4   EDOLLAR     201906          9  False         False        False
5   EDOLLAR     201909          4  False         False        False
17  EUROSTX     201606         -9  False         False        False
2    GAS_US     201607         -2  False         False        False
11      GBP     201606          1  False         False        False
14     KR10     201606          3  False         False        False
13      KR3     201606         10  False         False        False
15  LIVECOW     201610         -1  False         False        False
1       MXP     201606          1  False         False        False
12      NZD     201606          1  False         False        False
0      PLAT     201607          1  False         False        False
16  SOYBEAN     201611          4  False         False        False
18      US2     201606          2  False         False        False
9       US5     201606          1  False         False        False
8       V2X     201606         -4  False         False        False
10      VIX     201606         -1  False         False        False
7     WHEAT     201612         -1  False         False        False

A minor point to make here is I'm delaying my V2X roll because the rolldown on the June's I'm holding is extraordinary; a very weird shaped curve (I'm short).

Code:

Price matrix
                     201605  201606  201607  201608
2016-05-03 23:00:00   25.95  28.550   26.60   26.35
2016-05-04 23:00:00   26.50  28.700   27.05   26.70

Risk

Code:

       code  multisignal  expected_annual_risk  expected_annual_risk_per_contract  position  expected_annual_risk_rounded_pos
7      BOBL          2.0                  1366                               1799         1                              1799
15      US5          3.8                  2643                               2340         1                              2340
13      US2          4.6                  3190                               1367         2                              2734
28      MXP          2.5                  1743                               2950         1                              2950
6       KR3          8.9                  6109                                556        10                              5557
26      GBP         10.8                  7451                               5718         1                              5718
5      KR10         10.4                  7125                               2235         3                              6705
29      NZD          4.6                  3166                               6896         1                              6896
33     PLAT          8.4                  5791                               8251         1                              8251
36  EDOLLAR         25.2                 17308                               1333        13                             17333
3   SOYBEAN         27.5                 18887                               5091         4                             20363

16      V2X         -6.1                  4220                                804        -4                              3217
4     WHEAT         -9.9                  6788                               5074        -1                              5074
2   LIVECOW         -6.7                  4634                               5874        -1                              5874
17      VIX         -9.4                  6453                               6308        -1                              6308
0      CORN         -6.1                  4178                               3236        -2                              6471
35   GAS_US        -13.7                  9414                               6098        -2                             12195
36  EDOLLAR         25.2                 17308                               1333        13                             17333
3   SOYBEAN         27.5                 18887                               5091         4                             20363

Soybean my biggest long on the back of that huge rally. Otherwise my risk is really, really tiny. About 40% of the average.

Trades

Code:

          code contractid     filled_datetime  filledtrade  filledprice
9886       AEX     201605 2016-04-14 14:35:02            1   444.650000
10153      AEX     201605 2016-04-28 10:24:25           -1   440.000000
10165      AEX     201605 2016-04-29 08:01:05            1   442.250000
10168      AEX     201605 2016-04-29 14:22:33           -1   439.300000
9835       AUD     201606 2016-04-08 03:13:18           -1     0.750300
9853       AUD     201606 2016-04-12 02:52:10            1     0.759900
9931       AUD     201606 2016-04-18 15:35:50            1     0.771500
10078      AUD     201606 2016-04-25 02:38:11           -1     0.768200
10144      AUD     201606 2016-04-28 02:46:01           -1     0.757600
10057     BOBL     201606 2016-04-21 11:47:44           -1   131.060000
10150     BOBL     201606 2016-04-28 08:17:17           -1   131.160000
9799       BTP     201606 2016-04-07 08:40:22           -1   139.980000
9823       BTP     201606 2016-04-07 14:30:14           -1   138.790000
9838       BTP     201606 2016-04-08 08:55:41           -1   139.060000
10135      BTP     201606 2016-04-27 08:01:22           -1   137.390000
10156      BTP     201606 2016-04-28 11:19:33            1   138.290000
10066     BUND     201606 2016-04-22 08:41:45           -1   162.480000
9877       CAC     201605 2016-04-14 08:45:34            1  4407.000000
10216      CAC     201605 2016-05-04 11:06:17           -1  4295.500000
9820    COPPER     201609 2016-04-07 12:27:39           -1     2.123000
9940    COPPER     201609 2016-04-19 16:05:48            1     2.234500
9880      CORN     201612 2016-04-14 14:30:08            1   383.250000
9919      CORN     201612 2016-04-15 16:10:39            1   387.500000
10045     CORN     201612 2016-04-20 14:30:00            1   397.000000
10075     CORN     201612 2016-04-22 14:30:31            1   390.750000
9856   CRUDE_W     201612 2016-04-12 13:00:49            1    44.850000
10159  CRUDE_W     201612 2016-04-28 13:47:01            1    47.870000
10039  EDOLLAR     201909 2016-04-20 13:38:13            1    98.510000
9841   EUROSTX     201606 2016-04-08 09:02:31            3  2828.000000
9847   EUROSTX     201606 2016-04-11 08:01:22            1  2829.000000
9811    GAS_US     201606 2016-04-07 12:20:30           -1     1.994000
9832    GAS_US     201606 2016-04-07 16:21:10            1     2.068000
9871    GAS_US     201606 2016-04-13 14:01:26            1     2.091000
9874    GAS_US     201606 2016-04-14 06:28:17           -1     2.118000
10042   GAS_US     201606 2016-04-20 14:05:00            1     2.196000
10081   GAS_US     201606 2016-04-25 12:02:44            1     2.257000
10138   GAS_US     201607 2016-04-27 14:27:19           -1     2.270000
10222   GAS_US     201607 2016-05-04 12:29:34           -1     2.231000
10063      GBP     201606 2016-04-21 14:05:19            1     1.442800
10162      GBP     201606 2016-04-29 02:51:18            1     1.465000
10225      GBP     201606 2016-05-04 15:22:10           -1     1.449500
10186     GOLD     201606 2016-05-03 12:19:52            1  1295.600000
10219     GOLD     201606 2016-05-04 12:25:05           -1  1277.700000
9796       JPY     201606 2016-04-07 03:11:14            1     0.009158
9889       JPY     201606 2016-04-15 03:07:47           -1     0.009129
9922       JPY     201606 2016-04-18 02:34:29            1     0.009260
10051      JPY     201606 2016-04-20 20:14:28           -1     0.009129
9850       KR3     201606 2016-04-12 03:06:41            1   110.290000
10054      KR3     201606 2016-04-21 05:01:20           -1   110.200000
10147      KR3     201606 2016-04-28 04:07:16            1   110.360000
9913   LEANHOG     201606 2016-04-15 14:32:35           -1    77.025000
10183      MXP     201606 2016-05-03 02:13:27            1     0.057930
9829    NASDAQ     201606 2016-04-07 15:49:18           -1  4489.750000
9916    NASDAQ     201606 2016-04-15 14:34:44            1  4542.250000
10126   NASDAQ     201606 2016-04-25 14:00:27           -1  4457.000000
9925       NZD     201606 2016-04-18 04:40:43            1     0.689600
9865       OAT     201606 2016-04-13 08:20:03           -1   158.050000
9934       OAT     201606 2016-04-19 10:57:55           -1   157.810000
10177     PLAT     201607 2016-05-02 12:13:16            1  1083.200000
9892       SMI     201606 2016-04-15 08:43:26            1  7938.000000
10174      SMI     201606 2016-05-02 08:15:41           -1  7928.000000
9859   SOYBEAN     201611 2016-04-12 13:07:33            1   945.250000
9928   SOYBEAN     201611 2016-04-18 13:01:59            1   970.250000
10123  SOYBEAN     201611 2016-04-25 12:45:24           -1   978.750000
10210  SOYBEAN     201611 2016-05-03 12:54:58            1  1028.000000
9862     SP500     201606 2016-04-12 14:34:17           -1  2037.250000
9883     SP500     201606 2016-04-14 14:04:16            1  2078.250000
10171    SP500     201606 2016-04-29 15:12:06           -1  2058.500000
10213     US10     201606 2016-05-03 15:40:00           -1   130.421875
9826       US2     201606 2016-04-07 14:35:50            1   109.406250
9844       US5     201606 2016-04-08 16:56:09            1   121.453125
10048      US5     201606 2016-04-20 18:57:27           -1   120.789062
9868       V2X     201606 2016-04-13 08:33:22           -1    27.500000
9943       V2X     201605 2016-04-20 08:18:36            1    23.050000
9946       V2X     201606 2016-04-20 08:18:36           -1    27.600000
9949       V2X     201606 2016-04-20 08:24:07           -1    27.550000
10132      V2X     201606 2016-04-26 08:07:25           -1    27.200000
9937     WHEAT     201612 2016-04-19 13:14:32            1   507.250000
10060    WHEAT     201612 2016-04-21 12:02:34            1   537.500000
10141    WHEAT     201612 2016-04-27 14:32:11            1   512.250000

Expected slippage £328
Actual £115. A saving of £110 not much in the face of a 40 grand loss, but every little helps!

As you'd expect from a bad month, a lot of choppy trades.

GAT

tradrjoe · May 5, 2016

Sorry if you already have this posted somewhere in the thread -- what is your full universe of futures? Do you trade all of these instruments on the same platform?

globalarbtrader · May 6, 2016

tradrjoe said:
Sorry if you already have this posted somewhere in the thread -- what is your full universe of futures? Do you trade all of these instruments on the same platform?

Code:

       code       fx      vol  pointvalue  expected_annual_risk_per_contract
11    SHATZ  1.14905   0.0270        1000                                343
6       KR3  0.00087   0.0575     1000000                                550
16      V2X  1.14905   0.6170         100                                782
36  EDOLLAR  1.00000   0.0482        2500                               1330
13      US2  1.00000   0.0609        2000                               1345
7      BOBL  1.14905   0.1420        1000                               1801
5      KR10  0.00087   0.2311     1000000                               2213
15      US5  1.00000   0.2087        1000                               2303
28      MXP  1.00000   0.0005      500000                               2915
0      CORN  1.00000   5.7743          50                               3186
1   LEANHOG  1.00000   0.8491         400                               3748
12     US10  1.00000   0.3528        1000                               3893
37  EUROSTX  1.14905  37.2879          10                               4728
4     WHEAT  1.00000   9.1172          50                               5030
3   SOYBEAN  1.00000   9.9404          50                               5484
26      GBP  1.00000   0.0081       62500                               5586
8       BTP  1.14905   0.4580        1000                               5807
2   LIVECOW  1.00000   1.3716         400                               6054
17      VIX  1.00000   0.5501        1000                               6070
35   GAS_US  1.00000   0.0558       10000                               6157
9      BUND  1.14905   0.4899        1000                               6212
10      OAT  1.14905   0.5047        1000                               6400
20      CAC  1.14905  51.8623          10                               6576
29      NZD  1.00000   0.0060      100000                               6662
23    SP500  1.00000  13.2490          50                               7310
25      EUR  1.00000   0.0055      125000                               7542
24      AUD  1.00000   0.0069      100000                               7662
18    KOSPI  0.00087   1.6344      500000                               7828
22   NASDAQ  1.00000  35.6690          20                               7872
33     PLAT  1.00000  14.2950          50                               7887
30   COPPER  1.00000   0.0300       25000                               8274
34  CRUDE_W  1.00000   0.8250        1000                               9103
21      SMI  1.04477  82.6251          10                               9526
32   PALLAD  1.00000   9.1143         100                              10057
14     US20  1.00000   0.9651        1000                              10650
27      JPY  1.00000   0.0001    12500000                              11457
19      AEX  1.14905   4.6705         200                              11844
31     GOLD  1.00000  12.7812         100                              14104

Yes all on IB.

GAT

isotope1 · May 6, 2016

globalarbtrader said:
Clearer?

Yes, thank you! It's a bit of a leap thinking of 'volatility first' the whole time!

globalarbtrader · May 6, 2016

isotope1 said:
Yes, thank you! It's a bit of a leap thinking of 'volatility first' the whole time!

Yes, I still remember nearly 10 years ago when I first had the concept explained to me, and it certainly took a few attempts at working through it to "get" it.

GAT

tradrjoe · May 6, 2016

globalarbtrader said:

Code:

       code       fx      vol  pointvalue  expected_annual_risk_per_contract
11    SHATZ  1.14905   0.0270        1000                                343
6       KR3  0.00087   0.0575     1000000                                550
16      V2X  1.14905   0.6170         100                                782
36  EDOLLAR  1.00000   0.0482        2500                               1330
13      US2  1.00000   0.0609        2000                               1345
7      BOBL  1.14905   0.1420        1000                               1801
5      KR10  0.00087   0.2311     1000000                               2213
15      US5  1.00000   0.2087        1000                               2303
28      MXP  1.00000   0.0005      500000                               2915
0      CORN  1.00000   5.7743          50                               3186
1   LEANHOG  1.00000   0.8491         400                               3748
12     US10  1.00000   0.3528        1000                               3893
37  EUROSTX  1.14905  37.2879          10                               4728
4     WHEAT  1.00000   9.1172          50                               5030
3   SOYBEAN  1.00000   9.9404          50                               5484
26      GBP  1.00000   0.0081       62500                               5586
8       BTP  1.14905   0.4580        1000                               5807
2   LIVECOW  1.00000   1.3716         400                               6054
17      VIX  1.00000   0.5501        1000                               6070
35   GAS_US  1.00000   0.0558       10000                               6157
9      BUND  1.14905   0.4899        1000                               6212
10      OAT  1.14905   0.5047        1000                               6400
20      CAC  1.14905  51.8623          10                               6576
29      NZD  1.00000   0.0060      100000                               6662
23    SP500  1.00000  13.2490          50                               7310
25      EUR  1.00000   0.0055      125000                               7542
24      AUD  1.00000   0.0069      100000                               7662
18    KOSPI  0.00087   1.6344      500000                               7828
22   NASDAQ  1.00000  35.6690          20                               7872
33     PLAT  1.00000  14.2950          50                               7887
30   COPPER  1.00000   0.0300       25000                               8274
34  CRUDE_W  1.00000   0.8250        1000                               9103
21      SMI  1.04477  82.6251          10                               9526
32   PALLAD  1.00000   9.1143         100                              10057
14     US20  1.00000   0.9651        1000                              10650
27      JPY  1.00000   0.0001    12500000                              11457
19      AEX  1.14905   4.6705         200                              11844
31     GOLD  1.00000  12.7812         100                              14104

Yes all on IB.

GAT

Any particular reason why you don't have any ICE names on the list? (brent, sugar, coffee, etc). One would think they could further add to your diversification.

globalarbtrader · May 6, 2016

tradrjoe said:
Any particular reason why you don't have any ICE names on the list? (brent, sugar, coffee, etc). One would think they could further add to your diversification.

https://www.interactivebrokers.co.uk/en/index.php?f=14193

ICE futures data costs $120 a month.

Now, if I had a multiple of my current capital I'd probably swallow that. But I'm already at my limit for markets, so if I included ICE I'd have to take something else out.

GAT