Fully automated futures trading

globalarbtrader · Sep 13, 2017

isotope1 said:
I appreciate this is off topic, but I learnt pretty much all I know from this thread, so thought I'd ask here.

I trying to come up with a systematic argument to answer this question.

I own a London flat
* Market value £480k
* £250k mortgage.
* The net income is 7.2k after taxes, costs and interest.
* I have capital gains relief on the flat until 2022.

I also have £370k in a trend following futures account, running at 25% vol with after tax sharpe of 0.65. If I sell the flat, I will allocate the cash (230k) to trend following.

Arguments to sell flat:

* I hate dealing with problems. Both my neighbours hate me because of leaks which I cannot fix.
* Law of active management doesn't apply
* Diversification doesn't apply
* Unknown correlation to trend following (I would guess highly correlated with ftse)
* Illiquid
* Positive skew means potential hidden big loss in future

Arguments to keep flat:

* Positive skew investment, makes money most years
* Diversified from trend following, depending on unknown correlation
* If I screw up trend following, at least I have a backup
* No tax drag, cgt is deferred until flat is sold rather than paid out of capital annually
* Fundamentals that made this investment work (low interest rates) have not changed.

I just have no idea how to systematically evaluate this. How does one factor in property into an investment portfolio?

For your interest here's my own personal current asset allocation, including my house, in nominal terms:

Bonds 32%
Equities 44%
Property (mostly house, but also including about 2.5% in property ETFs) 32%
Cash 9%
Debt (including mortgage on house) -16%

And here's my estimated breakdown of my risk:

Bonds 16%
Equities 36%
Futures trading 21%
Property 27%

GAT

djames · Sep 13, 2017

Hi GAT,
I'm trying to understand how you rolled the V2X data (in order to understand the process in general)

looking at the recent file in pysystemtrade around the roll date of 2016-10-17

you have price of:

DATETIME,PRICE
2016-10-10,23.15
2016-10-11,23.4
2016-10-12,23.5
2016-10-13,23.8
2016-10-14,23.25
2016-10-17,23.4
2016-10-18,22.9

and carry of:
DATETIME,PRICE,CARRY,CARRY_CONTRACT,PRICE_CONTRACT
2016-10-13,24.2,21.4,201610,201611
2016-10-14,23.65,20.05,201610,201611
2016-10-17,23.4,20.5,201611,201612
2016-10-18,22.9,23.25,201611,201612

i'm struggling to see how you got the price 23.25 on 14th - are you averaging a window around the roll date to smooth out jumps? The shift you apply is 0.4 but how is this calculated?

lining up the 'current' contract 201612 and 'previous' contract 201611 gives:

current previous diff
2016-10-13 00:00:00+00:00 NaN 24.20 NaN
2016-10-14 00:00:00+00:00 NaN 23.65 NaN
2016-10-15 00:00:00+00:00 NaN 23.65 NaN
2016-10-16 00:00:00+00:00 NaN 23.65 NaN
2016-10-17 00:00:00+00:00 23.400 20.50 2.900
2016-10-18 00:00:00+00:00 22.900 23.25 -0.350
2016-10-19 00:00:00+00:00 22.450 22.60 -0.150
2016-10-20 00:00:00+00:00 22.350 22.00 0.350
2016-10-21 00:00:00+00:00 21.950 21.50 0.450
2016-10-22 00:00:00+00:00 21.950 21.50 0.450
2016-10-23 00:00:00+00:00 21.950 21.50 0.450

can you recall how the algorithm is smoothing out the diff column?

I hope you don't mind helping, 'tis much appreciated.

MarkBrown · Sep 14, 2017

have you tried to just be really good trading one market?

HobbyTrading · Sep 14, 2017

@globalarbtrader I am still contemplating ways of protecting my account from positions which start to move against me. The current system determines a position size based on (a) the value volatility versus the risk appetite and (b) the forecast, based on a combination of rules. The profit/loss of the open position is not directly included in any of these. Only indirectly: (a) if the loss increases the account value decreases (unless profits in other instruments offset this), thus the risk appetite becomes less as it is a percentage of the account size and (b) the forecast is (likely) going to change in the opposite direction of when the position was opened. I have the impression that these two indirect effects are rather slow and thus allow a position to rack up a substantial loss before being reduced/closed.
Would it make sense to add a parameter to the system, being the current loss versus the risk appetite, and have that influence the position size? This parameter would only be used if there is an open position and this open position has a loss. Positions with a profit would not be affected, and it would not influence the timing of opening a position in an instrument.
I don't have the right infrastructure to backtest such an idea and was wondering whether you have ever tested something like this in the past?

globalarbtrader · Sep 14, 2017

djames said:
Hi GAT,
I'm trying to understand how you rolled the V2X data (in order to understand the process in general)

looking at the recent file in pysystemtrade around the roll date of 2016-10-17

you have price of:

DATETIME,PRICE
2016-10-10,23.15
2016-10-11,23.4
2016-10-12,23.5
2016-10-13,23.8
2016-10-14,23.25
2016-10-17,23.4
2016-10-18,22.9

and carry of:
DATETIME,PRICE,CARRY,CARRY_CONTRACT,PRICE_CONTRACT
2016-10-13,24.2,21.4,201610,201611
2016-10-14,23.65,20.05,201610,201611
2016-10-17,23.4,20.5,201611,201612
2016-10-18,22.9,23.25,201611,201612

i'm struggling to see how you got the price 23.25 on 14th - are you averaging a window around the roll date to smooth out jumps? The shift you apply is 0.4 but how is this calculated?

lining up the 'current' contract 201612 and 'previous' contract 201611 gives:

current previous diff
2016-10-13 00:00:00+00:00 NaN 24.20 NaN
2016-10-14 00:00:00+00:00 NaN 23.65 NaN
2016-10-15 00:00:00+00:00 NaN 23.65 NaN
2016-10-16 00:00:00+00:00 NaN 23.65 NaN
2016-10-17 00:00:00+00:00 23.400 20.50 2.900
2016-10-18 00:00:00+00:00 22.900 23.25 -0.350
2016-10-19 00:00:00+00:00 22.450 22.60 -0.150
2016-10-20 00:00:00+00:00 22.350 22.00 0.350
2016-10-21 00:00:00+00:00 21.950 21.50 0.450
2016-10-22 00:00:00+00:00 21.950 21.50 0.450
2016-10-23 00:00:00+00:00 21.950 21.50 0.450

can you recall how the algorithm is smoothing out the diff column?

I hope you don't mind helping, 'tis much appreciated.

Part of the problem is you are looking at daily data which has been resampled from intraday.

Code:

                        ADJ   PRICE  FORWARD PRICE_CONTRACT CARRY_CONTRACT
2016-10-14 15:53:10  34.425  23.475      NaN         201611         201610
2016-10-14 23:00:00  34.600  23.650    23.15         201611         201610
2016-10-17 08:00:07  34.500  23.550      NaN         201611         201610
2016-10-17 09:00:31  34.525  23.575      NaN         201611         201610
2016-10-17 10:00:50  34.650  23.700      NaN         201611         201610
2016-10-17 11:01:17  34.650  23.700      NaN         201611         201610
2016-10-17 12:07:10  34.550  23.600      NaN         201611         201610
2016-10-17 13:27:47  34.550  23.600      NaN         201611         201610
2016-10-17 15:03:13  34.625  23.675      NaN         201611         201610
2016-10-17 23:00:00  34.750  23.800    23.40         201611         201610  *
2016-10-17 23:00:01  34.750  23.400      NaN         201612         201611
2016-10-18 08:00:03  34.525  23.175      NaN         201612         201611
2016-10-18 09:00:25  34.375  23.025      NaN         201612         201611

The asterixed line makes it clear what's going on. The roll is applied at the end of day price point, which by convention has a timestamp of 23:00, with prices 23.8 in the old contract and 23.4 in the new (forward) contract. I apply a fake timestamped price one second after this. Now the roll has happened. The old forward is now the new price contract. The adjusted price hasn't changed, since apart from the roll the market hasn't moved. You can't see it here but I would have changed all previous adjusted prices, applying a downward shift of 0.4.

GAT

globalarbtrader · Sep 14, 2017

HobbyTrading said:
@globalarbtrader I am still contemplating ways of protecting my account from positions which start to move against me. The current system determines a position size based on (a) the value volatility versus the risk appetite and (b) the forecast, based on a combination of rules. The profit/loss of the open position is not directly included in any of these. Only indirectly: (a) if the loss increases the account value decreases (unless profits in other instruments offset this), thus the risk appetite becomes less as it is a percentage of the account size and (b) the forecast is (likely) going to change in the opposite direction of when the position was opened. I have the impression that these two indirect effects are rather slow and thus allow a position to rack up a substantial loss before being reduced/closed.
Would it make sense to add a parameter to the system, being the current loss versus the risk appetite, and have that influence the position size? This parameter would only be used if there is an open position and this open position has a loss. Positions with a profit would not be affected, and it would not influence the timing of opening a position in an instrument.
I don't have the right infrastructure to backtest such an idea and was wondering whether you have ever tested something like this in the past?

Yes I've thought about it before, and dismissed it, because it's a terrible idea.

Firstly the market doesn't know you're making a loss. The future path of prices is unaffected by whether any individual trader is up or down. Which is why in a well designed system the forecast for future prices is unaffected by whether the system has made losses or gains. If you're trend following then your forecast should only be based on what the price has done recently. If you start behaving differently then you'll reduce the profitability of your system.

Secondly, you'll trade faster when you're losing money. Which means your overall trading costs will increase. Hopefully your system has been designed already at a given level of trading costs. Doing this means you'll either pay too much, or have to slow down your system further. Which means you'll be slower to get out of positions on which you've made a profit, but where the price has turned against you.

Thirdly, you're introducing path dependency into your system. This is a very bad idea.

Would this insane idea work in a backtest? Possibly; the change is relatively small so there is every chance that you'll get a statistically insignificant improvement even if the true expectation is that it won't work. But to do it would go completely against how I think about the markets.

Are there alternatives? Well you could trade faster. Then you'll suffer the pain for shorter periods of time. But you'll probably end up making less profits though your broker will be richer (unless your system trades way too slowly currently).

Let me give you advice - don't look at the p&l on individual positions, or at least do so as irregularly as possible. I can tell you right now roughly how well / badly my overall system is doing because I look at daily p& (it would be irresponsible not to). I don't know off hand what the overall drawdown is. But: I have almost no ideas what my positions are, and I certainly don't know if they're profitable - I haven't even written the code to check! Once a year I will see how they did over the year and I'll pick out some interesting markets to see what happened - but that is all.

If you can't follow this advice then frankly you shouldn't be trading - you don't have the self control (it goes without saying you don't have the self control to be a discretionary trader). Buy my second book and sort yourself out a nice portfolio of long only investments. In part four I advise annual rebalancing... :-)

GAT

djames · Sep 14, 2017

globalarbtrader said:
Part of the problem is you are looking at daily data which has been resampled from intraday.

GAT

ah, right that clarifies things (also re-read your blog post https://qoppac.blogspot.co.uk/2015/05/systems-building-futures-rolling.html)

I'm attempting to construct your price file for V2X from the carry file, not ideal but will be self contained within pysystemtrade, so i'm moving backward through the carry file. It's looking like the algorithm for stitching will have a few heuristics when calculating the adjustment price - in the case of this roll:

Code:

                          PRICE  CARRY   diff
2016-10-15     NaN  23.65    NaN
2016-10-16     NaN  23.65    NaN
2016-10-17  23.400  20.50  2.900
2016-10-18  22.900  23.25 -0.350
2016-10-19  22.450  22.60 -0.150

a heuristic could be if the diff on the roll date is larger than a certain percentage then use the next diff price to give the adjustment e.g -0.35, which is within bounds of your -0.4 adjustment. Does that seem sensible?

globalarbtrader · Sep 14, 2017

djames said:
ah, right that clarifies things (also re-read your blog post https://qoppac.blogspot.co.uk/2015/05/systems-building-futures-rolling.html)

I'm attempting to construct your price file for V2X from the carry file, not ideal but will be self contained within pysystemtrade, so i'm moving backward through the carry file. It's looking like the algorithm for stitching will have a few heuristics when calculating the adjustment price - in the case of this roll:

Code:

PRICE CARRY diff 2016-10-15 NaN 23.65 NaN 2016-10-16 NaN 23.65 NaN 2016-10-17 23.400 20.50 2.900 2016-10-18 22.900 23.25 -0.350 2016-10-19 22.450 22.60 -0.150

a heuristic could be if the diff on the roll date is larger than a certain percentage then use the next diff price to give the adjustment e.g -0.35, which is within bounds of your -0.4 adjustment. Does that seem sensible?

It sounds very very complicated. The "correct" value of the % diff will vary wildly from instrument to instrument and over time. If you believe that both price and carry are tradeable prices then why should you prefer one value to another?

I wouldn't do this.

Alternatively you could try taking an average value of the diff over a few days (equivalent to assuming you rolled your positions gradually). Again, personally, I wouldn't waste my time on coding this up.

GAT

PS you'll struggle to reproduce my results for every market since in many cases the "forward" contract I roll to is different from the "carry". And indeed this is the case for V2X.

HobbyTrading · Sep 14, 2017

globalarbtrader said:
Yes I've thought about it before, and dismissed it, because it's a terrible idea.

Firstly the market doesn't know you're making a loss. The future path of prices is unaffected by whether any individual trader is up or down. Which is why in a well designed system the forecast for future prices is unaffected by whether the system has made losses or gains. If you're trend following then your forecast should only be based on what the price has done recently. If you start behaving differently then you'll reduce the profitability of your system.

Secondly, you'll trade faster when you're losing money. Which means your overall trading costs will increase. Hopefully your system has been designed already at a given level of trading costs. Doing this means you'll either pay too much, or have to slow down your system further. Which means you'll be slower to get out of positions on which you've made a profit, but where the price has turned against you.

Thirdly, you're introducing path dependency into your system. This is a very bad idea.

Would this insane idea work in a backtest? Possibly; the change is relatively small so there is every chance that you'll get a statistically insignificant improvement even if the true expectation is that it won't work. But to do it would go completely against how I think about the markets.

Are there alternatives? Well you could trade faster. Then you'll suffer the pain for shorter periods of time. But you'll probably end up making less profits though your broker will be richer (unless your system trades way too slowly currently).

Let me give you advice - don't look at the p&l on individual positions, or at least do so as irregularly as possible. I can tell you right now roughly how well / badly my overall system is doing because I look at daily p& (it would be irresponsible not to). I don't know off hand what the overall drawdown is. But: I have almost no ideas what my positions are, and I certainly don't know if they're profitable - I haven't even written the code to check! Once a year I will see how they did over the year and I'll pick out some interesting markets to see what happened - but that is all.

If you can't follow this advice then frankly you shouldn't be trading - you don't have the self control (it goes without saying you don't have the self control to be a discretionary trader). Buy my second book and sort yourself out a nice portfolio of long only investments. In part four I advise annual rebalancing...

GAT

Thank you for explaining why you consider this a bad idea.

isotope1 · Sep 14, 2017

globalarbtrader said:
I'd sell the flat. I'd then use the proceeds in such a way to give me an asset allocation that looks something like this:

Trend following: I'd suggest no more than 20% of your wealth should be in this if you're not working; if you're working you could probably go a bit higher, but more than a third would make me uncomfortable.

Property: I'd suggest a 10% to 30% allocation is reasonable here; again no more than 50%. But that should factor in your other property (eg your own house or flat if you have one). If necessary buy ETFs to make up the difference.

This is what I thought too. The flat is a nightmare anyway, so I'm really looking for an excuse to sell it. I don't live off my investments at all.

I'm surprised you mentioned only 20-30% in a TF strategy. Surely the expected Sharpe of a TF strategy exceeds that of all other investments (property, ETFs, equities), so it makes the most 'systematic' sense to allocate heavily here. Or is that overconfidence?