Fully automated futures trading

globalarbtrader said:
Current positions

Code: 
       code contractid  position
1       AUD     202003        -2
17     CORN     202012        -5
3       EUR     202003        -3
2   EUROSTX     202003        -6
12   GAS_US     202005        -2
9       JPY     202003        -3
8   LEANHOG     202006        -1
10  LIVECOW     202010        -2
13      NZD     202003        -1
18  SOYBEAN     202011        -5
15      V2X     202004        -7
16      VIX     202004        -1
19    WHEAT     202012        -1

5      BOBL     202003         1
4      BUND     202003         1
6   EDOLLAR     202212         1
7   EDOLLAR     202303         7
0       KR3     202003         2
11      MXP     202003         7
14      OAT     202006         1

GAT
More...
Hi Rob, I just cross-checked my positions and directionally it looks the same, except that I'm flat KR3, which lead me to look at the signal weights., Turned out I have quite high allocation to carry for 3KTB: 66% and very low to EWMA just 8% (all just to one permutation 64_256), with 26% to breakout (only 160 and 320 look-back versions).
And I have a very similar situation with V2TX (10%-short, 57%-carry, 8%-EWMA(32_128 and 64_256), 23%-breakout (160 and 320)).
So very small EWMA allocation for both.
Weights for the other instruments are more evenly spread (~40%-carry, ~20%-EWMA, ~40%-Breakout). These weights came out of the bootstrap optimiser.
KR3 and V2TX contracts are relatively new (started in 1999?), and there's not too much price data for them, so I'm thinking maybe it was "wrong" to just automatically bootstrap-optimize their weights and better to use handcrafting instead (e.g. allocate 40%-carry, 30%-EWMA, 30%-breakout and then equal within each class except of fast permutations..) ?
 
wopr said:
I did cheat tho, I couldn't look at that MXP long anymore, so I sold that yesterday. In hindsight, it was a good call (at least for now) as it continues to go down, but I know I shouldn't be doing this.
More...
In my case the software closed the last remaining MXP contract. But yes, with a loss.
 
Kernfusion said:
Hi Rob, I just cross-checked my positions and directionally it looks the same, except that I'm flat KR3, which lead me to look at the signal weights., Turned out I have quite high allocation to carry for 3KTB: 66% and very low to EWMA just 8% (all just to one permutation 64_256), with 26% to breakout (only 160 and 320 look-back versions).
And I have a very similar situation with V2TX (10%-short, 57%-carry, 8%-EWMA(32_128 and 64_256), 23%-breakout (160 and 320)).
So very small EWMA allocation for both.
Weights for the other instruments are more evenly spread (~40%-carry, ~20%-EWMA, ~40%-Breakout). These weights came out of the bootstrap optimiser.
KR3 and V2TX contracts are relatively new (started in 1999?), and there's not too much price data for them, so I'm thinking maybe it was "wrong" to just automatically bootstrap-optimize their weights and better to use handcrafting instead (e.g. allocate 40%-carry, 30%-EWMA, 30%-breakout and then equal within each class except of fast permutations..) ?
More...

You optimised each instruments forecast weights individually?

Well, I wouldn't have done that

GAT
 
globalarbtrader said:
You optimised each instruments forecast weights individually?

Well, I wouldn't have done that

GAT
More...
I went back and rechecked everything, and it doesn't look like I've optimised signal-weights for every instrument individually.
My understanding is that "forecast_weight_estimat\pool_gross_returns = true" parameter in pysystemtrade makes the system optimise signal-weights on all returns together, so it gets one set of signal-weights for the whole system and then only adjusts them based on the costs for each individual instrument (e.g. some fast rules might get doped for an expensive instrument and their weights re-allocated to the rest of the forecasts)., so the final signal-weights will still be different between instruments but not significantly, at least signal-group allocation should be close (e.g. all carry permutations together should be say ~35% for each instrument).

And I'm running the optimisation mostly on the default parameters, only changed optimisation method to bootstrap and increased runs and length:
system.config.forecast_weight_estimate:
{'method': 'bootstrap', 'monte_runs': 300, 'bootstrap_length': 154, 'func': 'syscore.optimisation.GenericOptimiser', 'pool_gross_returns': True, 'equalise_gross': False, 'cost_multiplier': 2.0, 'apply_cost_weight': False, 'ceiling_cost_SR': 0.13, 'frequency': 'W', 'date_method': 'expanding', 'rollyears': 20, 'cleaning': True, 'equalise_SR': False, 'ann_target_SR': 0.5, 'equalise_vols': True, 'shrinkage_SR': 0.9, 'shrinkage_corr': 0.5, 'correlation_estimate': {'func': 'syscore.correlations.correlation_single_period', 'using_exponent': False, 'ew_lookback': 500, 'min_periods': 20, 'floor_at_zero': True}, 'mean_estimate': {'func': 'syscore.algos.mean_estimator', 'using_exponent': False, 'ew_lookback': 500, 'min_periods': 20}, 'vol_estimate': {'func': 'syscore.algos.vol_estimator', 'using_exponent': False, 'ew_lookback': 500, 'min_periods': 20}}
system.config.forecast_cost_estimates:
{'use_pooled_costs': False, 'use_pooled_turnover': True}

I noticed that "cost_multiplier=2.0 and apply_cost_weight=false" by default and in the original post they were set to 0\true, I changed these and re-run backtest but that only made signal-weights on V2X and KR3 even more extreme(only 3% to all EWMA and 65\75% to carry).
I'm now thinking maybe it's because I'm running backtest on only 14 instruments, and pooling 40 instruments makes a much better sample to fit forecast-weights..
Also, maybe when a rule permutation is dropped because of costs it's weight isn't allocated to the signals from it's group (as I would expect, maybe incorrectly)? (in my case it seems that when an EWMA permutation is dropped all it's weight goes into carry bucket, how does the system even know btw which signals belong together ?)..

I'm using
Version 0.26.0\20200205
 
Last edited:
Kernfusion said:
I went back and rechecked everything, and it doesn't look like I've optimised signal-weights for every instrument individually.
My understanding is that "forecast_weight_estimat\pool_gross_returns = true" parameter in pysystemtrade makes the system optimise signal-weights on all returns together, so it gets one set of signal-weights for the whole system and then only adjusts them based on the costs for each individual instrument (e.g. some fast rules might get doped for an expensive instrument and their weights re-allocated to the rest of the forecasts)., so the final signal-weights will still be different between instruments but not significantly, at least signal-group allocation should be close (e.g. all carry permutations together should be say ~35% for each instrument).

And I'm running the optimisation mostly on the default parameters, only changed optimisation method to bootstrap and increased runs and length:
system.config.forecast_weight_estimate:
{'method': 'bootstrap', 'monte_runs': 300, 'bootstrap_length': 154, 'func': 'syscore.optimisation.GenericOptimiser', 'pool_gross_returns': True, 'equalise_gross': False, 'cost_multiplier': 2.0, 'apply_cost_weight': False, 'ceiling_cost_SR': 0.13, 'frequency': 'W', 'date_method': 'expanding', 'rollyears': 20, 'cleaning': True, 'equalise_SR': False, 'ann_target_SR': 0.5, 'equalise_vols': True, 'shrinkage_SR': 0.9, 'shrinkage_corr': 0.5, 'correlation_estimate': {'func': 'syscore.correlations.correlation_single_period', 'using_exponent': False, 'ew_lookback': 500, 'min_periods': 20, 'floor_at_zero': True}, 'mean_estimate': {'func': 'syscore.algos.mean_estimator', 'using_exponent': False, 'ew_lookback': 500, 'min_periods': 20}, 'vol_estimate': {'func': 'syscore.algos.vol_estimator', 'using_exponent': False, 'ew_lookback': 500, 'min_periods': 20}}
system.config.forecast_cost_estimates:
{'use_pooled_costs': False, 'use_pooled_turnover': True}

I noticed that "cost_multiplier=2.0 and apply_cost_weight=false" by default and in the original post they were set to 0\true, I changed these and re-run backtest but that only made signal-weights on V2X and KR3 even more extreme(only 3% to all EWMA and 65\75% to carry).
I'm now thinking maybe it's because I'm running backtest on only 14 instruments, and pooling 40 instruments makes a much better sample to fit forecast-weights..
Also, maybe when a rule permutation is dropped because of costs it's weight isn't allocated to the signals from it's group (as I would expect, maybe incorrectly)? (in my case it seems that when an EWMA permutation is dropped all it's weight goes into carry bucket, how does the system even know btw which signals belong together ?)..

I'm using
Version 0.26.0\20200205
More...
This is a good example of the risk you're running when you use somebody else's software. It might do things differently from what you expected, or it might do things you didn't fully grasp.
Be careful of not falling into the trap "the computer says so, so it must be true". Keep it simple, use a set of parameters you yourself have determined and seem appropriate.
 
Well, the good thing is that this software is open-source :) so I dug into the code a little, and looks like the way it works is slightly different than I thought:
when we pool gross returns, don't pool costs and apply SR cost ceiling, the system first drops expensive rules for the instrument and then runs the optimisation on the returns of all the remaining rules but these rules are applied to the returns of ALL instruments but with costs of the CURRENT instrument spread to the returns of all the other instruments.
So I'm sort of doing a separate optimisation for every instrument, but with the returns of ALL instruments, and what comes from that instrument are only it's costs and the set of rules which are cheap enough. Does it still mean that I'm optimising each instruments forecast individually? - I don't think so. But the question still remains, why when fast EWMA and breakout rules are dropped all their weight goes to carry? (I mean I see now that there's no explicit logic in the code which will re-distribute weights of the dropped EWMAs to the remaining EWMAs, all the survived rules will simply be optimised together in the same bucket. But the result of that is that carry gets ~65% for V2Tx and KR3, this doesn't look very "correct" to me, I'll probably redistribute some of these weights back to the remaining EWMA and Breakout rules.. )
 
Kernfusion said:
Well, the good thing is that this software is open-source :) so I dug into the code a little, and looks like the way it works is slightly different than I thought:
when we pool gross returns, don't pool costs and apply SR cost ceiling, the system first drops expensive rules for the instrument and then runs the optimisation on the returns of all the remaining rules but these rules are applied to the returns of ALL instruments but with costs of the CURRENT instrument spread to the returns of all the other instruments.
So I'm sort of doing a separate optimisation for every instrument, but with the returns of ALL instruments, and what comes from that instrument are only it's costs and the set of rules which are cheap enough. Does it still mean that I'm optimising each instruments forecast individually? - I don't think so. But the question still remains, why when fast EWMA and breakout rules are dropped all their weight goes to carry? (I mean I see now that there's no explicit logic in the code which will re-distribute weights of the dropped EWMAs to the remaining EWMAs, all the survived rules will simply be optimised together in the same bucket. But the result of that is that carry gets ~65% for V2Tx and KR3, this doesn't look very "correct" to me, I'll probably redistribute some of these weights back to the remaining EWMA and Breakout rules.. )
More...

Sorry I jumped to concusions - I clearly don't know my own code :-) Anyway you've learned a lot about what is going on (this blog post might help also). The complexity of options for optimisation may be overkill, but people seem to like playing with them.

To get more intuitive results you might want to switch from bootstrapping to handcrafting, as I've done myself.

Code: 
system.config.forecast_weight_estimate['method']='handcraft'

GAT
 
Kernfusion said:
I went back and rechecked everything, and it doesn't look like I've optimised signal-weights for every instrument individually.
My understanding is that "forecast_weight_estimat\pool_gross_returns = true" parameter in pysystemtrade makes the system optimise signal-weights on all returns together, so it gets one set of signal-weights for the whole system and then only adjusts them based on the costs for each individual instrument (e.g. some fast rules might get doped for an expensive instrument and their weights re-allocated to the rest of the forecasts)., so the final signal-weights will still be different between instruments but not significantly, at least signal-group allocation should be close (e.g. all carry permutations together should be say ~35% for each instrument).

And I'm running the optimisation mostly on the default parameters, only changed optimisation method to bootstrap and increased runs and length:
system.config.forecast_weight_estimate:
{'method': 'bootstrap', 'monte_runs': 300, 'bootstrap_length': 154, 'func': 'syscore.optimisation.GenericOptimiser', 'pool_gross_returns': True, 'equalise_gross': False, 'cost_multiplier': 2.0, 'apply_cost_weight': False, 'ceiling_cost_SR': 0.13, 'frequency': 'W', 'date_method': 'expanding', 'rollyears': 20, 'cleaning': True, 'equalise_SR': False, 'ann_target_SR': 0.5, 'equalise_vols': True, 'shrinkage_SR': 0.9, 'shrinkage_corr': 0.5, 'correlation_estimate': {'func': 'syscore.correlations.correlation_single_period', 'using_exponent': False, 'ew_lookback': 500, 'min_periods': 20, 'floor_at_zero': True}, 'mean_estimate': {'func': 'syscore.algos.mean_estimator', 'using_exponent': False, 'ew_lookback': 500, 'min_periods': 20}, 'vol_estimate': {'func': 'syscore.algos.vol_estimator', 'using_exponent': False, 'ew_lookback': 500, 'min_periods': 20}}
system.config.forecast_cost_estimates:
{'use_pooled_costs': False, 'use_pooled_turnover': True}

I noticed that "cost_multiplier=2.0 and apply_cost_weight=false" by default and in the original post they were set to 0\true, I changed these and re-run backtest but that only made signal-weights on V2X and KR3 even more extreme(only 3% to all EWMA and 65\75% to carry).
I'm now thinking maybe it's because I'm running backtest on only 14 instruments, and pooling 40 instruments makes a much better sample to fit forecast-weights..
Also, maybe when a rule permutation is dropped because of costs it's weight isn't allocated to the signals from it's group (as I would expect, maybe incorrectly)? (in my case it seems that when an EWMA permutation is dropped all it's weight goes into carry bucket, how does the system even know btw which signals belong together ?)..

I'm using
Version 0.26.0\20200205
More...
I had the same situation in my own backtester where cost filtering was giving extreme weights to certain rules for instruments which could not be pooled. So over the weekend I applied a redistribution overlay fn which still allows the optimisation but then checks for extreme weights and rebalances those extreme weights. Sort of partial handcrafting. This didn’t yield an improvement in backtested SR, but will help me sleep better.
 
I wander why this is happening.. I mean when you say "for instruments which could not be pooled. " - the expensive instruments with fewer rules are still pooled (at least in pysystemtrade I think) - i.e. it still uses returns of ALL other instruments, just the set of rules drops from like 11 to like 6 and from these 6 the optimizer decides that carry rules are much better than normally for all instruments..
But yes, if we sort of already know what weights we want, why even run optimization if we still override it when we don't like the results ? :) Although one reason could be to double-check the original intuition (if the bootstrap gives the theiretically expected results it's reassuring).
And also from the purely automation perspective - setting 14×11+31×11 weight numbers on prod+test systems manually is a lot of work, easier to run an optimizer and then check the results..
I will try with handcrasting in pysystemtrade, although, it doesn't adjust the rules based on performance at all I think.. Although, trend and carry seem pretty-robust and it's pretty hard to ruin their performance with different weights..
 
Kernfusion said:
I wander why this is happening.. I mean when you say "for instruments which could not be pooled. " - the expensive instruments with fewer rules are still pooled (at least in pysystemtrade I think) - i.e. it still uses returns of ALL other instruments, just the set of rules drops from like 11 to like 6 and from these 6 the optimizer decides that carry rules are much better than normally for all instruments..
But yes, if we sort of already know what weights we want, why even run optimization if we still override it when we don't like the results ? :) Although one reason could be to double-check the original intuition (if the bootstrap gives the theiretically expected results it's reassuring).
And also from the purely automation perspective - setting 14×11+31×11 weight numbers on prod+test systems manually is a lot of work, easier to run an optimizer and then check the results..
I will try with handcrasting in pysystemtrade, although, it doesn't adjust the rules based on performance at all I think.. Although, trend and carry seem pretty-robust and it's pretty hard to ruin their performance with different weights..
More...

I am not setting any of the weights manually. Only algorithmically redistributing extreme weights if they crop up.
 
You must log in or register to reply here.
Back
Top