Fully automated futures trading

[karmicp]

Hi @globalarbtrader

I'm just discovering this thread, so sorry if that is a question that comes often:

Is it possible for me to run your trading system with about $20k and if not, what is the minimum balance you would recommend?

Lastly, does your system offer profiles, such as small account, medium account, large account, and then the system adapts to said profile to lower/increase the number of instruments or position sizing, depending on the current account profile?

Thank you in advance for your time

 

[globalarbtrader]

- I think all these weight variables are expressed in the "abstract instrument weight units", the same weight units as come out of the handcrafting process for static instrument weights, so we just need to calculate what amount of that abstract weight corresponds to a single contract in this instrument to be able to add the whole contracts when we try different portfolios (this variable contains per_contract_value)

No they're not abstract weight units, they are notional value as a % of capital. This is explained in the initial post which explored dynamic optimisation.

GAT

 

[globalarbtrader]

Hi @globalarbtrader

I'm just discovering this thread, so sorry if that is a question that comes often:

Is it possible for me to run your trading system with about $20k and if not, what is the minimum balance you would recommend?

Lastly, does your system offer profiles, such as small account, medium account, large account, and then the system adapts to said profile to lower/increase the number of instruments or position sizing, depending on the current account profile?

Thank you in advance for your time

I discuss this in great length in my third book, "Leveraged Trading". Briefly it depends on what you are trading; with futures you certainly couldn't trade with $20K.

There's also this series of posts on the subject:

Diversification and small account size
Optimising my way out of a small fund (part one): an idea for dynamic optimisation
Optimising my way out of a small fund (part two): testing dynamic optimisation (doesn't work)
Optimising my way out of a small fund (part three): static optimisation (works)
Optimising my way out of a small fund (part four): simple heuristic ranking: (doesn't work)
Optimising my way out of a small fund (part five): dynamic optimisation with greedy algo (it works!)

GAT

 

[Kernfusion]

• changing to a measure of volatility that uses both historic and current

• introducing an absolute position size limit which is effectively a maximum leverage limit of 100% on any instrument (discussed in this weeks TTU episode)

Listened to the last TTU, interesting stuff as usual,
but regarding limiting any single position to 100% of the system's base-capital by the position's notional, that seems to have a couple of issues:

- first, I checked and currently quite a few instruments I trade have notional close to 200k USD (NQ, PA, GE, ZT, GBL, OAT, BTP), even the US2, which is currently trading at ~110 and with multiplier 2,000 that's 220kUSD, so even with 500k capital you can only hold 2 of them, which doesn't seem to be "enough" according to your old post where you concluded that it's kind of nice to be able to hold at least 4 contracts each way. And for smaller accounts, e.g. 100-200k I couldn't even hold a single Eurodollar, which currently has notional of 245K (98.3725 * 2500multiplier)

- secondly, all these 2-3 contracts of a low-volatility instrument limited by notional-maximum will be entered very early on a weak signal, and then as the signal becomes stronger, the position will stay the same, making the system close to binary but also skewed towards "trading weak signals".

From this perspective, flooring the estimate of the current volatility seems like a better alternative, as the the positions will be entered more evenly with increasing forecast, if for example the vol-floor is chosen such that it will roughly translate to 4-5 contracts at max-signal.
Also in general this doesn't seem right that I can hold only 1 eurodollar or US2 contract with 200k capital, these are not huge contracts.. I mean they sort of are, by notional, but not by volatility and margin-requirements..

Regarding using 40\60 long\short term volatility instead of just short term, have you described it in the blog before of it's something new? I vaguely remember something about volatility in one of the earlier posts, but not sure..

 

[globalarbtrader]

Listened to the last TTU, interesting stuff as usual,
but regarding limiting any single position to 100% of the system's base-capital by the position's notional, that seems to have a couple of issues:

- first, I checked and currently quite a few instruments I trade have notional close to 200k USD (NQ, PA, GE, ZT, GBL, OAT, BTP), even the US2, which is currently trading at ~110 and with multiplier 2,000 that's 220kUSD, so even with 500k capital you can only hold 2 of them, which doesn't seem to be "enough" according to your old post where you concluded that it's kind of nice to be able to hold at least 4 contracts each way. And for smaller accounts, e.g. 100-200k I couldn't even hold a single Eurodollar, which currently has notional of 245K (98.3725 * 2500multiplier)

- secondly, all these 2-3 contracts of a low-volatility instrument limited by notional-maximum will be entered very early on a weak signal, and then as the signal becomes stronger, the position will stay the same, making the system close to binary but also skewed towards "trading weak signals".

From this perspective, flooring the estimate of the current volatility seems like a better alternative, as the the positions will be entered more evenly with increasing forecast, if for example the vol-floor is chosen such that it will roughly translate to 4-5 contracts at max-signal.
Also in general this doesn't seem right that I can hold only 1 eurodollar or US2 contract with 200k capital, these are not huge contracts.. I mean they sort of are, by notional, but not by volatility and margin-requirements..

Unfortunately we were supposed to talk about both this and my new dynamic optimisation method, but we ran out of time. This simple position limit only really makes sense if you're also using dynamic optimisation. With that, once you're 'full' in a low vol contract you will still be able to get the exposure you want elsewhere (eg shifting from US2 to US5, or spreading around a bet in short term bonds across all of them - taking the maximum position in US2, KR3, DE2 and so on).

Having said that I'd probably still quite like to have a maximum position limit even under my current system, but it would probably be more generous than 100% of notional. As a general rule, if you have a crude constraint, it should mostly be non binding i.e. quite generous.

Regarding using 40\60 long\short term volatility instead of just short term, have you described it in the blog before of it's something new? I vaguely remember something about volatility in one of the earlier posts, but not sure..

Was in the blog last year.

GAT

 

[Kernfusion]

I see., yeah, this dynamic optimization really seems like a great thing. Need to start implementing it in my system.. and onboarding new contracts, which is so much work, oh..

Also, because I do everything in real time, i.e. I react to every new price tick from every contract I trade, I'll try to incorporate this optimization in the real-time processing as well (I understand that it has small benefit compared to once-a-day processing\trading, but I already wend down this road, so no other way now), the difficulties will be computing speed, but I have a pretty-beefy server and wanted to buy a new toy from ebay anyway a used dual-socket server with 2 xeons is 1-2k$ and I need a stand-by backup anyway, and it's good that I can run this algo in parallel..
Another problem is now I'll need to get access to all other instruments' target weights when I process only one instrument's new tick, so far I didn't need it, but it should be doable by just storing the last target weight of every instrument in a common place(RAM and db between restarts) somehow....
So yeah, should be fun, will let you know how it went

 

[cholo]

Hi Rob,

Thank you (and Doug) for sharing the dynamic optimization post. It's really helpful as always.

I had started to implement something very similar in my system inspired but the previous posts, but Doug was faster! However, instead using the greedy algorithm, I have modeled the optimization problem so I can solve it with Mixed Integer Quadratic Programming solver. This way you can get the global optimal solution and it's much faster (~0.08 seconds for 25 instruments). I have an implementation in C#, but if you are interested I can translate it to python. Probably will need to do it anyway as I usually do lots of checks in jupyter notebooks.

I don't know if you have used this kind of solver in the past. The one I'm using is CPLEX, that has an academic and a free version that is enough to solve this problem up to 500 instruments. Probably is something that you don't want to depend on, specially in pysystemtrade. There are other solvers as well: for instance, XPRESS free edition should work up to 2500 instruments, but I haven't tried it.

If you want to know more I can send you a more detailed email or whatever is more convenient for you.

Thanks

 

[Kernfusion]

Hi Rob,

Thank you (and Doug) for sharing the dynamic optimization post. It's really helpful as always.

I had started to implement something very similar in my system inspired but the previous posts, but Doug was faster! However, instead using the greedy algorithm, I have modeled the optimization problem so I can solve it with Mixed Integer Quadratic Programming solver. This way you can get the global optimal solution and it's much faster (~0.08 seconds for 25 instruments). I have an implementation in C#, but if you are interested I can translate it to python. Probably will need to do it anyway as I usually do lots of checks in jupyter notebooks.

I don't know if you have used this kind of solver in the past. The one I'm using is CPLEX, that has an academic and a free version that is enough to solve this problem up to 500 instruments. Probably is something that you don't want to depend on, specially in pysystemtrade. There are other solvers as well: for instance, XPRESS free edition should work up to 2500 instruments, but I haven't tried it.

If you want to know more I can send you a more detailed email or whatever is more convenient for you.

Thanks

A C# solver is something I would be very interested in ., but your C# solution does depend on CPLEX right? I checked it out, seems like a bulky framework with it's own IDE, can't be simply added to the solution as a nuget package.. A Matlab\Python solution would be nice too.. But maybe you can describe how exactly the solution works? I guess it has to be a some kind of numeric derivative-free optimization, because the function isn't convex (has multiple local extremums), or you were able to find derivative of the function in analytical form ?
Or actually maybe these modern commercial solvers are so sophisticated that it's very non-trivial to repeat what they do from scratch using just linear algebra...

 

[globalarbtrader]

Hi Rob,

Thank you (and Doug) for sharing the dynamic optimization post. It's really helpful as always.

I had started to implement something very similar in my system inspired but the previous posts, but Doug was faster! However, instead using the greedy algorithm, I have modeled the optimization problem so I can solve it with Mixed Integer Quadratic Programming solver. This way you can get the global optimal solution and it's much faster (~0.08 seconds for 25 instruments). I have an implementation in C#, but if you are interested I can translate it to python. Probably will need to do it anyway as I usually do lots of checks in jupyter notebooks.

I don't know if you have used this kind of solver in the past. The one I'm using is CPLEX, that has an academic and a free version that is enough to solve this problem up to 500 instruments. Probably is something that you don't want to depend on, specially in pysystemtrade. There are other solvers as well: for instance, XPRESS free edition should work up to 2500 instruments, but I haven't tried it.

If you want to know more I can send you a more detailed email or whatever is more convenient for you.

Thanks

A C# solver is something I would be very interested in ., but your C# solution does depend on CPLEX right? I checked it out, seems like a bulky framework with it's own IDE, can't be simply added to the solution as a nuget package.. A Matlab\Python solution would be nice too.. But maybe you can describe how exactly the solution works? I guess it has to be a some kind of numeric derivative-free optimization, because the function isn't convex (has multiple local extremums), or you were able to find derivative of the function in analytical form ?
Or actually maybe these modern commercial solvers are so sophisticated that it's very non-trivial to repeat what they do from scratch using just linear algebra...

I haven't used it but a quick google suggests there might be a solution; https://www.cvxpy.org/

https://www.cvxpy.org/examples/basic/mixed_integer_quadratic_program.html

GAT

 

[djames]

Hi Rob,
...I have an implementation in C#, but if you are interested I can translate it to python. Probably will need to do it anyway as I usually do lots of checks in jupyter notebooks.
...
I don't know if you have used this kind of solver in the past. The one I'm using is CPLEX, that has an academic and a free version that is enough to solve this problem up to 500 instruments. Probably is something that you don't want to depend on, specially in pysystemtrade. There are other solvers as well: for instance, XPRESS free edition should work up to 2500 instruments, but I haven't tried it.
...
Thanks

Hi, I'd be interested in the Python version, it's been on my todo list to play with cvxpy (suports CPLEX and others).
Also, there is a nice multi-horizon optimiser from the same Stanford team as cvxpy: https://github.com/cvxgrp/cvxportfolio
paper: https://web.stanford.edu/~boyd/papers/cvx_portfolio.html
would be interesting to see if we could incorporate information from forecasts at future times (than t+1).
Cheers