Yeah it ended when I asked them to stop sharing my screen. They probably pressed the wrong button
GAT
Fully automated futures trading
- Thread starter globalarbtrader
- Start date
Yeah it ended when I asked them to stop sharing my screen. They probably pressed the wrong button
GAT
I felt sorry that you were not able to show the material you had prepared. Only one slide was used but it was obvious that you had prepared at least some 20 others as well. Which turned the session more into an interview than a presentation.Yeah it ended when I asked them to stop sharing my screen. They probably pressed the wrong button
GAT
Oh no it was supposed to be an interview! I just had that slide ready to illustrate a particular point. I was never planning to use more than one slide.
GAT
I see. Thank you for that clarification.
This is really cool, thanks Rob, I thought you gave up on the idea, but I guess the researcher's curiosity took over
So as I understand, you're applying this optimization at the very end (but before the risk-overlay logic, which serves as the final fail-safe), when you already have the final unrounded weights, which take into account static handcrafted weights, current forecasts and the position-sizing logic (based on system's base-capital, and volatility target).,
So generally, if you're trading lots of instruments on a small capital, almost all of the these unrounded-weights will be tiny, not even enough for a single contract.
Then you take an EMPTY portfolio and start adding\removing one contract at a time of each instrument separately, remembering the smallest sigma of the resulting gap-portfolio (between the current-search attempt and unrounded ideal portfolios), and you break out of the outer loop when this next best gap-portfolio-sigma isn't smaller than the current smallest. So the current positions your system is holding don't play any role in this optimization process, you're "narrowing the volatility gap" between empty and unrounded-ideal portfolios?
A couple more questions:
- when you calculate direction, if the optimal-unrounded weight of an instrument is zero, direction will also be zero, therefore you wouldn't attempt to put any positions in these instruments when doing greedy-search, right?
- how do you limit maximum positions per instrument for the greedy-algo to try? There's "maximum_position_weights" variable in the code but I can't find where it's assigned. Can it be for example some static system settings for the max number of contracts in any instrument (e.g., 20), or maybe per instrument, like no more than 15 Eurodollar, no more than 5 copper and so on?
- in general, does this mean that there's now no limit on how many instruments the system trades(monitors), i.e., even with say $50k you can add 200 instruments and it should not negatively affect performance compared to having only 20 contracts? (I guess there will be diminishing returns at some point though...)
This is really cool, thanks Rob, I thought you gave up on the idea, but I guess the researcher's curiosity took over
Actually all credit to Doug here, I really had given up...
Broadly yes.
Actually I'm still toying with the idea of whether I need the risk overlay logic, for a couple of reasons:
- changing to a measure of volatility that uses both historic and current
- introducing an absolute position size limit which is effectively a maximum leverage limit of 100% on any instrument (discussed in this weeks TTU episode)
If I used it the risk overlay logic would go before this final step. I'm constructing the production code now, and basically the optimisation is run as a seperate process at the point when we decide which orders to issue to the execution components. The overnight backtest process basically produces a load of (unrounded) optimal positions, that would
Correct, with the possible exception of instruments with very small contract size and/or very low volatility.
To give you some idea here's the current set of optimal positions with my current capital (about £380K) and 103 instruments:
Code:
{'AEX': 0.007, 'AUD': 0.008, 'BBCOMM': 1.552, 'BITCOIN': -0.118, 'BOBL': -0.773, 'BONO': 0.496, 'BRE': -0.110, 'BRENT-LAST': 0.145, 'BTP': -0.047, 'BTP3': -0.103, 'BUND': 0.263, 'BUXL': 0.065, 'CAC': -0.008, 'CAD': -0.384, 'CHF': -0.381, 'CNH': -0.473, 'COPPER': -0.079, 'CORN': -0.070, 'CRUDE_W_mini': 0.561, 'DAX': 0.254, 'DJSTX-SMALL': 0.324, 'DOW': -0.060, 'EDOLLAR': 1.347, 'EU-AUTO': -0.393, 'EU-BANKS': 0.414, 'EU-BASIC': -0.591, 'EU-CHEM': -0.383, 'EU-DIV30': -0.181, 'EU-FOOD': -0.357, 'EU-HEALTH': -0.186, 'EU-INSURE': -0.244, 'EU-OIL': 1.282, 'EU-RETAIL': -0.615, 'EU-TECH': 0.318, 'EU-TRAVEL': 0.361, 'EU-UTILS': 0.058, 'EUR': -0.185, 'EURO600': 0.120, 'EUROSTX': 0.002, 'FEEDCOW': -0.076, 'FTSECHINAA': -1.833, 'FTSECHINAH': 0.006, 'FTSETAIWAN': 0.351, 'GAS-LAST': 0.379, 'GASOILINE': 0.210, 'GAS_US_mini': 0.642, 'GBP': -0.986, 'GBPEUR': -1.037, 'GOLD_micro': 0.392, 'HEATOIL': 0.139, 'IRON': 0.084, 'JGB': -0.354, 'JPY': 0.189, 'KOSDAQ': 3.554, 'KOSPI_mini': -0.720, 'KR10': -2.159, 'KR3': 12.154, 'KRWUSD': -0.585, 'LEANHOG': 0.065, 'LIVECOW': -0.206, 'MILK': -0.833, 'MSCISING': -0.148, 'MUMMY': 1.283, 'MXP': 0.378, 'NASDAQ_micro': 1.097, 'NIFTY': 1.372, 'NIKKEI': 1.548, 'NIKKEI400': 3.713, 'NOK': 0.060, 'NZD': 0.120, 'OAT': -0.876, 'OATIES': 0.167, 'PALLAD': -0.032, 'PLAT': -0.135, 'R1000': 0.146, 'REDWHEAT': -0.377, 'RICE': -0.269, 'RUR': 0.174, 'RUSSELL': -0.347, 'SEK': -0.028, 'SHATZ': -3.067, 'SILVER': -0.246, 'SMI': -0.004, 'SOYBEAN': 0.016, 'SOYMEAL': -0.234, 'SOYOIL': 0.046, 'SP400': -0.028, 'SP500_micro': 0.215, 'TOPIX': 1.855, 'US-REALESTATE': -0.012, 'US10': 0.361, 'US10U': 1.115, 'US2': 10.687, 'US20': 0.334, 'US3': 2.502, 'US30': 0.266, 'US5': 0.682, 'USIRS10': -0.696, 'USIRS5': -0.261, 'V2X': -2.008, 'VIX': -0.516, 'WHEAT': -0.640, 'YENEUR': -0.785}There are a few single contracts in there, but not many!
That's right, except there is a cost penalty which is calculated using current positions.
Actually I effectively put in a direction of 1 (this line). But it's irrelevant anyway, since there is no possible situation in which the unrounded optimal position would be zero (catastrophic data failure aside).
So again this is in the production code only, since I don't set maximum positions in my backtest. But basically I set maximum positions per instrument (or you can actually do it per instrument / strategy combination, but the most conservative of those settings is used). You can set these manually, but I use an 'auto population' function which is here.
Yes you could even do something very silly like run 200 instruments with $10K of capital. You'll still (in expectation) do better than trading a single randomly chosen future (not that there are many ).
And yes there will be diminishing returns: going from 1 to 2 instruments increases expected SR by 40% (from ~0.4 to 0.56), going from 35 to 100 (as I expect to do with this change) produces a similar increase (from about 1.0 to 1.4), and not all of that will be captured by the dynamic optimisation (YMV depending on your account size).
GAT
PS in fact this has given me an idea for a little project where I run the optimisation daily with different levels of capital producing 'CTA tracking portfolios' suitable for different account sizes, and then auto publish these on some website. I can also autopublish other interesting information like my own actual positions, risk, and so on.
The problem I've had is trying to find an easy way of doing the autopublishing without having to turn myself into a full stack developer with an EC2 account. I'm looking for a simple API that will allow me to do this:
Code:
import somelib
somelib.publish(very_long_string_of_simple_html, credentials = my_credentials)... any ideas welcome.
GAT
@Kernfusion & @globalarbtrader I appreciate your discussion here about Rob's latest blog entry. This further clarifies the approach as described in the blog. I understand that the so-called "gap-portfolio" is used to reduce the difference in volatility of the "rounded portfolio" and "unrounded portfolio". Does it matter how the volatility of the three portfolios is calculated?
The original code calculates the volatility of each instrument by taking 25 day historical values. Should the portfolio volatility then also be calculated based on a 25 days lookback period? Maybe not, but it might be the most convenient as the already calculated instrument volatility can then directly be used to determine the gap-portfolio?
The original code calculates the volatility of each instrument by taking 25 day historical values. Should the portfolio volatility then also be calculated based on a 25 days lookback period? Maybe not, but it might be the most convenient as the already calculated instrument volatility can then directly be used to determine the gap-portfolio?
Actually we dont' calculate the variance of all three portfolios, we just measure the variance of the 'gap portfolio'.
A simple example if the optimal was in two instruments (-0.5, 1.5) and the rounded portfolio we were checking was (-1.0, 1.0) then the gap portfolio would be (-0.5, -0.5) and we'd measure the variance of that third portfolio.
Yes I calculate the covariance matrix using the underlying returns of each instrument. I calculate the correlation and the standard deviation seperately; the correlation is configured here (it has a 6 month lookback), and the volatility using my 'standard' 25 day lookback (not exactly that as it's exponentiall weighted but close enough) - I don't see the point in having lots of different volatility estimates for different purposes..
GAT
Rob, I think S3 would be great for this. You can host a static website on it easily, you even get some benefits like versioning for free, you don't have to manage a box, you can also stick your custom domain in front of it if you want. Also, API is almost as simple as in your example, few lines of code does the job.
Knowing your tendencies to keep things simple, static website would work great.
If you're interested in something like that, I'd be happy to help set that up.
Docs are here https://docs.aws.amazon.com/AmazonS3/latest/userguide/WebsiteHosting.html
Is it free though? Can I get something like a single 1MB web page up all the time at no cost?
GAT