Fully automated futures trading

[wopr]

Just glancing at your risk report, compared to mine, you're running slightly more risk but as you say with a lot more positions.

Total 5.8 (lower than yours but not by much)
Sum of abs(notional exposure % capital) 75.4 (much lower)
Sum of abs(annualised risk % capital) 13.8 (about half)
Net sum of annualised risk % capital 5.8

Basically you have a hedged book with lots of longs and shorts and a fair amount of leverage, balancing out to not much risk, whilst I have more of an outright bet on, albeit not a very big one.

This might be a consequence of your instrument choice. To take an example, suppose someone invested just in equities and currently overall equity signals were weak; with some markets short and some long. Their book and risk would look very much like yours. Whereas, take someone with a spread of asset classes, but where only one or two asset classes had a strong signal. Their book would look like mine.

It also looks like you might have fewer problems with discretization than me, eithier because you have fewer markets or more capital, or more markets with small tick size.

Rob

Makes sense.
I was looking at your strategy reports while I was debugging this, and found some interesting things so will mention here. Your forecasts for gold and silver are both decently negative (actually, subsystem position for gold is -52!) but all DO did with that is short just 2 micro gold contracts. I would have expected overall more short exposure to metals from that.

Also, I'm sure I'm misreading this, but your strategy reports are saying iron has small negative forecast and subsystem position for the last few days, but risk report says you're long. If I recall correctly, that shouldn't be possible in the DO land.

 

[VMatthijs]

Whever I've tried it in the past I've found it fairly painful.

Of course things are always improving.

Rob

For what it's worth, I'm very happy with running pysystemtrade in the cloud. I'm using https://hetzner.com/ . They're pretty cheap, user friendly and reliable. Their servers are close enough to the Netherlands to be very usable with low latency. Not sure about the UK. Probably, it's better to find a provider that's actually physically based there.

 

[wopr]

What makes you think so? I just recently migrated VPS and getting IB Gateway to run on it was a breeze. There's guides online for it as well (it's gotten even simpler since that was written).

I am running IB Gateway in the cloud, and would agree it's not that complex, but this guide doesn't simplify things a lot (if that was the goal).
None of these services (x11vnc, xvfb, IB Controller) will be started up on their own in case of a reboot or if they error out, and reboots on cloud providers do happen. Also, you should update packages regularly and reboot.
Also, I wouldn't feel ok leaving open ports like that, especially VNC. It's a standard port and any scanner will detect it. From there, anybody can see your account number, which port is the service running on, and my favorite, add their IP to the list of valid IPs, and go to town Next thing you know, you wake up and your strategic exposure to March 2025 Eurodollars is all GME stock now.

For security, the simplest thing I've found so far is to use a VPN, I use tailscale (I'm not affiliated in any way). It's free and plug and play solution, point-to-point VPN, does authn/authz and encryption.

 

[globalarbtrader]

Makes sense.
I was looking at your strategy reports while I was debugging this, and found some interesting things so will mention here. Your forecasts for gold and silver are both decently negative (actually, subsystem position for gold is -52!) but all DO did with that is short just 2 micro gold contracts. I would have expected overall more short exposure to metals from that.

Also, I'm sure I'm misreading this, but your strategy reports are saying iron has small negative forecast and subsystem position for the last few days, but risk report says you're long. If I recall correctly, that shouldn't be possible in the DO land.

It's not possible unless the forecast flipped sign but I haven't traded since that happened due to buffering OR a trade limit is preventing the trade. I think it's the former.

Rob

 

[d08]

I am running IB Gateway in the cloud, and would agree it's not that complex, but this guide doesn't simplify things a lot (if that was the goal).
None of these services (x11vnc, xvfb, IB Controller) will be started up on their own in case of a reboot or if they error out, and reboots on cloud providers do happen. Also, you should update packages regularly and reboot.
Also, I wouldn't feel ok leaving open ports like that, especially VNC. It's a standard port and any scanner will detect it. From there, anybody can see your account number, which port is the service running on, and my favorite, add their IP to the list of valid IPs, and go to town Next thing you know, you wake up and your strategic exposure to March 2025 Eurodollars is all GME stock now.

For security, the simplest thing I've found so far is to use a VPN, I use tailscale (I'm not affiliated in any way). It's free and plug and play solution, point-to-point VPN, does authn/authz and encryption.

Never had a reboot in some 4-5 years of VPS usage but yes, it could happen. I rationalise it by understanding that a reboot on my home machine is much more likely and the connection is far less stable. This depends on where you live and so forth but generally, uptime for cloud services is far above that of any home connection.

I keep my VNC port open but the service off when not using it. Very rarely is it needed as everything is CLI anyway. Not sure how anyone would just bypass SSH security when using VNC (I believe that wasn't in the tutorial but can be easily implemented). Or you might use the less secure option of using a password.
Gateway by default only accepts localhost and that's how it should be.
SSH security is not bulletproof but it's pretty good and used globally when dealing with any sensitive data.

Many run Windows on their trading machines, that itself is a far bigger security risk than using a well secured VNC.

 

[Kernfusion]

My current positions if anyone is interested (damn, there's a lot of instruments in my system now ):

 

[wopr]

My current positions if anyone is interested (damn, there's a lot of instruments in my system now ):
View attachment 292593

6 positions, am I reading this right?
Dang, I probably have a giant bug somewhere.

 

[globalarbtrader]

Makes sense.
I was looking at your strategy reports while I was debugging this, and found some interesting things so will mention here. Your forecasts for gold and silver are both decently negative (actually, subsystem position for gold is -52!) but all DO did with that is short just 2 micro gold contracts. I would have expected overall more short exposure to metals from that.

Also, I'm sure I'm misreading this, but your strategy reports are saying iron has small negative forecast and subsystem position for the last few days, but risk report says you're long. If I recall correctly, that shouldn't be possible in the DO land.

Incidentally, my lron closed today

Rob

 

[wopr]

Well, I've potentially found "a" problem, not sure it's "the" problem.
Kernfusion, thanks for posting that screenshot. It got me looking into correlations and it's interesting. I use Rob's `correlation_over_time_for_returns` and it seems like correlations are capped at 0.55. I've ran correlation across the entire portfolio, and I have nothing over 0.55, and a bunch right at 0.55. Even S&P 500 mini and micro are 0.55, and I'd assume those should be well over 0.9, especially if you roll on the same date, as I do. Same goes for the other way around, VSTXX and ESTX50 are -0.25 for me, and I'd expect that to be a bit more.

Rob, looking at your risk reports, I can't find a correlation that's above 0.6. I'm a few hours into understanding the implementation of the correlation code, but if you have any pointers on why that is, that'd be helpful. My params are the same as yours here with an override for sampling method as done here though it doesn't make a difference in result really.

That said, I likely have a bug elsewhere too, as even though DO could open more positions because instrument are less correlated, I see Rob doesn't have that problem

 

[globalarbtrader]

Well, I've potentially found "a" problem, not sure it's "the" problem.
Kernfusion, thanks for posting that screenshot. It got me looking into correlations and it's interesting. I use Rob's `correlation_over_time_for_returns` and it seems like correlations are capped at 0.55. I've ran correlation across the entire portfolio, and I have nothing over 0.55, and a bunch right at 0.55. Even S&P 500 mini and micro are 0.55, and I'd assume those should be well over 0.9, especially if you roll on the same date, as I do. Same goes for the other way around, VSTXX and ESTX50 are -0.25 for me, and I'd expect that to be a bit more.

Rob, looking at your risk reports, I can't find a correlation that's above 0.6. I'm a few hours into understanding the implementation of the correlation code, but if you have any pointers on why that is, that'd be helpful. My params are the same as yours here with an override for sampling method as done here though it doesn't make a difference in result really.

That said, I likely have a bug elsewhere too, as even though DO could open more positions because instrument are less correlated, I see Rob doesn't have that problem

Got the problem.

The correlation is shrunk (and clipped at 0.9 FWIW although that won't come into effect because of the shrinkage) towards zero.

instrument_returns_correlation:
   ....
  clip: 0.90
  shrinkage_parameter: 0.5

That makes sense for the DO as it makes it more stable (see the relevant post), but not for risk reporting, nor does it make sense for risk overlay calculations. For these we should use unclipped, unshrunk correlations.

I've opened an issue.

Rob