Fully automated futures trading

HobbyTrading said:
Using your own trading server works fine until it doesn't.
Yesterday I decided to upgrade my trading server from Ubuntu 20.04 to 22.04, the new Long Term Support version. I had previously determined on my desktop computer that all relevant software runs properly on 22.04. The trading server is a headless machine: neither keyboard nor monitor is connected. I use a remote desktop program (Remmina) to control it. After the upgrade this remote control did not work any more. I found two problems, one of which I can not solve. The default setting in 22.04 is to not allow screen sharing. I had to connect a monitor and mouse to find that setting and enable it (solved). That's when I ran into the second problem (unsolved). 22.04 has improved security for screen sharing: authentication is required before a host can see the screen or take control of the machine. Authentication involves providing a username and password. Things go wrong with the password: each time when you reboot the server the password get modified into a random string. Even if you manually modified the password, this gets overridden at each reboot.
The purpose of a headless server is to not have a monitor. But without monitor you can't see the current password. So you can't remotely log in to this server. It is therefore impossible to use 22.04 on a headless server.
Once I encountered this issue I started looking for a solution online. To my horror I find that many people complain about this. And that there is no solution for this problem available. Some people have their servers turned into bricks: these computers are physically inaccessible (thus a monitor cannot be attached) and cannot be repaired remotely.

[end of rant]
More...
umm ... I have 2 x Ubuntu 22.04 computers, one without keyboard and screen, and connect from one to the other using Remmina. Sure, I had that initial problem (of screen sharing from memory), but everything else just works.
I'm not a Ubuntu expert, but DM if necessary.
KH
 
KevinBB said:
umm ... I have 2 x Ubuntu 22.04 computers, one without keyboard and screen, and connect from one to the other using Remmina. Sure, I had that initial problem (of screen sharing from memory), but everything else just works.
I'm not a Ubuntu expert, but DM if necessary.
KH
More...
With the active help from @KevinBB was I able to get it working. It involved installing and configuring the xrdp software. It is not working to the standard how it used to be, but at least the very basic functionality is there.

Thank you Kevin for your time and patience!
 
newbunch said:
Hey Rob or anyone else who uses Rob's execution algo (or something similar):

Have you noticed a change in executions the last month or so? Historically, I paid about 60% of the bid-ask spread (I know Rob has paid about 50% historically), but over the last month it's been more like 80%. I did make some changes to my trade system whereby it now chooses at the outset to trade contracts with tighter spreads and that very well could be the cause of the change I've seen (tighter spreads could mean less opportunity for price improvement). Indeed, even though my executions in terms of bid-ask spread has been worse, my overall costs have gone down (big caveat, one month is not enough of a sample to draw any definitive conclusions).

Anyway, has anyone else noticed a change in executions related to the percentage of the bid-ask spread paid?

Thanks. (I may need to make changes to my execution algo.)
More...

A month is hard to draw any conclusions as you say, but I haven't personally noticed anything.

Rob
 
HobbyTrading said:
Using your own trading server works fine until it doesn't.
Yesterday I decided to upgrade my trading server from Ubuntu 20.04 to 22.04, the new Long Term Support version. I had previously determined on my desktop computer that all relevant software runs properly on 22.04. The trading server is a headless machine: neither keyboard nor monitor is connected. I use a remote desktop program (Remmina) to control it. After the upgrade this remote control did not work any more. I found two problems, one of which I can not solve. The default setting in 22.04 is to not allow screen sharing. I had to connect a monitor and mouse to find that setting and enable it (solved). That's when I ran into the second problem (unsolved). 22.04 has improved security for screen sharing: authentication is required before a host can see the screen or take control of the machine. Authentication involves providing a username and password. Things go wrong with the password: each time when you reboot the server the password get modified into a random string. Even if you manually modified the password, this gets overridden at each reboot.
The purpose of a headless server is to not have a monitor. But without monitor you can't see the current password. So you can't remotely log in to this server. It is therefore impossible to use 22.04 on a headless server.
Once I encountered this issue I started looking for a solution online. To my horror I find that many people complain about this. And that there is no solution for this problem available. Some people have their servers turned into bricks: these computers are physically inaccessible (thus a monitor cannot be attached) and cannot be repaired remotely.

[end of rant]
More...

Blimey. Have you tried other solutions? I use vnc. Otherwise I guess a different distribution?

Rob
 
globalarbtrader said:
A month is hard to draw any conclusions as you say, but I haven't personally noticed anything.

Rob
More...
I noticed in https://qoppac.blogspot.com/2022/04/trading-and-investing-performance-year.html, you wrote that "Total slippage was a rather hefty 2.4%, which meant I paid around 3% in total costs. This is more than I normally pay (my backtest says 1%), but I have made quite a few changes to my models this year... My slippage would have been higher but I earned about 0.5% using my simple execution algo which reduced it."
Does this mean your slippage is back to your "normal" 50% rather than the 83% implied above (2.4% / (2.4% + 0.5%))?
 
globalarbtrader said:
I use vnc. Otherwise I guess a different distribution?
More...
In Ubuntu 22.04 has VNC been downgraded to "legacy" status. The preferred method now is RDP, as that makes it compatible to Microsoft's Windows environment. You can still enable VNC in the remote server but you'll run in the same changing password conundrum. Meanwhile I read a lot of things online about this problem. But nobody is suggesting to switch to another distribution. Which makes me wonder whether other distributions have the same problem (I haven't verified this though). I get the impression that the Ubuntu community has recognized that there is a problem. But there doesn't seem to be a solution for it yet, and I don't know ow much longer this is going to take.
 
HobbyTrading said:
In Ubuntu 22.04 has VNC been downgraded to "legacy" status. The preferred method now is RDP, as that makes it compatible to Microsoft's Windows environment. You can still enable VNC in the remote server but you'll run in the same changing password conundrum. Meanwhile I read a lot of things online about this problem. But nobody is suggesting to switch to another distribution. Which makes me wonder whether other distributions have the same problem (I haven't verified this though). I get the impression that the Ubuntu community has recognized that there is a problem. But there doesn't seem to be a solution for it yet, and I don't know ow much longer this is going to take.
More...
Can you still ssh into the machine (via command line) to set the password to whatever you need for the remote desktop? Machine reboots shouldn't be too frequent, and my typical uptime is months at a time.
 
tradrjoe said:
Can you still ssh into the machine (via command line) to set the password to whatever you need for the remote desktop?
More...
I'm not sure. I did read some hacky method somewhere. It involved deleting the entire keyring, including all stored passwords, via ssh on the remote server and then reboot. This would somehow influence the password which is generated for authentication of a remote desktop session. However, whether this works or not I can't confirm.
 
newbunch said:
I noticed in https://qoppac.blogspot.com/2022/04/trading-and-investing-performance-year.html, you wrote that "Total slippage was a rather hefty 2.4%, which meant I paid around 3% in total costs. This is more than I normally pay (my backtest says 1%), but I have made quite a few changes to my models this year... My slippage would have been higher but I earned about 0.5% using my simple execution algo which reduced it."
Does this mean your slippage is back to your "normal" 50% rather than the 83% implied above (2.4% / (2.4% + 0.5%))?
More...

I just did a report from 1st April (since I haven't made any serious changes)

RzTuwxLh5n873YVPA


https://photos.app.goo.gl/RzTuwxLh5n873YVPA

Code: 
********************************************************************************
Trades report report produced on 13/09/2022 08:57 from 01/04/2022 00:00 to 13/09/2022 08:57
********************************************************************************


delay_cash Sum
                                        delay_cash
strategy_name         instrument_code             
_ROLL_PSEUDO_STRATEGY BRE               -21.358850
                      CRUDE_W_mini       64.076550
                      DJSTX-SMALL       -47.608556
                      EU-BASIC          -71.412835
                      EU-OIL             73.576860
                      IRON               12.815310
                      KR10                9.289177
                      MXP                 2.135885
                      SILVER            -18.368611
                      TOPIX              -0.600178
                      V2X                58.428683
dynamic_TF_carry      AUD               108.930135
                      BITCOIN          -364.595570
                      BRE                -8.543540
                      BUND              528.022170
                      CORN             1569.875478
                      CRUDE_W_mini     3006.258143
                      DAX                -2.596830
                      DJSTX-SMALL        23.804278
                      DOW                17.514257
                      EU-BASIC          705.472244
                      EU-OIL            -38.952455
                      EUR              1516.478353
                      EUROSTX          -510.709968
                      FTSETAIWAN        111.066020
                      GAS_US_mini     -3481.492556
                      GBP               400.478438
                      GBPEUR          -1006.250000
                      GOLD_micro       -164.035968
                      IRON              -66.212435
                      JPY               576.688951
                      KOSDAQ            283.629552
                      KR10              120.759307
                      LEANHOG          -264.849740
                      LIVECOW          -410.089921
                      MSCISING          -33.684812
                      MUMMY              57.016951
                      MXP              -207.180845
                      NASDAQ_micro      831.713621
                      NIKKEI           -201.059776
                      NIKKEI400        -151.545055
                      RUSSELL            41.649758
                      SILVER             17.087080
                      SOYBEAN             5.339713
                      SOYOIL          -1391.742669
                      SP500_micro      1146.970247
                      TOPIX             -15.004461
                      US10U            1468.420940
                      US20             -574.019095
                      V2X                75.740885
                      WHEAT           -2803.349068
                      YENEUR          -1200.356869
delay_cash Total Sum
                       delay_cash
strategy_name                   
_ROLL_PSEUDO_STRATEGY   60.973435
dynamic_TF_carry      -283.355111
bid_ask_cash Sum
                                       bid_ask_cash
strategy_name         instrument_code             
_ROLL_PSEUDO_STRATEGY BRE                 -4.271770
                      CRUDE_W_mini       -32.038275
                      DJSTX-SMALL         -4.328051
                      EU-BASIC            -2.164025
                      EU-OIL              -4.328051
                      IRON                -8.543540
                      KR10                -9.289177
                      MXP                 -2.135885
                      SILVER              -0.427177
                      TOPIX               -5.101517
                      V2X                -10.820126
dynamic_TF_carry      AUD                 -2.135885
                      BITCOIN            -14.737607
                      BRE                 -8.543540
                      BUND               -17.312202
                      CORN               -21.358850
                      CRUDE_W_mini       -96.114825
                      DAX                -77.039300
                      DJSTX-SMALL       -201.254352
                      DOW                 -5.126124
                      EU-BASIC          -670.847840
                      EU-OIL            -428.477007
                      EUR                -21.358850
                      EUROSTX             -8.656101
                      FTSETAIWAN          -8.543540
                      GAS_US_mini        -96.114825
                      GBP                -16.019138
                      GBPEUR             -31.250000
                      GOLD_micro         -20.077319
                      IRON              -177.278455
                      JPY                -10.679425
                      KOSDAQ              -8.669899
                      KR10                -9.289177
                      LEANHOG            -17.087080
                      LIVECOW             -8.543540
                      MSCISING            -3.062256
                      MUMMY              -21.006245
                      MXP                -14.951195
                      NASDAQ_micro        -1.708708
                      NIKKEI              -6.001784
                      NIKKEI400          -16.504907
                      RUSSELL             -6.621244
                      SILVER             -42.717700
                      SOYBEAN             -5.339713
                      SOYOIL             -99.959418
                      SP500_micro         -5.339713
                      TOPIX              -13.504015
                      US10U              -13.349281
                      US20               -13.349281
                      V2X                -23.804278
                      WHEAT             -368.440163
                      YENEUR             -22.506691
bid_ask_cash Total Sum
                       bid_ask_cash
strategy_name                     
_ROLL_PSEUDO_STRATEGY    -83.447594
dynamic_TF_carry       -2654.681474
execution_cash Sum
                                       execution_cash
strategy_name         instrument_code               
_ROLL_PSEUDO_STRATEGY BRE                2.593619e-13
                      CRUDE_W_mini      -6.165403e-12
                      DJSTX-SMALL       -4.901203e-13
                      EU-BASIC          -0.000000e+00
                      EU-OIL             2.959942e-12
                      IRON              -2.442448e-13
                      KR10               0.000000e+00
                      MXP                2.130473e-12
                      SILVER             7.054643e-13
                      TOPIX              0.000000e+00
                      V2X               -1.837951e-13
dynamic_TF_carry      AUD               -4.271770e+00
                      BITCOIN            7.689186e+00
                      BRE                0.000000e+00
                      BUND               1.731220e+01
                      CORN               1.067943e+01
                      CRUDE_W_mini      -4.271770e+01
                      DAX                5.193661e+00
                      DJSTX-SMALL        2.596830e+01
                      DOW               -9.825071e+00
                      EU-BASIC           5.713027e+02
                      EU-OIL            -1.731220e+01
                      EUR                2.135885e+01
                      EUROSTX            0.000000e+00
                      FTSETAIWAN        -1.708708e+01
                      GAS_US_mini       -3.203828e+01
                      GBP                2.135885e+01
                      GBPEUR             1.250000e+01
                      GOLD_micro        -5.126124e+00
                      IRON               0.000000e+00
                      JPY               -9.261925e-12
                      KOSDAQ            -4.334949e+00
                      KR10              -1.238557e+01
                      LEANHOG            0.000000e+00
                      LIVECOW            0.000000e+00
                      MSCISING          -3.062256e+00
                      MUMMY              0.000000e+00
                      MXP               -4.271770e+00
                      NASDAQ_micro       1.623273e+01
                      NIKKEI             0.000000e+00
                      NIKKEI400         -6.001784e+00
                      RUSSELL            1.281531e+00
                      SILVER            -2.563062e+01
                      SOYBEAN            1.067943e+01
                      SOYOIL             5.638736e+01
                      SP500_micro       -1.174737e+01
                      TOPIX              4.501338e+00
                      US10U              0.000000e+00
                      US20              -0.000000e+00
                      V2X               -3.073097e-13
                      WHEAT              2.135885e+01
                      YENEUR             7.502230e+00
execution_cash Total Sum
                       execution_cash
strategy_name                       
_ROLL_PSEUDO_STRATEGY   -1.028322e-12
dynamic_TF_carry         6.154941e+02
versus_limit_cash Sum
                                       versus_limit_cash
strategy_name         instrument_code                   
_ROLL_PSEUDO_STRATEGY BRE                   0.000000e+00
                      CRUDE_W_mini          0.000000e+00
                      DJSTX-SMALL          -8.656101e+00
                      EU-BASIC             -4.328051e+00
                      EU-OIL               -8.656101e+00
                      IRON                  0.000000e+00
                      KR10                  0.000000e+00
                      MXP                   0.000000e+00
                      SILVER                0.000000e+00
                      TOPIX                 0.000000e+00
                      V2X                  -2.164025e+01
dynamic_TF_carry      AUD                   0.000000e+00
                      BITCOIN               0.000000e+00
                      BRE                   0.000000e+00
                      BUND                  0.000000e+00
                      CORN                  0.000000e+00
                      CRUDE_W_mini         -5.339713e+01
                      DAX                  -5.713027e+01
                      DJSTX-SMALL           0.000000e+00
                      DOW                  -1.836861e+01
                      EU-BASIC              0.000000e+00
                      EU-OIL               -3.029635e+01
                      EUR                   0.000000e+00
                      EUROSTX              -1.731220e+01
                      FTSETAIWAN            0.000000e+00
                      GAS_US_mini           0.000000e+00
                      GBP                   0.000000e+00
                      GBPEUR               -1.387779e-11
                      GOLD_micro           -6.834832e+00
                      IRON                  0.000000e+00
                      JPY                   0.000000e+00
                      KOSDAQ                0.000000e+00
                      KR10                  0.000000e+00
                      LEANHOG               0.000000e+00
                      LIVECOW               0.000000e+00
                      MSCISING              0.000000e+00
                      MUMMY                 0.000000e+00
                      MXP                   0.000000e+00
                      NASDAQ_micro          0.000000e+00
                      NIKKEI                0.000000e+00
                      NIKKEI400             0.000000e+00
                      RUSSELL              -2.990239e+00
                      SILVER                0.000000e+00
                      SOYBEAN               0.000000e+00
                      SOYOIL                0.000000e+00
                      SP500_micro           0.000000e+00
                      TOPIX                 0.000000e+00
                      US10U                 0.000000e+00
                      US20                  0.000000e+00
                      V2X                  -4.328051e+00
                      WHEAT                 0.000000e+00
                      YENEUR                0.000000e+00
versus_limit_cash Total Sum
                       versus_limit_cash
strategy_name                           
_ROLL_PSEUDO_STRATEGY         -43.280506
dynamic_TF_carry             -190.657682
versus_parent_limit_cash Sum
                                       versus_parent_limit_cash
strategy_name         instrument_code                         
_ROLL_PSEUDO_STRATEGY BRE                                     0
                      CRUDE_W_mini                            0
                      DJSTX-SMALL                             0
                      EU-BASIC                                0
                      EU-OIL                                  0
                      IRON                                    0
                      KR10                                    0
                      MXP                                     0
                      SILVER                                  0
                      TOPIX                                   0
                      V2X                                     0
dynamic_TF_carry      AUD                                     0
                      BITCOIN                                 0
                      BRE                                     0
                      BUND                                    0
                      CORN                                    0
                      CRUDE_W_mini                            0
                      DAX                                     0
                      DJSTX-SMALL                             0
                      DOW                                     0
                      EU-BASIC                                0
                      EU-OIL                                  0
                      EUR                                     0
                      EUROSTX                                 0
                      FTSETAIWAN                              0
                      GAS_US_mini                             0
                      GBP                                     0
                      GBPEUR                                  0
                      GOLD_micro                              0
                      IRON                                    0
                      JPY                                     0
                      KOSDAQ                                  0
                      KR10                                    0
                      LEANHOG                                 0
                      LIVECOW                                 0
                      MSCISING                                0
                      MUMMY                                   0
                      MXP                                     0
                      NASDAQ_micro                            0
                      NIKKEI                                  0
                      NIKKEI400                               0
                      RUSSELL                                 0
                      SILVER                                  0
                      SOYBEAN                                 0
                      SOYOIL                                  0
                      SP500_micro                             0
                      TOPIX                                   0
                      US10U                                   0
                      US20                                    0
                      V2X                                     0
                      WHEAT                                   0
                      YENEUR                                  0
versus_parent_limit_cash Total Sum
                       versus_parent_limit_cash
strategy_name                                 
_ROLL_PSEUDO_STRATEGY                         0
dynamic_TF_carry                              0
total_trading_cash Sum
                                       total_trading_cash
strategy_name         instrument_code                   
_ROLL_PSEUDO_STRATEGY BRE                   -4.271770e+00
                      CRUDE_W_mini          -3.203828e+01
                      DJSTX-SMALL           -4.328051e+00
                      EU-BASIC              -2.164025e+00
                      EU-OIL                -4.328051e+00
                      IRON                  -8.543540e+00
                      KR10                   0.000000e+00
                      MXP                   -2.135885e+00
                      SILVER                -4.271770e-01
                      TOPIX                  0.000000e+00
                      V2X                   -1.082013e+01
dynamic_TF_carry      AUD                   -6.407655e+00
                      BITCOIN               -7.048421e+00
                      BRE                   -8.543540e+00
                      BUND                   2.460165e-11
                      CORN                  -1.067943e+01
                      CRUDE_W_mini          -1.388325e+02
                      DAX                   -7.184564e+01
                      DJSTX-SMALL           -1.752860e+02
                      DOW                   -1.495120e+01
                      EU-BASIC              -9.954516e+01
                      EU-OIL                -4.457892e+02
                      EUR                   -2.371348e-11
                      EUROSTX               -8.656101e+00
                      FTSETAIWAN            -2.563062e+01
                      GAS_US_mini           -1.281531e+02
                      GBP                    5.339713e+00
                      GBPEUR                -1.875000e+01
                      GOLD_micro            -2.520344e+01
                      IRON                  -1.772785e+02
                      JPY                   -1.067943e+01
                      KOSDAQ                -1.300485e+01
                      KR10                  -2.167475e+01
                      LEANHOG               -1.708708e+01
                      LIVECOW               -8.543540e+00
                      MSCISING              -6.124511e+00
                      MUMMY                 -2.100625e+01
                      MXP                   -1.922297e+01
                      NASDAQ_micro           1.452402e+01
                      NIKKEI                -6.001784e+00
                      NIKKEI400             -2.250669e+01
                      RUSSELL               -5.339713e+00
                      SILVER                -6.834832e+01
                      SOYBEAN                5.339713e+00
                      SOYOIL                -4.357205e+01
                      SP500_micro           -1.708708e+01
                      TOPIX                 -9.002677e+00
                      US10U                 -1.334928e+01
                      US20                  -1.334928e+01
                      V2X                   -2.380428e+01
                      WHEAT                 -3.470813e+02
                      YENEUR                -1.500446e+01
total_trading_cash Total Sum
                       total_trading_cash
strategy_name                           
_ROLL_PSEUDO_STRATEGY          -69.056900
dynamic_TF_carry             -2039.187394

So in ~5.5 months my execution algo 'earned' £600 versus I would have paid £2700 in bid-ask spread costs for a net cost of £2100; so it's reducing my slippage to more like 78% of what you'd expect. I also paid £350 in commissions.

Those results are however heavily affected by a few trades on which the slippage was quite large as the plot makes clear. It certainly doesn't look like the algo is adding much value, but I don't think it's losing any money eithier.

While I'm here, a useful sense check is to annualise the above figures and divide by maximum capital to get a percentage (400k). I get 1.1% in annual slippage (net of execution algo) plus 0.2% in commissions for a net of 1.3%. That's much lower than the 3% from last year, and only a tad above the 1% in my backtest.

Rob
 
You must log in or register to reply here.
Back
Top