Always-in automated system

dom993

dom993

I have been working on a prototype system which is best described as "always-in". In very simple terms, it uses only 2 kinds of signals, "Long" and "Short", both to close/reverse the current position as well as possibly scale-in the current position. A new position is always started with 1 contract, and every subsequent signal in the same direction is used to scale-in by 1-contract, to the max number of contracts configured.

(this system is not a mean-reversion system, it simply tries to follow the trend)

I am backtesting this system on CL, 6 years (2007-2012) on 100-volume. When using only 1 contract, it generates ~3300 trades - 1/2 long & 1/2 short, of course, with a non-stellar P/F of 1.38, but a very decent avg/trade of $128, using 1-tick slippage on entry, same on exits, and $5 comms / round-turn. Win% 51%, avgW/avgL 1.33.

But this is ignoring about 1/2 of the signals, because of the limit set to 1 contract. Using 5 contracts max, the total number of trades increases to 6380 (about 200 less than the total # of signals), P/F 1.43, avg/trade $142, Win% 50.6%, avgW/avgL 1.39. My point with this being, the performance using all the signals is very-much in-line (actually, better) with the performance using only the 1st signal in each direction.

(I didn't mention DD, it is actually fairly large, -21k for 1co, -52k for 5co, and these figures in both cases are below the mean DD as computed through MonteCarlo simulation using the backtested trade distribution on 3300 / 6400 trades)

Initial testing of "classic" trade-management techniques (initial stop, target) couldn't match the "always-in" version - I still have to work on adding trailing-stop, but given the performance difference just using initial stop/target (or just one of the 2), I will be surprised if I can beat the "always-in" performance.

So at this point, I am considering evolving that prototype into a full system, but "always-in" brings a bunch of challenges, compared to classic intraday setups. I made a (small) list of these challenges, and would be interested in anyone's experience (or even opinion) on the subject ("always-in" automated system).


"Always-in" challenges:

1) 24/7 operations (duh)
2) Must be able to stop then restart the strategy and/or the platform with a trade on (there's always a trade on) without losing track of that open trade
3) Must be able to reload price-data after a loss of connectivity, without impacting the current trade
4) Must be able to automatically roll the current trade to the next contract at the date/time chosen for rollover
5) ...


Thanks in advance
 
Quote from dom993:

I have been working on a prototype system which is best described as "always-in". In very simple terms, it uses only 2 kinds of signals, "Long" and "Short", both to close/reverse the current position as well as possibly scale-in the current position. A new position is always started with 1 contract, and every subsequent signal in the same direction is used to scale-in by 1-contract, to the max number of contracts configured.

(this system is not a mean-reversion system, it simply tries to follow the trend)

I am backtesting this system on CL, 6 years (2007-2012) on 100-volume. When using only 1 contract, it generates ~3300 trades - 1/2 long & 1/2 short, of course, with a non-stellar P/F of 1.38, but a very decent avg/trade of $128, using 1-tick slippage on entry, same on exits, and $5 comms / round-turn. Win% 51%, avgW/avgL 1.33.

But this is ignoring about 1/2 of the signals, because of the limit set to 1 contract. Using 5 contracts max, the total number of trades increases to 6380 (about 200 less than the total # of signals), P/F 1.43, avg/trade $142, Win% 50.6%, avgW/avgL 1.39. My point with this being, the performance using all the signals is very-much in-line (actually, better) with the performance using only the 1st signal in each direction.

(I didn't mention DD, it is actually fairly large, -21k for 1co, -52k for 5co, and these figures in both cases are below the mean DD as computed through MonteCarlo simulation using the backtested trade distribution on 3300 / 6400 trades)

Initial testing of "classic" trade-management techniques (initial stop, target) couldn't match the "always-in" version - I still have to work on adding trailing-stop, but given the performance difference just using initial stop/target (or just one of the 2), I will be surprised if I can beat the "always-in" performance.

So at this point, I am considering evolving that prototype into a full system, but "always-in" brings a bunch of challenges, compared to classic intraday setups. I made a (small) list of these challenges, and would be interested in anyone's experience (or even opinion) on the subject ("always-in" automated system).


"Always-in" challenges:

1) 24/7 operations (duh)
2) Must be able to stop then restart the strategy and/or the platform with a trade on (there's always a trade on) without losing track of that open trade
3) Must be able to reload price-data after a loss of connectivity, without impacting the current trade
4) Must be able to automatically roll the current trade to the next contract at the date/time chosen for rollover
5) ...


Thanks in advance
More...

theoretically wouldn't it be better to have a machine at a data center for all kinds of redundancy issues.. ... can't you virtualize your machine such that it could be seemlessly picked up after a crash.. there are tons of mission critical apps that have these same issues.. think about it .. you spread your risk..
 
Quote from dom993:
... "Always-in" challenges:
1) 24/7 operations (duh)
2) Must be able to stop then restart the strategy and/or the platform with a trade on (there's always a trade on) without losing track of that open trade
3) Must be able to reload price-data after a loss of connectivity, without impacting the current trade
4) Must be able to automatically roll the current trade to the next contract at the date/time chosen for rollover
5) ...
...
More...
24/7 operations
- assuming you've cracked 2) and 3), then "24/7" becomes "24/5"... this gives you an opportunity to at least power down your modem and computer for a few hours each week (yes, you wouldn't have to worry about this with VPS, but in my experience there are still different availability/reliability challenges to be addressed in the case of VPS ... it's a trade off ...).

- IMO "automated" and "unattended" do not mean the same thing, and the former does not imply the latter... "24/7" provides a whole new set of challenges in this regard. I use sound alerts (not popular with my wife and kids!)/SMS to keep track of what's happening ... if you're looking at 6000+ trades over 2500 trading days, I guess that might still be feasible ... a trade or two a day ... I guess your sleep will be interrupted about as frequently as if you had a young child at home ... we can all put up with this ... for a while ... :)

- brokers. IB forces a re-start of TWS at least once every 24-hours. So you need to crack 2) and 3) to deal with that sort of thing ...

... stop and re-start ...
I always find this confusing, but I believe you address 2) with NT using "Immediately submit live working historical orders, Sync account position = false".

... reload data ...
Difficult one, I guess. Although, "philosophically speaking", if the data has changed then the "true" trade to be in is the one once the data has re-loaded, and the trade based on bad data was a "false" trade. So forget about about the false trade and get into the true one. Or, if your view is the opposite, just don't update the data ...

Rolling contracts
Perhaps the following? Use a multi-instrument strategy which has a roll-over window defined for a few days/a week before roll-over. During that window the strategy looks for a single opportunity during a reversal trade to exit the one contract and then enter the other ... ?
 
Quote from dom993:


"Always-in" challenges:

1) 24/7 operations (duh)
2) Must be able to stop then restart the strategy and/or the platform with a trade on (there's always a trade on) without losing track of that open trade
3) Must be able to reload price-data after a loss of connectivity, without impacting the current trade
4) Must be able to automatically roll the current trade to the next contract at the date/time chosen for rollover
5) ...


Thanks in advance
More...



Always-in can be done with stops in place in case of loss of connectivity. When you are connected your system moves the [broker or exchange resident] stops.

#2 above: I use third party off the shelf software for nearly everything, it has settings to accomplish that. I have to assume that most software available has that capability.

#3 above: My chart package does that automatically, seems safe to assume that most do

#4 above: Use the continuous contract symbol if your data provider has same
 
I run my system on Amazon EC2 (virtual computer). This solves the 24/7 and connectivity. I trade with InteractiveBrokers and the connectivity issues were solved once I moved from my own computer to EC2.

On the EC2 I use InteractiveBrokers Gateway. This solves the forced TWS restart issue that abattia mentioned. IB Gateway may run indefinitely (never testet as I restart it every weekend).

For the automated system I use customized JSystemTrader/JBookTrader application. Some of my strategies are also always-in so I had the same requirement about reading the current open positions at restart. I read it from the local disk and check it with the current position that I get from broker.

On which platform did you implement your system? Which broker do you plan to use?
 
Sorry folks, I was in the middle of a very long & detailed response, and just lost it all (my bad - I am tired).

Thank you for all the suggestions & comments, I will get back to this after a good night of sleep - I need it :(
 
Take-two ... What I am most worried about is missing a "big" thing (the blank item 5) ...), but so far so good.

I am on Ninja, using Kinetick(aka IQfeed) as primady datafeed, with accounts at IB & Dorman.


1) 24/7 operations
For the last 6 months, for my current systems I have been using a VM (actually 2 VMs, see 3) in a boutique ... enough to understand and appreciate the benefits of it, however there have been a few hiccups along the way and I might look for an alternative. Any suggestion beyond Amazon EC2 ?

Re. IB/TWS, I am using IBController, which provides automated response to TWS "5min to logout" dialog-box. I tested yesterday using 2 TWS instances (as I am going to soak this system in my IB sim trading account for a few weeks, to iron-out the operations side of it), it works well as long as each TWS instance is launched by a separate IBController instance.

I use Kinetick w/ the IQfeed connector (as the Kinetick connector has random issues after loss of connectivity) as my primary datafeed. I also have Zenfire for an account at Dorman, I have never tried using it as backup datafeed - and I certainly should do some testing in that area.

I am monitoring my systems during day-time, I even have them also running on a computer here (on sim) to get the trades called locally, and just have to check the live instances on the VM are doing the same. I am not so much worried about the system taking trades unattended at night (although I agree getting a SMS per trade entry/exit would be good), rather by a technical issue taking the systems on the VM off-line. I want to automate the monitoring of those systems from here, however I don't have much idea re. how to proceed. Any suggestion?


2) stop/restart the strategy
Thank you abattia for steering me towards "Immediately submit live working historical orders, Sync account position = false". So far I have implemented a tiny position monitoring engine, which saves the current position to file, so upon restart the strat. knows what it is (I don't think it is a good idea to rely on the replay of historical data, as after reloading that historical data the replay might come-up with a different view on things). I think I can use the same idea & record the current stop/target working orders (should I ever use that), so that on restart the strat can issue just the known set of orders (1 to enter the current position + the open stop/target orders), which will allow "Immediately submit live working historical orders, Sync account position = false" to get the strat in a perfectly matching state.

The only problem I see with "Immediately submit live working historical orders" is that this applies to all strategies, so I will have to adapt my other strategies to work properly in that mode.


3) reload data
I already have (from my current systems) a ReloadManager, which reloads data after a loss of connectivity (the strat is automatically stop/restart in the process). That ReloadManager used to wait for an instrument to be flat on all client strategies before reloading data, I just changed it so that each strategy provides a "ReloadReadyStatus", so my existing strategies will keep waiting to be flat, while this one will (almost) always advertise being reload-ready.

The Ninja folks never agreed to provide an API to reload data, on top of the Ctrl+Shift+R chart command ... as a result my ReloadManager uses SendKeys (which is very challenging to get to work under all circumstances), and for remote systems SendKey requires a remote desktop connection open & active (not minimized) to work. I use 2 VMs for this to work, 1 access VM connected 24/7 to the trading VM. Obviously, having the 2 VMs on the same server communicating directly over the datacenter LAN (rather than through the Internet) helps keeping the remote access useable.

I agree there is a risk that after reloading, the current position might not be what it should have been, however what I discovered in testing the prototype is that it is better to wait for a qualified signal to do anything (I also have disqualified/not-better-than-random signals, and whenever I tried using those for anything, even simple exit, the net outcome was far worse than just waiting for the next qualified signal - which is pretty interesting, as the qualified signals where identified by just looking to what happens until the next signal, qualified or not).


4) automated rollover
Yes, using a multi-instrument strategy is what I am thinking of doing, targetting 2:29:55pm EST on the chosen rollover-day to close the current position on the expiring contract & open same size position on the new front-month. I think those seconds just prior to 2:30pm have the largest volume & smallest volatility, I hope to see minimum slippage using 2 simultaneous MKT orders.

It would be nice to have this fully automated (by this, I mean beyond moving the position to the next contract, also moving the strategy to use the next contract), an easy way to do it would be to change the primary dataseries instrument on the fly, not sure there is even an unsupported way of doing that. The other way is multi-instrument, and might actually end-up simpler to
get to work. Anyway, as a start I can stop the strat after the rollover, reconfigure the primary dataseries, and restart the strategy. I know the merge policy must be set to "DoNotMerge" for the few hours before midnight of the configured rollover date, as otherwise Ninja does erroneous price adjustment on the new front-month contract.
 
Quick update ...

1) 24/7 operations - just started to look around for another hosting provider. Initial contact with EOTPRO (Chicago Equinox datacenter). Would appreciate other suggestions.

2) stop/restart strategy w/ position on - short-term version implemented & tested, checks & warns on restart if discrepancy between last position on file and account position, also ensure that last position on file is coherent with most recent valid signal direction, else closes current position & initiate opposite direction (in case that last valid signal was missed during a connection loss or the reload/restart process itself).

3) reload data - evolved my existing ReloadManager as required.

4) auto-rollover - implemented & tested - already been used today to roll the position on my IB sim-account to CL 04-13. Interestingly, I had forgotten today was the day to roll CL positions at the close (post Valentine Day trauma :), so it really came as a surprise - but worked like a charm.


So I have the system running real-time in my IB sim-account since Wednesday - and issuing trade signals to the Collective2 system I created for it (http://crudeoilalwaysin.collective2.com/). Already found & fixed a handful of operations related bugs (including in this connectivity issues w/ C2) - that's exactly the purpose of this dry-run.
 
You must log in or register to reply here.
Back
Top