Trading with automation (with IB)

[fullautotrading]

Great thread, fullautotrading! I like that you're experiencing ...

Thank you rlx01. Some enthusiasm is always refreshing

Yes, we are still in a DD phase mostly due to the "stretch" of the mkt (especially NQ, YM and ES). The DD has been amplified by the correlations of these 3 instruments (lesson for next time), which when the mkt pulls are practically the "same" instrument.

Anyway, we have opened just 2 "corridors" per instrument so far. So, nothing particularly worrying, we are just starting the dances, actually

Some more DD is due to NG (natural gas) which has been pulling down and we have been also hit by the large contango on a long position. Again, here too we have only 2 set of players active, so no big deal yet.

Our plan is obviously to gradually populate all the various "price corridors" (which we can imagine in the price range) with "set of layers" and recover all stops of a given corridor on each reversion.

One rule to determine the "price corridors" could be as follows:

open a new set of players when the price moves of the following relative amount: price move% = volatility% / 4

(e.g., approximately, about 8% for CL, or 4% for ES). Clearly, one could use a general rule as: price move% = volatility% / K

where K is a user parameter. I will be following such guideline to populate the price range of each instrument with new set of players.

Note that distributing capital over the price range is also a meaningful a form of "diversification".

 

[fullautotrading]

We are still "underwater" at the moment, but nothing particularly worrying or abnormal: just following our plan.
So far most of the DD is due to the strongly correlated instruments NQ, YM, ES. SI is also momentarily shooting down, causing some "load" on its upper set of layers. Same thing for ZW.

It may be interesting to examine the instruments one by one, to analyse in detail our current situation (in the next posts) and how the various different components are doing.

I have also been doing some refinements to the order sizing algorithm (to address the problem with "fractional" hedging), and I will explain later the changes that I have made.

 

[fullautotrading]

As anticipated, while we are still "underwater" and patiently waiting to get back in the green, it's a good time to check one by one the instruments and see how they are doing.

We may start from CL which is one of the first we started.

The global situation of all layers is as follows:

We are currently at about -24K PNL with a G-L of about 176K. Most of the unrealized is on the mkt indexes that lately have been stretching up a bit.

Let's examine the CL layers. We have been trading for 65 (solar) days so far and since we started CL has been fluctuating in a range of about 16%. The price was initially around 43.9, then it went up to around 51, then down to about 42, up again to around 48, and finally down to 41.6 where it is at this moment.

We have been opening 2 sets of scalping/hedging layers. One set, when we just started and another set when the price seemed to escape upward.
As we can see from the picture above, the first set is largely positive because it could recover all the hedging orders (price went up and returned in the scalping "corridor").

The second set of layers is mainly in a load phase ("losing", if you prefer). In fact, being active in the upper price corridor, it would realized the profits and "unwind" the various hedging order when the price goes back again in that "corridor" (higher price).

So, this is how the most profitable CL layer looks (in the bottom "corridor"), with a view of all the automated orders in that price corridor:

The negative upper layers are instead still basically mostly plain buy orders, which had the function to "stop" (hedge) the loss of the first set of layers when the price seemed to be shooting up. We will recover all those "stops" (which in this approach remain as "open" players and forever in the memory of the application) when the price will go up again in that price corridor.

Obviously if we had not open the second set of layers, we would not have those losing layers. But of course, we take a non-predictive approach, where instead of pure gambling, we trigger a new set of layers when we see the price escape from a price corridor, and we proceed this way, mechanically, without attempting to predict anything, but simply scalping and hedging in the various price corridors . In any case, sooner or later the price will return in most price corridors we may define (at least for the most suitable instruments), and we get a chance to recover all our hedging ("stopping") orders, while in a predictive approach, once one takes a stop, normally the relevant trading information is also "erased", and the "monetary" loss which is "realized" in case of unfavorable bet is essentially represented by the loss of trading information.

 

[fullautotrading]

Let's now examine SI (silver index). SI practically did essentially the same thing as CL. It initially went up (enough that we opened a new set of layers) and then went down (currently 14.3) moving in a range of about 16%. So we have the first (initial) set of layers in profit, and the second one holding hedging players which are waiting for the next surge (they correspond to "stops" in an ordinary approach, with the difference that since they are represented as open players, they will be "undone" when the price returns to the upper corridor).

It is probably more clear if we take a look at one layer of the first "corridor" (lower price level), compared with a layer of the second corridor (higher price):

"initial" price level:

higher price level:

Imho, this concept of representing the "stops" as open players could possibly be interpreted intuitively, with a stretch of imagination, almost like having a sort special "reusable protective option" in place, which is not subject to time decay.
Clearly, the concept of players provides a simple way of working with "components" of the buy/sell averages and avoiding replicating the stop orders multiple times (which is what removes most profits from the common automated approaches).

 

[fullautotrading]

A situation similar to both CL and SI we also find with ZL (soybean oil). In this case too we had the price first going up enough to let us open a new set of layers, and then the price returned in the first corridor. So just like SI before, we have a first set of players getting its deserved profit, and a second set holding hedging orders, which may be closed in a future price surge:

Here is a picture of one layer of ZL belonging to the initial set of layers with the various automated orders (initial "corridor"):

[ In the meantime, our DD has been "reabsorbed" considerably in the last hours (currently PNL is around -10K and G-L around 164K). ]

 

[fullautotrading]

As the mkt is stretching again we are still in DD: about -3.26% of our capital, while the maximum margin we have be using is around 200K, so we have still way to go and distribute $$$ across the price range of the various selected instruments.

What appears evident is that some instruments do systematically better than others as far as their microstructure is concerned: they simply are richer in "good fluctuations" (even when they wildly trend).

One idea that just became evident to possibly greatly reduce the risk and improve performances (apart focusing on the best instruments and removing those less effective) is to allocate systematically capital on the drifting ETFs as an extra source of hedging.

In fact, especially looking at NQ, YM and ES, and also NG, it is quite evident that a main source of DD in our approach is provided by the various "systematic drifts".

So the idea is to grab the possible price "peaks" on ETFs and allocate some capital so that we can ride the "natural drift" of the ETFs. We will not automatically trade them, but instead we just leave the position there as a form of fixed hedge and let it run.

To do that, we can place some "enqueued orders", that is, orders which automatically triggers at especially high or low prices to take positions on some specific ETFs, and the leave it in "manual mode" so they do not autotrade (in case we might just add more enqueued orders to overcharge the position in case of real extreme moves).

To demonstrate this, I have added an enqueued order on DGAZ (which will provide some protection against the contango of NG, and also ride the "structural" ETF decay, due to leverage and consequent daily rebalancing) at level 14.15 (2000 shares), which has actually triggered on Friday:

Similarly I have added other enqueued order on other ETFs, for instance: VXX, TZA, ERY, NUGT, etc. (to hedge ES, NQ, YM, CL, SI, etc.... btw as soon as possible I will remove NQ, YM leaving only ES):

In other words, since our greatest "weakness" is obviously in the various form of drifts (mkt drift, contango, etc.) (while our greatest strength is in the "choppiness" and fluctuations, we hedge against that by riding the ETFs and thus also riding their "natural" decay). However, we do not actually automatically trade the ETFs, for the reason explained in the first posts. (For such purpose, we will generally prefer the "bear" versions of the ETFs or those which are essentially "inverse" wrt to mkt.)

 

[fullautotrading]

Yesterday (black Friday) I was a bit surprised to see that the application had gone out of sync due to sudden "disappear" of the SI and ZM positions.

============================================== NOT MATCHING PRESENT IN THIS INSTANCE ===============================================
Symb                                                                       This G-BOT             IB            Exp     Sync
SI                                 [SIZ5]                                           6              0     31.20 days     SELL 6 @ XXX
ZM                                 [ZM   DEC 15]                                    2              0     16.20 days     SELL 2 @ XXX
------------------------------------------------------------------------------------------------------------------------------------

Fine with ZM, but liquidating SI more than a month before expiration seems a bit too early. Anyway, the only reason I can think of for this liquidation must be the so called "physical delivery rule". Given that, next time, let's remember to roll these guys earlier.

What is quite shameful is that the IB API do not report these kind of actions on their side and one has to wait an entire day to look at the daily online report (Nov 27):

This is unacceptable and any order carried out from IB side should immediately reported via API. Both time and order details at least.

Clearly, we can re-sync as soon as the mkt opens up, by making the rollover and placing "virtual orders" just equal to IB "liquidation" orders. So no big deal, as we are well equipped to deal with these kind of situations, while it's surely very irritating not being notified (actually not even by email, as far as I can see) timely when the event occurs through the API channel.

 

[fullautotrading]

As anticipated in the previous post I have now re-synced the application with the account. As to SI which had 5 layers needed to roll over, I have proceeded as follow:

1. I have added the virtual order on the old layers (in "pause" mode) to re sync

2. Then I have chosen the target instrument, in this case SIH6 because of spread and volume (I looked at SIF6, SIG6 too). Then I "cloned" this layer in order to create the new 5 target layers for the rollover:

3. Finally made the rollover (trading "information transfer" from the old layers to the corresponding new ones), one by one.

After that, we are back in sync with the account and also perfectly ready to restart full autotrading on SI. Clearly, we need first to restore our position which was liquidated without notice by IB.

We have at least 2 possible ways to do that.

1. Just add the new buy players which were liquidated

2. Transform the "virtual" liquidation orders we just added into the previous layers into sell players (just click on them to make the conversion) and let them close at a minimal profit (like $50 or so) to recover all rollover expenses (an automatic trigger can be set for players, just by clicking on them).

Since when I did the procedure the price was randomly fluctuating near the liquidation price, I opted for the second way, so in practice the rollover did not cost anything (and actually also made a few bucks).

I have done the same procedure for ZM, and ZW (ZW was actually overall flat, but orders are not executing anymore because of the physical delivery rule: so I just made the trading "information transfer")

Note also that this the IB documentation about Physical Delivery Rules:
http://ibkb.interactivebrokers.com/article/992

Liquidations and anything occurring in the account on IB side should be reported immediately via API. Especially time, order price/size/commission and reason for execution.

So, anyway, as you see, it's good to have complete flexibility in the operations related to rollover or liquidations (which can occur for several reason, like margin issues, close-out deadlines, etc).

 

[fullautotrading]

Let's continue our review of instruments while we are still "underwater" (about -2.47% respect to our initial capital) but anyway with a good "potential" for the longer term result (G-L is currently over 200K). We have taken a look so far at CL, ZI and ZL.

Let's now take a look at ZM (now ZM MAR 16) which we have rolled over after the IB liquidation. This is the only automated instrument in our entire folio which so far is still using using the initial "corridor". In fact, while it dropped a bit, actually the DD was not as large as to scream for the creation of a new trading "corridor".

Overall (over the 3 layers on the initial "corridor") it still has a positive balance and, in particular, its 3 layers have developed the following order clouds:

which means that it has managed to make a profit even if the instrument has been going steadily down (due to enough price fluctuations).

 

[fullautotrading]

This morning I have taken care of the NKD rollover, by moving 8 layers to the new contract NKDH6, so we don't have to do anything about it for another 3 months.

It the meantime CL and NG, have been pulling down quite a bit simultaneously to the mkt reaching up, which caused a sharper DD (as noted before, we have too many instruments active for the mkt component, and I am looking forward to eliminate NQ and YM). We almost reached a -7% on the sharpest negative peak. However, the situation is not worrying so far: pretty much in line with expectations.

In the meantime, as anticipated in the previous post, I have begun to place some out-of-game "enqueued orders" (that is, orders scheduled to execute at given prices) in order to ride the "structural" drift of the ETFs, which can provide an hedging action.

So far I am considering UVXY, VXX, ERY to edge the mkt drift. And for CL and NG, I am using SCO and DGAZ respectively. If I have a chance (price peaking up) [I would also like to hedge the contango component, in which case I might also use UCO, UGAZ].