Btw, don't be fooled by the number of position. We are just warming up. Sizes have been very small so far, and we have been using no more than 5K margins (globally) most of the time, and we will gradually ramp up.
Trading algorithmically a folio without stops (with IB), real $$$
- Thread starter fullautotrading
- Start date
I have been totally overwhelmed coding new ideas and features this weekend. Anyway, let me give you some initial ideas about some hedging technique, so that you can sleep on them, and then provide some feedback and discussion.
I have mentioned "superposition" and "layering". Let me introduce the basic ideas.
We have mentioned that our goal is to favor the scalping action against the runaways. This means that we cannot let an instrument "run away" too much or else it would unbalance the folio and spoil forever the whole action.
On the other hand, if every time we have a significant unfavorable move we "stop it and forget about it", it's also a sure way no to make a dime (and possibly <b>stack up infinite losses</b>).
Now, I have quickly drawn a simple picture on a piece of paper which should illustrate in a simple way the idea of "layering":
<img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3892828" />
On the left is what one would do just "stopping" and then continuing trading. On the right we use a second layer to block the runaway.
It looks like that the 2 are doing something similar or possibly the same thing. However, there is a crucial difference both in the philosophy and in the practical application. In the first case the common approach is to run after some "signal" which one believes to identify in the mkt, thus essentially "forgetting" about the losses which have been realized. (Forgetting losses and keep hoping is also a psychological need for some beginner traders.)
In the second case, there are 2 positions (on the two layers) which remain "open" and that are, each one waiting to be closed in profit.
This leads naturally to the concept of trading "<b>players</b>". But I will introduce them in the next installment. For now, think about the main message of this post (and if it's really necessary to have the 2 physical "layers" or the thing can actually be also "projected" on a unique layer: this will lead to the concept of <b>player superposition</b>).
I have mentioned "superposition" and "layering". Let me introduce the basic ideas.
We have mentioned that our goal is to favor the scalping action against the runaways. This means that we cannot let an instrument "run away" too much or else it would unbalance the folio and spoil forever the whole action.
On the other hand, if every time we have a significant unfavorable move we "stop it and forget about it", it's also a sure way no to make a dime (and possibly <b>stack up infinite losses</b>).
Now, I have quickly drawn a simple picture on a piece of paper which should illustrate in a simple way the idea of "layering":
<img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3892828" />
On the left is what one would do just "stopping" and then continuing trading. On the right we use a second layer to block the runaway.
It looks like that the 2 are doing something similar or possibly the same thing. However, there is a crucial difference both in the philosophy and in the practical application. In the first case the common approach is to run after some "signal" which one believes to identify in the mkt, thus essentially "forgetting" about the losses which have been realized. (Forgetting losses and keep hoping is also a psychological need for some beginner traders.)
In the second case, there are 2 positions (on the two layers) which remain "open" and that are, each one waiting to be closed in profit.
This leads naturally to the concept of trading "<b>players</b>". But I will introduce them in the next installment. For now, think about the main message of this post (and if it's really necessary to have the 2 physical "layers" or the thing can actually be also "projected" on a unique layer: this will lead to the concept of <b>player superposition</b>).
And here is a real world example (these are real trades on NG, tickdata is actual and current).
<img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3892832" />
(Imagine the difference, if we have simply "stopped and forgot" when the long position (2 contracts) on the first layer was painfully running away...)
<img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3892832" />
(Imagine the difference, if we have simply "stopped and forgot" when the long position (2 contracts) on the first layer was painfully running away...)
At this point it becomes clear that in order to play a (statistically) consistently profitable trading game we need to organize ourselves in a way which allows us no to realize losses in a systematic manner and we need a proper architecture. This is done in my approach by first introducing the concept of <b>"Player"</b>.
So let's see some more concepts and terminology.
A player is simply a "virtual trader" which at any instant of time retains all useful information about itself.
A number of players on a layer play a <b>scalping-hedging game</b>.
Players come in 2 types (hedging one another):
<b>T</b> = a "Trader"
<b>CT</b> = a "CounterTrader"
and roles: BUY/SELL. So that we have in all 4 types and roles: T BUY, CT Sell, T Sell, CT Buy. See picture:
<img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3894654" />
By <b>Player Superposition</b> it is meant the appropriate use of T and CT player types to give rise to statistically optimal scalping and hedging effects on a layer.
By <b>Layer Overlay</b> it is meant the appropriate use of layers in order to give rise to statistically good extra hedging effects.
In the layer overlay hedging computations, also current status and "relationships" across layers (<b>cointegrations</b> and <b>volatility-adjusted-exposures</b> are taken into account, to supervise centrally the players.) The layers can contain either different instruments or "clones" of same instruments.
(We will see some analytic details about exposures and "codirections" in in the next posts.)
So let's see some more concepts and terminology.
A player is simply a "virtual trader" which at any instant of time retains all useful information about itself.
A number of players on a layer play a <b>scalping-hedging game</b>.
Players come in 2 types (hedging one another):
<b>T</b> = a "Trader"
<b>CT</b> = a "CounterTrader"
and roles: BUY/SELL. So that we have in all 4 types and roles: T BUY, CT Sell, T Sell, CT Buy. See picture:
<img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3894654" />
By <b>Player Superposition</b> it is meant the appropriate use of T and CT player types to give rise to statistically optimal scalping and hedging effects on a layer.
By <b>Layer Overlay</b> it is meant the appropriate use of layers in order to give rise to statistically good extra hedging effects.
In the layer overlay hedging computations, also current status and "relationships" across layers (<b>cointegrations</b> and <b>volatility-adjusted-exposures</b> are taken into account, to supervise centrally the players.) The layers can contain either different instruments or "clones" of same instruments.
(We will see some analytic details about exposures and "codirections" in in the next posts.)
Only "virtually" (purely in the application "imagination" and mathematics).
For instance, If you hypothetically had (only) 1 CY BUY and 1 T SELL, your "physical" account will have 0, of course.
See the pictures with the NG example before. Assuming that only these 2 layers were working and only 1 program instance running, after the 2 long contracts on the first layer there was a sell order on the second layer, so the "physical" account was reduced to 1 (by the second layer). However the first layer still was "seeing" its 2 contracts and behaving accordingly.
It's a purely virtual (mathematical) game, allowing us to keep in memory information to avoid "realizing losses", as one often does by using the ordinary <b>"memoryless" stops</b>, when "running" after delusional "signals" (thus giving rise to statistically inefficient "order clouds").
This is a picture showing a possible order cloud generated by <b>player superposition</b> on a single layer:
<img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3894676" />
[ This particular picture is taken from the "game analysis" tool, a facility to study the statistical efficiency of the player superposition and layer overlay. In real application the situation is exactly the same.]
Simulations indicates that by increasing the frequency of a statistically good game, we usually can achieve a superior smoothing of the equity curve. The higher frequency tends to make "irrelevant" the random price curve fed into the game rules.
<img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3894676" />
[ This particular picture is taken from the "game analysis" tool, a facility to study the statistical efficiency of the player superposition and layer overlay. In real application the situation is exactly the same.]
Simulations indicates that by increasing the frequency of a statistically good game, we usually can achieve a superior smoothing of the equity curve. The higher frequency tends to make "irrelevant" the random price curve fed into the game rules.
Now let's proceed explaining a bit more the architecture, and the motivation. There are 2 main aspects that we need to address.
1. Player superposition
- How it works, and why we need it.
2. Layer overlay.
- Why we need this too and how do we coordinate the different layers by using the instrument real time relationships (correlations, "codirections", etc) and volatility-adjusted exposures, for optimal hedging.
But while we do that, let's not forget what is our objective. Let's repeat it because it's obviously the most fundamental thing. We want to generate a massive "scalping action" in such a way that it would at the same time realize as much as possible and hedge the runaway instruments. In more formal terms, if we think of our PNL decomposition:
PNL = Realized gain + Realized losses + Unrealized
(I am assuming the various quantities are taken with sign), what we want to do is to first of all maximize the Realized gain, then we want to use the "temporary" Realized Losses which are generated in the process for hedging the third component ("Unrealized"), which in intuitive terms expresses what is being invested in new (possibly large) moves. Why the (old) Realized Losses are imagined as "temporary" ? Well simply because we don't really "realize" losses at player lever, so in time, due to multiple passage on the same prices, most of the open players will be anyway closed in profit (remember that single players only take positive profits, by definition).
The point is that, at the same time we close some players, new ones open up. So while the past losses have "temporary" character (and most of them will be recovered on retracements), the bad news is that new ones are also been continuously generated in the trading process.
So, essentially everything boils down to a "tug of war" between the 2 <b>growth rates</b>: that of the "Realized gain", on one side, and the one of the (Realized Losses + Unrealized) on the other side.
So what is necessary is to create frequent and tight scalping-hedging games which have the <b>statistical</b> property to favor the first growth rate, against the second one. Clearly we will never able to have certainty, because the prices come in as a (n ever changing) random process, however we can put in place games with "good" statistical behavior. So that using diversification we can hope in a good outcome.
In terms of equity curve, this could be visualized as in the following picture.
<img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3895981" />
You see here the PNL and the 3 components of the PNL decomposition dynamically at work. While the old losses (in red) are recovered and transformed into realized gain when the prices allows the old ("stranded") players to close, new players are created which continue the hedging action (creating the corresponding new "temporary losses").
The goal is to devise games which keep all this in balance but with a steady statistically <b>positive drift</b> for the resulting PNL. In other words, the realized gain must be such to be able to "pull up" the PNL, in dynamic competition against the realized losses and the unrealized.
Creating such scalping-hedging games clearly becomes the challenge, but at least we are putting ourselves in a position to aspire for the most "efficient" $$$ extraction process. One which, at least, does not take "memoryless" stops.
So at least we get the "right" architecture. Next problem is to put in place the right superposition and hedging mechanisms.
[ Any other trading procedure taking (ordinary) stops way will necessary be dominated by this kind of architecture, and usually the amount of inefficiency they embed by taking those "memoryless" losses is so high that they are doomed even before starting. No matter what else is used in the rest of the system ("signals", fancy modeling or whatever. ]
1. Player superposition
- How it works, and why we need it.
2. Layer overlay.
- Why we need this too and how do we coordinate the different layers by using the instrument real time relationships (correlations, "codirections", etc) and volatility-adjusted exposures, for optimal hedging.
But while we do that, let's not forget what is our objective. Let's repeat it because it's obviously the most fundamental thing. We want to generate a massive "scalping action" in such a way that it would at the same time realize as much as possible and hedge the runaway instruments. In more formal terms, if we think of our PNL decomposition:
PNL = Realized gain + Realized losses + Unrealized
(I am assuming the various quantities are taken with sign), what we want to do is to first of all maximize the Realized gain, then we want to use the "temporary" Realized Losses which are generated in the process for hedging the third component ("Unrealized"), which in intuitive terms expresses what is being invested in new (possibly large) moves. Why the (old) Realized Losses are imagined as "temporary" ? Well simply because we don't really "realize" losses at player lever, so in time, due to multiple passage on the same prices, most of the open players will be anyway closed in profit (remember that single players only take positive profits, by definition).
The point is that, at the same time we close some players, new ones open up. So while the past losses have "temporary" character (and most of them will be recovered on retracements), the bad news is that new ones are also been continuously generated in the trading process.
So, essentially everything boils down to a "tug of war" between the 2 <b>growth rates</b>: that of the "Realized gain", on one side, and the one of the (Realized Losses + Unrealized) on the other side.
So what is necessary is to create frequent and tight scalping-hedging games which have the <b>statistical</b> property to favor the first growth rate, against the second one. Clearly we will never able to have certainty, because the prices come in as a (n ever changing) random process, however we can put in place games with "good" statistical behavior. So that using diversification we can hope in a good outcome.
In terms of equity curve, this could be visualized as in the following picture.
<img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3895981" />
You see here the PNL and the 3 components of the PNL decomposition dynamically at work. While the old losses (in red) are recovered and transformed into realized gain when the prices allows the old ("stranded") players to close, new players are created which continue the hedging action (creating the corresponding new "temporary losses").
The goal is to devise games which keep all this in balance but with a steady statistically <b>positive drift</b> for the resulting PNL. In other words, the realized gain must be such to be able to "pull up" the PNL, in dynamic competition against the realized losses and the unrealized.
Creating such scalping-hedging games clearly becomes the challenge, but at least we are putting ourselves in a position to aspire for the most "efficient" $$$ extraction process. One which, at least, does not take "memoryless" stops.
So at least we get the "right" architecture. Next problem is to put in place the right superposition and hedging mechanisms.
[ Any other trading procedure taking (ordinary) stops way will necessary be dominated by this kind of architecture, and usually the amount of inefficiency they embed by taking those "memoryless" losses is so high that they are doomed even before starting. No matter what else is used in the rest of the system ("signals", fancy modeling or whatever. ]
So, yes, the key is to devise strategies that have an edge over time. Otherwise, Realized Gains + Realized Losses + Unrealized will still sum to a negative number and the line will slope down instead of nicely upwards.
Forgive my intelectual sluggishness, but I don't get why it is insightful to think that, instead of an exit, a given trade is a new player entering a new game in the opposite sense. What's the difference?
Thanks.
Yes sure, thank you for asking abattia.
The conceptual point is that what on "physical" account will possibly "look like" stop orders, to an external observer (and to the MMs), are instead all "kept in memory" by the application, which if possible at a later time will "resolve" as positive take profits (thus continuously "throwing" the losses dynamically into the "realized gain" component).
In other words, the orders you enter to "hedge", will later be transformed into positive scalps if possible (that is when the price retraces enough). See the picture I posted with the 2 layers drawing.
The difference with the ordinary approach is that one would not keep track of the stopping orders (in this sense I am calling them "memoryless"), thus introducing large inefficiencies in the order cloud creation (in the grand scheme of things).
In my explanation I used 2 layers to make the point clearer, but, in reality, the application does all with a unique layer (per instrument), thus by creating a "superposition" (and channeling multiple player request into "pool" orders).
(Multiple layers are still used, but for different reasons, which I will discuss later. Essentially when you wish to hedge by using a correlated instrument, or starting a new scalping engine at a different price level.)