What is profit factor, std of losses, std of wins, average win, average loss?
Trading algorithmically a folio without stops (with IB), real $$$
- Thread starter fullautotrading
- Start date
Hi tradingjournals. Thank you for your interest.
I am checking the terminology.
<b>win</b> = profit in a "trade" which "ends up" with a profit
<b>loss</b> = loss in a "trade" which is "stopped" with a loss
<b>average win</b> = arithmetic average of the "wins"
<b>average loss</b> = arithmetic average of the "losses"
<b>std of wins</b> = standard deviation of the "wins"
<b>std of losses</b> = standard deviation of the "losses"
<b>profit factor</b> = ratio obtained by dividing the sum of all wins by the sum of all losses and commissions
where by <b>"trade"</b> it is meant an ordered pair of orders, say (buy/sell) or (sell/buy), and the sequence of wins/losses is normally intended as taken on temporally consecutive non-overlapping trades,
and by <b>profit</b> or <b> loss</b> it is meant the following quantity: (sell price - buy price) * Instrument Multiplier - commissions, where (sell, buy) are the 2 orders in a "trade", as defined above.
Can you confirm if these are correct definitions, or you can correct or make them more precise, so I can look for an answer to your question ?
Seems correct, but make sure each trade is equal size, or adjust to same dollar amount. If size changes, say so and report numbers with and without adjustment.
>where by "trade" it is meant an ordered pair of orders, say (buy/sell) or (sell/buy), and the sequence of wins/losses is normally intended as taken on temporally consecutive non-overlapping trades,
>>Seems correct
If the above definitions are correct, in order for me to proceed, you need to advise what a "<b>trade</b>" is in my architecture. In fact, if we take the above definition - which is perfectly suitable for naive architectures - a "trade" is <b>not</b> defined in my context, because orders are issued by virtual players and pooled into physical orders, and they have all kind of temporal overlapping.
[Of course, I could assume that by "trade" you want to mean the "open" and "close" of a virtual player. But I had to exclude that possibility because, a player is architecturally constrained to close with a profit only. Since we already know that, from the previous discussion, I have to assume that you are not interested in that, or else you would not have made your question.]
So, in order to be able to answer to your curiosity, you need to define what a "trade" is in this architecture. Once you have done that, it is possible to proceed with the avgs, stds and the ratio computations over the "trades".
I am enclosing a file (a compressed .rar file containing an .xml that can be normally open by Excel) with a possible example of orders issued by the player superposition or manual injection on a layer, and I need you to provide a unambiguous criterion to <b>define a "trade"</b> and to list the "trades". So when have some time, think a bit about that, and let me know what definition you come about.
Once that is done, it's clearly no problem to compute win and losses and make the elementary statistics you indicate, on the "trades" which are identified.
Let's see how the folio is evolving.
Last folio shot we watched here (http://www.elitetrader.com/vb/showthread.php?s=&threadid=279057&perpage=6&pagenumber=6) we had a net gain (difference between gains and losses) of about 7.5K, about $185 per day.
In the meantime I have expanded the portion of the capital (201K) traded up so that the margins now take in avg about half of the capital.
At this point we have 90 layers active, 15.8K of "net gain"(about $272 per day), which is currently "invested" in a number of instruments, pushing the PNL to about -12K. First of all GASX, which at moment holds a pretty large position (in relative terms), VXX, ZSL, SLV and others.
Almost 2.5K commissions in 2 months and more that 210,000 shares moved, should entitle us to some commissions discount from IB
The "relative drift" is currently RD = 19.43%, while HI = 83.75% and II = 35.68%. Which seems to indicate that the current "investment" is a bit too aggressive (HI should be higher).
<img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3914529" />
FAS is still "locked" with the option unit, and we need to wait for some development over there (either options' expiry or the price hitting the bottom strike, 72).
The one which right now needs to watched closely is GASX, which either unloads some of those shares, or we need to put in place some defense mechanism.
The pictures above do not include the layers which have been removed because positive and with no players open (their PNLs are accounted for anyway in the above pictures, as explained in a previous post). They are:
Removed layers: <code>
XOP STK SMART PNL: 38.63, comms: 4.35
XLB STK ARCA PNL: 14.47, comms: 3.03
XHB STK ARCA PNL: 40.43, comms: 8.81
VXX STK SMART_USD PNL: 41.18, comms: 6.35
UVXY STK SMART PNL: 206.42, comms: 14.74
TQQQ STK SMART PNL: 195.70, comms: 28.60
SQQQ STK SMART_L1 PNL: 26.52, comms: 4.10
SQQQ STK SMART PNL: 22.32, comms: 4.58
QLD STK SMART PNL: 17.84, comms: 4.38
PNR STK SMART_USD PNL: 6.98, comms: 1.42
OIH STK SMART PNL: 5.08, comms: 1.52
IYR STK SMART_L1 PNL: 12.26, comms: 1.54
GASX STK SMART_L3 PNL: 479.81, comms: 6.39
FAS STK SMART_USD PNL: 28.43, comms: 5.91
ERX STK SMART_L2 PNL: 222.30, comms: 7.20
EEM STK ARCA_USD_L1 PNL: 6.62, comms: 1.38
DIA STK SMART PNL: 5.02, comms: 1.46
CVX STK SMART PNL: 9.82, comms: 1.46
FAS OPT 20131122 81 C SMART 100 PNL: 236.93, comms: 1.07
FAS OPT 20131122 88 C SMART 100 PNL: -207.65, comms: 3.65
UVXY STK SMART PNL: 20.39, comms: 2.07
</code>
In next posts, I will show in detail some instrument and "order clouds".
I am going to post some "order clouds" to convey a feeling of what the application is doing.
Today we had some serious moves, starting from about 14:00 and the bot almost got "crazy" sending out some hundreds orders (some difficulties to lock some profits were found with some instruments flagged as "not shortable").
Anyway, that was good as it accelerated a bit the scalping process. This has pushed up the PNL of 5-6K, in the matter of few minutes.
Commissions are now almost 3K and it has moved over 255.000 shares, with a "net gain" in excess of 20K.
Here is a picture with the folio evolution (92 layers currently active). In the next post I am going to discuss a bit the GASX situation, which is absorbing a large part of the current investment (over 2K shares) (in addition, of course, to the FAS "lockup", which is instead under control, even though the mkt spiked up violently: Dow and S&P 500 to all-time closing highs).
<img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3916185" />
Today we had some serious moves, starting from about 14:00 and the bot almost got "crazy" sending out some hundreds orders (some difficulties to lock some profits were found with some instruments flagged as "not shortable").
Anyway, that was good as it accelerated a bit the scalping process. This has pushed up the PNL of 5-6K, in the matter of few minutes.
Commissions are now almost 3K and it has moved over 255.000 shares, with a "net gain" in excess of 20K.
Here is a picture with the folio evolution (92 layers currently active). In the next post I am going to discuss a bit the GASX situation, which is absorbing a large part of the current investment (over 2K shares) (in addition, of course, to the FAS "lockup", which is instead under control, even though the mkt spiked up violently: Dow and S&P 500 to all-time closing highs).
<img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3916185" />
Here is an example of <b>order cloud</b>. This is one of the 92 layers currently active, and is in particular my favorite instrument VXX. Note that is meaningless to try to talk in terms of "trades": we, more in general, reason in terms of "order cloud".
Note that the second picture below is the same screenshot, but with <b>"Player-view" mode</b> checked.
Personally, I almost always use the "Player-view" mode because it allows me to <b>follow only the players who are active</b> (I also use a multitude of "tricks", which I will explain later in this thread). In fact we can forget about the past orders and players, because we are guaranteed they all closed in profit (architectural constraint).
You see there those 3 T Buy players "stranded" above. They sure provided some protection" or hedging when the application was loading a short position the way up (you might think of them as "stop loss orders" if you want to attempt reconciling this with a more naive approach). But note that <b>they too</b>, sooner or later <b>will be closed in profit</b>. So the "stops" while provide protection, do not necessarily come with a loss.
<img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3916216" />
That is the essence of the method, which is relying for its edge on the <b>temporary "losses" systematically recycled into "gains"</b> (a necessary, but <b>not</b> sufficient, condition).
So, as you see, it is a continuous scalping and hedging action ("game"), where, when it is needed, we "superpose" players in protection, but later on, when the "danger" is passed, are turned themselves into profits (if possible) if the prices passes again near them.
Note that the second picture below is the same screenshot, but with <b>"Player-view" mode</b> checked.
Personally, I almost always use the "Player-view" mode because it allows me to <b>follow only the players who are active</b> (I also use a multitude of "tricks", which I will explain later in this thread). In fact we can forget about the past orders and players, because we are guaranteed they all closed in profit (architectural constraint).
You see there those 3 T Buy players "stranded" above. They sure provided some protection" or hedging when the application was loading a short position the way up (you might think of them as "stop loss orders" if you want to attempt reconciling this with a more naive approach). But note that <b>they too</b>, sooner or later <b>will be closed in profit</b>. So the "stops" while provide protection, do not necessarily come with a loss.
<img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3916216" />
That is the essence of the method, which is relying for its edge on the <b>temporary "losses" systematically recycled into "gains"</b> (a necessary, but <b>not</b> sufficient, condition).
So, as you see, it is a continuous scalping and hedging action ("game"), where, when it is needed, we "superpose" players in protection, but later on, when the "danger" is passed, are turned themselves into profits (if possible) if the prices passes again near them.
Hi Fullautotrading,
thank you for posting the portfolio update and explaining about the order cloud.
Could you please comment more on your note
> some difficulties to lock some profits were found with some
> instruments flagged as "not shortable").
Could you give an example of instruments not shortable (was it a temporary reason or a permanent one)?
Is G-bot aware about them not being shortable before starting trading them?
or will it discover the case only when trying to short them?
Finally, why are you also considering not shortable instruments?
If one can only be long them, isn't scalping seriously limited?
Thanks!
thank you for posting the portfolio update and explaining about the order cloud.
Could you please comment more on your note
> some difficulties to lock some profits were found with some
> instruments flagged as "not shortable").
Could you give an example of instruments not shortable (was it a temporary reason or a permanent one)?
Is G-bot aware about them not being shortable before starting trading them?
or will it discover the case only when trying to short them?
Finally, why are you also considering not shortable instruments?
If one can only be long them, isn't scalping seriously limited?
Thanks!
Hi AItrader,
For a list of instruments available for shorting see:
https://www.interactivebrokers.com/...g=United%2520States&ib_entity=llc&ln=
Note that : "Clients who short instruments pay a fee expressed as a reduced <b>interest</b> on the sale proceeds."IB
Practically any instrument can be not shortable (not all instrument are shortable anyway). You see for instance, from my screenshot in the previous post, that even VXX (a very liquid instrument) was in that condition yesterday. So if one wants to trade ETFs, I am afraid he has to learn live with these facts. Take it as yet another form of "execution risk".
Not only, but be also aware that " Technically <b>you can get bought in anytime</b> after the 3rd day from the short sale of a stock. "IB
See here: http://ibkb.interactivebrokers.com/article/845
See also: https://www.interactivebrokers.com/en/index.php?f=commission&p=stocks3
for IB Pre-Borrow program. "The IB Pre-Borrow program allows customers to <b>pre-borrow stocks in anticipation of a short sale to decrease the chances of being bought-in on settlement date</b>." IB
The application can be set to skip buy or sell entries when there is a condition of non shortability. Anyway, the condition is usually temporary and hopefully will not hurt very much a large and diversified folio. However, one must be aware of these facts. Inverse ETFs are very often in such condition of limited or no shortability. (Well, they have also other specific features, like a marked drift due to decay, which must be carefully considered.)
Futures don't have this specific issue (they have other difficulties and problems though, as we will see).
So, contrary to the naive belief, algorithmic trading does <b>not</b> happen like growing the <b>Pinocchio's "money tree"</b> while one is relaxing at the beach. It's anyway hard work, getting informed, and dealing every day with new hardships and situations.
People sometimes ask me what is the <b>CAGR</b> of the procedure, just like one was a sort of retiree enjoying his steady pension wire/check. I just smile. If one has such a naive mindset and expectation as to conceive that the complexity and risk of algorithmic trading can be reduced to a CAGR, just go buy gov bonds and hope inflation will not eat you alive
(It's hard work, lifetime study and risk, in any case.)<b>Inside a "player"</b>
Let's make a little divagation on the "theory" of the "players", providing some basic information useful for a fund manager. So far you have probably got an approximate idea of what a "(virtual) player" is. One way to picture it, intuitively, is to imagine a trader using an account while other traders (other "players") are also using it, each one pursuing his own profit, but also each one "directed" by the application to behave in such a way that it provides protection" and defense against the actions of the other players.
In more abstract terms, a "player" is just, I would say, an "accounting unit". There is, of course, no one-to-one correspondence between players and orders.
When a players opens, there is 1 "open" order associated with it. When a player closes, its position can go "fragmented" into multiple partial fills in different "closing" orders. And each order might collect "contributions" or "requests" of multiple players.
On the other hand, an order can be a result of the pooling of multiple requests from players (open/close requests), so it can be "associated" with multiple players.
From the point of view of the trading strategy, we could totally disregard the orders and just reason only in terms of players.
How a player maintains its "status" ?
Each single player maintains a lot of information inside of it about its "status". Simplifying, the most crucial piece of information are:
<code>
"PosAbs" := Current (absolute) Position
"Avg" := Current Average
</code>
Both these quantities vary in time due to various reasons:
- When the player is open, the player receives the fill events of the opening order
and while they arrive both PosAbs and Avg, change instantaneously at each fill event.
- When the player closes, there could be multiple orders (partial fills) which discharge the players.
In addition, each of these orders, may have multiple fill events.
So the situation, in general is that Position and Average of a player may change by effect of 2 most basic reasons:
- Add/Open events
- Remove/Close events
For each of these events, both "PosAbs" and "Avg" are updated instantaneously, according to the following:
<b>Add/Open event</b>
<code>
Avg = (Avg * PosAbs + Fill * AvgFillPrice ) / ( PosAbs + Fill )
PosAbs = PosAbs + Fill
</code>
<b>Remove/Close event</b>
Same thing, but with negative sign for the contribution (Fill * AvgFillPrice):
<code>
Avg = (Avg * PosAbs - Fill * AvgFillPrice ) / ( PosAbs - Fill )
PosAbs = PosAbs - Fill
</code>
Where the Fill and the new position is assumed > 0.
Another factor influencing the realtime value of the Avg, are the <b>rollovers</b> (futures). In order for you to possibly benefit (or adjust if unfavorable) of <b>contango or backwardation</b> (if you are short/long) the application automatically translates active players to reflect the price difference at rollover.
In this case, we have this update rule:
<code>
Avg = Avg + Contango/Backwardation PriceDifference
PosAbs = unchanged
</code>
where: PriceDifference := New instrument price - Old instrument price
The <b>rollover adjustment</b> is stored separately within the player, and incrementally modified at <b>each</b> rollover. This value is constantly used to adjust the current average. So no loss can escape due to rollovers.
Finally, let's make a remark, which will have parallels when we discuss the math of the player superposition.
Let's examine the update rules for the Average for a remove/close event:
<code>
Avg = (Avg * PosAbs - Fill * AvgFillPrice ) / ( PosAbs - Fill )
</code> if we denote:
NewAbsPos := PosAbs - Fill
we can write the updated Avg for a "close/remove" event (which would be a "sell") as:
<code>
(Avg * PosAbs - Fill * AvgFillPrice ) / ( PosAbs - Fill )
= (Avg * (NewAbsPos + Fill) - Fill * AvgFillPrice ) / NewAbsPos
= (Avg * NewAbsPos + Avg * Fill - Fill * AvgFillPrice ) / NewAbsPos
= Avg + Fill * (Avg - AvgFillPrice ) / NewAbsPos
</code>
so the updated Avg is actually : <pre>
Fill
Avg - (AvgFillPrice - Avg ) -----------
NewAbsPos </pre>
Imagine this were a BUY Player. This expression is saying that when it partially closes
by getting rid (selling) of "Fill" units, its average is <b>decreased</b> by an amount which is proportional
to the size of the sell and which is larger when the AvgFillPrice is larger than the current Avg.
<img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3916386" />
In other words, the higher we sell, the lower gets the Avg of the BUY Player. This is of course very intuitive.
In case it were a SELL player, we had similarly (opposite sign):
<code>
(Avg * PosAbs + Fill * AvgFillPrice ) / ( PosAbs + Fill )
= (Avg * (NewAbsPos - Fill) + Fill * AvgFillPrice ) / NewAbsPos
= (Avg * NewAbsPos - Avg * Fill + Fill * AvgFillPrice ) / NewAbsPos
= Avg - Fill * (- Avg + AvgFillPrice) / NewAbsPos
</code><pre>
Fill
Avg + (Avg - AvgFillPrice) -----------
NewAbsPos
</pre> which intuitively means, that the lower we buy (to reduce/close it), the higher gets the player's Avg. That is, when the SELL player partially closes by getting rid (buying) of "Fill" units, its average is <b>increased</b> by an amount which is proportional to the size of the buy and which is larger when the AvgFillPrice is smaller than the current Avg.
In general terms we have: the Avg improves (it goes <b>down</b> for a BUY / goes <b>up</b>, for a SELL) of an amount proportional to the size of the fill and the "favorable size" of the scalp. A totally intuitive thing.
Let's make a little divagation on the "theory" of the "players", providing some basic information useful for a fund manager. So far you have probably got an approximate idea of what a "(virtual) player" is. One way to picture it, intuitively, is to imagine a trader using an account while other traders (other "players") are also using it, each one pursuing his own profit, but also each one "directed" by the application to behave in such a way that it provides protection" and defense against the actions of the other players.
In more abstract terms, a "player" is just, I would say, an "accounting unit". There is, of course, no one-to-one correspondence between players and orders.
When a players opens, there is 1 "open" order associated with it. When a player closes, its position can go "fragmented" into multiple partial fills in different "closing" orders. And each order might collect "contributions" or "requests" of multiple players.
On the other hand, an order can be a result of the pooling of multiple requests from players (open/close requests), so it can be "associated" with multiple players.
From the point of view of the trading strategy, we could totally disregard the orders and just reason only in terms of players.
How a player maintains its "status" ?
Each single player maintains a lot of information inside of it about its "status". Simplifying, the most crucial piece of information are:
<code>
"PosAbs" := Current (absolute) Position
"Avg" := Current Average
</code>
Both these quantities vary in time due to various reasons:
- When the player is open, the player receives the fill events of the opening order
and while they arrive both PosAbs and Avg, change instantaneously at each fill event.
- When the player closes, there could be multiple orders (partial fills) which discharge the players.
In addition, each of these orders, may have multiple fill events.
So the situation, in general is that Position and Average of a player may change by effect of 2 most basic reasons:
- Add/Open events
- Remove/Close events
For each of these events, both "PosAbs" and "Avg" are updated instantaneously, according to the following:
<b>Add/Open event</b>
<code>
Avg = (Avg * PosAbs + Fill * AvgFillPrice ) / ( PosAbs + Fill )
PosAbs = PosAbs + Fill
</code>
<b>Remove/Close event</b>
Same thing, but with negative sign for the contribution (Fill * AvgFillPrice):
<code>
Avg = (Avg * PosAbs - Fill * AvgFillPrice ) / ( PosAbs - Fill )
PosAbs = PosAbs - Fill
</code>
Where the Fill and the new position is assumed > 0.
Another factor influencing the realtime value of the Avg, are the <b>rollovers</b> (futures). In order for you to possibly benefit (or adjust if unfavorable) of <b>contango or backwardation</b> (if you are short/long) the application automatically translates active players to reflect the price difference at rollover.
In this case, we have this update rule:
<code>
Avg = Avg + Contango/Backwardation PriceDifference
PosAbs = unchanged
</code>
where: PriceDifference := New instrument price - Old instrument price
The <b>rollover adjustment</b> is stored separately within the player, and incrementally modified at <b>each</b> rollover. This value is constantly used to adjust the current average. So no loss can escape due to rollovers.
Finally, let's make a remark, which will have parallels when we discuss the math of the player superposition.
Let's examine the update rules for the Average for a remove/close event:
<code>
Avg = (Avg * PosAbs - Fill * AvgFillPrice ) / ( PosAbs - Fill )
</code> if we denote:
NewAbsPos := PosAbs - Fill
we can write the updated Avg for a "close/remove" event (which would be a "sell") as:
<code>
(Avg * PosAbs - Fill * AvgFillPrice ) / ( PosAbs - Fill )
= (Avg * (NewAbsPos + Fill) - Fill * AvgFillPrice ) / NewAbsPos
= (Avg * NewAbsPos + Avg * Fill - Fill * AvgFillPrice ) / NewAbsPos
= Avg + Fill * (Avg - AvgFillPrice ) / NewAbsPos
</code>
so the updated Avg is actually : <pre>
Fill
Avg - (AvgFillPrice - Avg ) -----------
NewAbsPos </pre>
Imagine this were a BUY Player. This expression is saying that when it partially closes
by getting rid (selling) of "Fill" units, its average is <b>decreased</b> by an amount which is proportional
to the size of the sell and which is larger when the AvgFillPrice is larger than the current Avg.
<img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3916386" />
In other words, the higher we sell, the lower gets the Avg of the BUY Player. This is of course very intuitive.
In case it were a SELL player, we had similarly (opposite sign):
<code>
(Avg * PosAbs + Fill * AvgFillPrice ) / ( PosAbs + Fill )
= (Avg * (NewAbsPos - Fill) + Fill * AvgFillPrice ) / NewAbsPos
= (Avg * NewAbsPos - Avg * Fill + Fill * AvgFillPrice ) / NewAbsPos
= Avg - Fill * (- Avg + AvgFillPrice) / NewAbsPos
</code><pre>
Fill
Avg + (Avg - AvgFillPrice) -----------
NewAbsPos
</pre> which intuitively means, that the lower we buy (to reduce/close it), the higher gets the player's Avg. That is, when the SELL player partially closes by getting rid (buying) of "Fill" units, its average is <b>increased</b> by an amount which is proportional to the size of the buy and which is larger when the AvgFillPrice is smaller than the current Avg.
In general terms we have: the Avg improves (it goes <b>down</b> for a BUY / goes <b>up</b>, for a SELL) of an amount proportional to the size of the fill and the "favorable size" of the scalp. A totally intuitive thing.