Trading algorithmically a folio without stops (with IB), real $$$

[DR.BOUYE]

Dear Tom,

I have been following your posts for years now, going back to when you were first simulation testing this thing. I admire all your work and I think the software you have created is really impressive. However, I feel obligated to tell you that I think there may be no edge in your current strategy, and that is why you are now losing money.

I doubt the widely available scalping and correlation indicators you use are themselves a source of edge. As to the game, there is no reason to believe that having two players is any better than one player who can essentially trade the same overall strategy -- obviously you are free to code up arbitrarily complex logic in trading the account, regardless of the number of virtual players. This means, for example, that remembering stops is not in and of itself a source of edge, and that if anything, having the two players at all independent can only hurt you vs. one player with fully coordinated effort.

I think the main problem is that you have convinced yourself that the players waiting around for mean reversion (or really, price reversion to previous levels) is in fact a source of edge. You repeatedly make statements taking this reversion for granted. The problem is, there are no guarantees, so you are likely to make a lot of money on the scalping until such time as you take large losses, where some of the instruments "run away." For example, suppose you had been trading gold - would it be reasonable to include in your "unrealized PNL" the profits that might come from price going all the way back up to prior levels? Those levels may never come back... or it may take longer than the horizon of your investors. In particular, I think your "unrealized PNL" should really just be called "losses", because it is essentially a tally of losses incurred from prices not reverting to previous levels. Your true PNL is in fact the liquidation PNL you show in the charts, which is negative.

If you don't believe me that you have fallen victim to this hidden assumption of mean reversion, go all the way back to your out of sample simulations... didn't you always use a brownian motion with mean reversion instead of one without? I guarantee you that your method would show long run zero profits, and a loss including commissions, if you remove the mean reversion assumption from your simulation tests and just use a true martingale brownian motion.

Does this mean all hope is lost? No. But I think in order to be profitable you need to focus on instruments which are likely mean reverting. Commodities futures, especially agricultural ones, are known for having this property. I also think you should stop trading leveraged ETFs against their inverses "as a hedge", unless you are specifically after some statistical arbitrage on the spread. Otherwise, there is little reason to believe that simultaneously scalping almost-perfectly-negatively-correlated instruments is any better than just scalping one of them with stops in place. Finally, if there is any edge to be had on mean reversion of equities and ETFs, it is because they go through periods of mean reversion and then periods of "price discovery" to a new mean. Those periods of price discovery are the ones that cause your big drawdowns. Without efforts to identify these periods, and adjust stops/scalping accordingly, it will be very difficult to have a lasting edge.

Just my $0.02. I don't claim to be a market expert, but I strongly want to see you succeed. If there is anything I can do to help, please let me know.

mj

Thank you very much MJ, at last, I can breathe again.

'A tantalisingly thought provoking read', is how I would describe this journal.

But, remove the labour intense outer layer, the intricate detailing and pseudo linguistics, and what remains?

A rather dull core inhabited by the usual 'off the shelf' retail methodology such as, grid trading and mean reversion.

Presentation and Effort 9/10. Trading principle 4.5/10.



Regards.

 

[fullautotrading]

Thank you very much MJ, at last, I can breathe again.

'A tantalisingly thought provoking read', is how I would describe this journal.

But, remove the labour intense outer layer, the intricate detailing and pseudo linguistics, and what remains?

A rather dull core inhabited by the usual 'off the shelf' retail methodology such as, grid trading and mean reversion.

Presentation and Effort 9/10. Trading principle 4.5/10.

Regards.

Thank you DR.BOUYE,

It is always useful to have the opinion of a "doctor".

Unfortunately, I need to inform you that the very mention of "grid trading and mean reversion" in your post just means you have understood nothing of what I have been saying in this thread and especially the "Trading principle". (You may also have been omitting to read the next posts by monkeyjoe, where he makes a subsequent successful effort to get the concept.)

In addition, you are not even "internally consistent", as, if that were actually your personal understanding of the methodology, your vote should have been as follows:

"Presentation and Effort 9/10. Trading principle <b>0/10</b>."

(as "grid trading and mean reversion" are sure failure in the long term and in an ordinary architecture.)

 

[fullautotrading]

It's probably a good idea to post another quick "flashcard", with a synopsis of some key ideas I have been proposing and explaining here.

After all, it's probably easier to look at pictures than to read posts, and flashcards can also serve later as quick remainders.

<img src="http://www.elitetrader.com/vb/attachment.php?attachmentid=144016&stc=1&d=1394378517" />

as we assumed before, the PNL components in the above decomposition are meant to be taken with their sign (L is always, the Unrealized is almost always negative)

(of course, take always anything as "work in progress", subject to changes, corrections, and improvements...)

 

[DR.BOUYE]

Thank you DR.BOUYE,

It is always useful to have the opinion of a "doctor".

Unfortunately, I need to inform you that the very mention of "grid trading and mean reversion" in your post just means you have understood nothing of what I have been saying in this thread and especially the "Trading principle". (You may also have been omitting to read the next posts by monkeyjoe, where he makes a subsequent successful effort to get the concept.)

In addition, you are not even "internally consistent", as, if that were actually your personal understanding of the methodology, your vote should have been as follows:

"Presentation and Effort 9/10. Trading principle <b>0/10</b>."

(as "grid trading and mean reversion" are sure failure in the long term and in an ordinary architecture.)

I was under the impression that my brief critique was quite fair, considering that the project is unfinished.

I failed to mention that the mark of 4.5 for trading principle, was based upon the refreshing notion of options being employed, rather than futures.

Believe me, 0/10 was an ominous contender for the final mark of trading principle.



Regards.

 

[fullautotrading]

I was under the impression that my brief critique was quite fair, considering that the project is unfinished.
I failed to mention that the mark of 4.5 for trading principle, was based upon the refreshing notion of options being employed, rather than futures.
Believe me, 0/10 was an ominous contender for the final mark of trading principle.
Regards.

As I already said, your problem is that you are not getting what is being explained. Not even a small clue of light so far. So, your vote is about your wrong understanding, and, in that sense, it is correctly 0, in the given scale.

Also, in case you did not notice, we never used futures so far ("options being employed, rather than futures").

 

[monkeyjoe]

Well, as we said before, this is really a "tug of war" between gain and loss growth rates. So the goal of creating good games is to realize a good balance, which provides some protection, which be comfortable to trade (and different people may have different "pain thresholds"), but at the same time does not make the growth too small, or even negative. Our edge is the fact we are "recycling losses" when possible.

Where "edge" here does not mean clearly we are guaranteed to become rich soon, but more reasonably that our architecture is providing a <b>statistical advantage</b> respect to a similar strategy that would not be using the past trading information and it's actively using that advantage to "unbalance" the natural 50/50 gain/loss ratio (normally obtained with the odinary "memoryless" stops) in our favor. You just establish a positive "drift", then time must develop it and gradually show it in the PNL (clearly, the larger the folio, also the slower could be the process). In this sense I humbly talk of <b>"strategic dominance".</b>

This is really the crux of the issue. I think what people are struggling with (myself included), are the concepts of strategic dominance and statistical advantage. I still don't exactly understand the source of dominance and advantage from having the players "recycle losses", or of remembering the stops. If you need to explain it in a fully technical/mathematically rigorous way to make it clear and precise, please do. I have a PhD in economics, and I think many others here have similar levels of education, so don't be afraid to be overly technical.

I also understand that if this is the "secret sauce" in your method, and you don't want to share that's ok, too. I just can't tell if you are saying that the players' memory creates the edge, or if there is some aspect of overarching game management which you have not discussed as much in your posts.

Thanks for your help.
mj

 

[fullautotrading]

...I think what people are struggling with (myself included), are the concepts of strategic dominance and statistical advantage. I still don't exactly understand the source of dominance and advantage from having the players "recycle losses", or of remembering the stops. If you need to explain it in a fully technical/mathematically rigorous way to make it clear and precise, please do. I have a PhD in economics, and I think many others here have similar levels of education, so don't be afraid to be overly technical.

I also understand that if this is the "secret sauce" in your method, and you don't want to share that's ok, too. I just can't tell if you are saying that the players' memory creates the edge, or if there is some aspect of overarching game management which you have not discussed as much in your posts.

Thanks for your help.
mj

<b>> I still don't exactly understand the source of dominance and advantage from having the players "recycle losses", or of remembering the stops. </b>

That's the same thing: you "remember" in order to "recycle". It should be clear enough that if you have 2 situations:

- stop and "forget"
- stop and store the information

they can't be the same thing. And a "loss of information" cannot ever beat a condition where the information is preserved, and later used.

This, in very practical terms, would mean that in settings where you "forget", for instance, you could take multiple times similar "stops" in similar conditions.

The "doctor" was possibly impressed by the use of options, because probably he had some perception that a defensive option in place is possibly saving you from taking multiple stops. However, it may create other problems, both related to time decay, and the difficulty to continue grow the G-L even underwater. It's nevertheless an hedging possibility that can be explored, if one is sufficiently experienced to be able to cope with them. But, clearly, it is just way easier and more convenient doing everything through "player superposition", whenever possible.

The player superposition somehow creates a situation which could be envisioned as the old hedging players (still "open") being sort of "artificial" defensive options, as the smart jb514 was implicitly noticing. However, the games, can offer much more as they allow you to assume the perpective you wish, from volatility trading, directional, contrarian or whatever "optical illusion" you like to "see" in the mkt.

<b>> If you need to explain it in a fully technical/mathematically rigorous way to make it clear and precise</b>

Yes that could be done in a scientific paper. I am not sure that I want to do that at the moment. For now, I think that discussing the intuitive aspects is probably more powerful than getting lost within the rigid constraints of some math/stat formalization. Maybe someone else can do it in the future (I have not much academic incentive at the moment to embark).

<b>> I just can't tell if you are saying that the players' memory creates the edge, or if there...</b>

Of course <b>it does not</b>. I am merely talking of a <b>statistical "strategic dominance"</b> towards a "corresponding" strategy which would not be not inspired to the principle of conservation of the trading information". That is, I am posing a preliminary architectural argument.
Since for any strategy that does not use the "trading information", a "new" corresponding strategy can always be built which uses it, and it is better than it ("strategic dominance"), then it makes no sense to use architectures which do not comply with the principle.

Clearly, if you trade like a fool (assume you create a silly game which does for instance a "martingale"), then there is no information and no architecture in the world that will ever save you, as your capital will never suffice, and you will never see the effects of your advantage.
Therefore, it's also important to create meaningful scalping/hedging games which allow you to use at best your information advantage, that a gradual and patient hedging action can allow emerging in the long term.
Essentially, if a game does a good job <b>bounding the unrealized</b> in a finite range (through hedging orders), then the continuous "recovery" of "stops" (<b>recoverable because the relevant information is maintained</b>) will create a "drift". In "very" intuitive terms, some of the 50% losing "trades" in the 50/50 statistical game of win/lose trades are later recovered, thus unbalancing your gain/loss ratio.

 

[monkeyjoe]

Thank you for clarifying, Tom. I certainly understand about not wanting to embark on a full-scale academic endeavor!

So, to paraphrase and make sure I am interpreting your explanation correctly:

1. remembering the stops is an improvement over a naive strategy that forgets them
2. it is ultimately the scalping/hedging game rules that determine the edge

It would be great if you could share more about the game rules. I read your past posts about evaluating correlation, and about the entry/exit (scalping/hedging) determinations for individual layers (the diagrams really help a lot ), but I don't recall the details about rules with respect to player superposition and over-arching game management.

Can you tell us more about which types of rules have been successful thus far? Any intuition about why certain ones succeed and others fail?

Thanks,
mj

 

[fullautotrading]

@monkeyjoe

<b>>So, to paraphrase and make sure I am interpreting your explanation correctly:

1. remembering the stops is an improvement over a naive strategy that forgets them

2. it is ultimately the scalping/hedging game rules that determine the edge</b>

It sounds good. I'd just refine a bit:

2. Provided that 1 holds, it is ultimately the scalping/hedging game rules that can allow the edge provided by 1 to emerge, and keep feeding the drift
(one might argue on what is meant exactly by "edge", but for now let's leave it at an intuitive level)

<b>>It would be great if you could share more about the game rules.</b>

Sure. That is a important area and I am actively working on it. So far the aspects which seems to have the greater impact seems to be:

- order sizing (especially the hedging orders)
- order spacing (which dictates the trading frequency, along with the entry rules)

Note that here there is no hint of "going after the mkt". That is why all concepts related to "prediction", trend, reversion, or whatever should be immaterial. What we are actually creating is an "order cloud" which essentially be "self-consistent" and allows us to scalp and hedge safely, no matter what is the price data stream. We could perhaps poetically say, that "the order cloud is not even looking at the mkt, but just reflecting on itself" On the other hand, being us human, we certainly can look at the mkt, and adjust the game to the available information (for instance, create specialized games for different classes of instruments).

<b>>Can you tell us more about which types of rules have been successful thus far? Any intuition about why certain ones succeed and others fail?</b>

This is a work in progress, and I do encourage people to experiment and participate in this endeavor. Clearly, this is an approach that is particularly suitable for large capital, and does not make sense for trading "peanuts" (well, of course, unless testing or researching.)

It looks like the most damaging actions within this fully automated context, would be either :

- 1 too little hedging
- 2 too strong hedging (for instance practicing abrupt "inversions" of position)

For instance in case 1, you end up with what mostly "looks like", to an external observer, an averaging up/down scenario, and we know where this can lead.

In case 2, you may build up too much losses and it can take too long to make up. Or if the loss rate is high enough you can even never catch up with the scalping and recovering action.

So most likely, <i>"in medio stat virtus"</i>, and certainly the simulation facility can help a lot to understand what the right trade-offs can be and what are the best actions, sizing modes, etc.
I think I previously made the example of keeping a bird in the hand. If holding too loose it will fly away, if holding too tight, you may kill it. But that is not preventing anyone of us being able to actually hold it in the right way. A similar thing happens with an automated management system.

Of course, your point 1 above is <b>crucial</b> (loss recovery). Otherwise, even with the "right balance", you will end up eternally fluctuating (with high variance) around 0, plus commissions, spread and interests imposing a negative drift. And there you have yet another loser!

Clearly it may depend also on the types of instruments. You may want to diversify a commodity from an index, and from "structurally" drifting instruments (like the ushort ETFs), etc.

 

[DR.BOUYE]

<b>> The "doctor" was possibly impressed by the use of options, because probably he had some perception that a defensive option in place is possibly saving you from taking multiple stops. However, it may create other problems, both related to time decay, and the difficulty to continue grow the G-L even underwater. It's nevertheless an hedging possibility that can be explored, if one is sufficiently experienced to be able to cope with them. But, clearly, it is just way easier and more convenient doing everything through "player superposition", whenever possible.


The curse of theta. FAT, what exactly does your algorithm take advantage of, within your chosen market(s)?

For example, the most common type being pricing/bid-ask inconsistencies, whereby speed is of the essence.

Allow me to explain further. If you are not taking advantage by using a defined and definite edge, then you have nothing more than average Joe, at your disposal.



Regards.