Ever ask the question: Is it robust?

[TSGannGalt]

Markets are always changing. Trends end. Tendencies change. Patterns fade. Edges diminshes.

All system traders and advanced discretionary traders come to the question of:

Is this robust enough?

Or as others may ask:

Is this working?

When you start thinking of robustness of anything you end up with a lot of questions to yourself:

So... how do you deal with it?

When you get a good backtested trading system, what do you do to confirm it is robust?

When you optimize or use some sort of AI., what do you use as a fitness?

What is robustness? How do you define it?

Empirical... Quantitative... Risk Management?

... let's discuss about robustness.

 

[TSGannGalt]

Let me start off the thread with myself. (while keeping it short)

I tend to look at robustness as a part of risk management. I, currently, have a tendency of thinking... "What ifs" in every case I can think of.

What if the market goes against this position? What if the system stops working? What if the network crashes? What if I have a black out? What if there's a terrorist attack? .... etc. etc.

I'm a paranoid guy and I come up with all these scenarios, or risks that may happen. My 11 year experience has taught me that anything can happen (So far, my extreme has been 9/11). So I take the pre-cautions to "control" and minimize the risk as much as possible.

So far, my conclusion, or answer to robustness is the ability to identify and act on the arising risk as efficiently as possible.

You never know how and when it will happen, but I work hard to find risk when it does.

 

[MGJ]

Since this is the "Automated Trading" area, I will confine myself to 100% mechanical, algorithmic, hands-off, no-human-beings-involved, trading. Boxes trading with/against other boxes.

It it robust? Not over the long term, in the John Maynard Keynes sense of the expression ("In the long term, we are all dead."). The trick, if there is one, is to include some adaptation in the 100% mechanical trading programming. Make the portfolio fluid, dynamic, adaptable. Make the betsizing fluid, dynamic, adaptable. Make the composition of the multiple-systems-being-traded-simultaneously, adaptable. It doesn't guarantee success; nothing does. (Consider the expression: "The Risk Free Investment" ... the investment whose return is guaranteed. This investment is T-Bills). But adaptability does allow your trading bot to smoothly transition from good offense to good defense when your pre-programmed scenarios believe it is warranted.

 

[man]

to me the question is two-fold: is a backtest robust and
is real time trading robust. and acutally i think they are
quite close to each other.

i am convinced that every approach build at time t0 has
an expiration date at tx, no matter how much adaptability
one builds in at t0. though adaptability helps it is just one
of the things that were available at t0. so other guys
can build exactly that kind of adaptability in and erode
the edge.

constant development is the key. on a meta-level it is
IMHO the only possible means against erosion. buit it
is not easy to organise since people who found something
tend to "find" the "same" thing again, just using other
tools. at least i am paranoid about that.

in quant-terms we are still using sharpe as our utility
"function". and ttest, edge test and alikes for validity.
but i think what is often missed in backtesting and
online system evaluation is to take into account how
a system was found. to me a system that comes out
of a statistical analysis of thousands of data points
has more vailidity than a system that is found by
programming in tradestation only, as well as a system
thatis coming out of a successful discretionary trading
as well. it makes a difference for all robustness if there
had been a thick analytic process below, though the
actual various tests of the final backtest might be
the same.

 

[itmediaco]

Quote from MGJ:

Since this is the "Automated Trading" area, I will confine myself to 100% mechanical, algorithmic, hands-off, no-human-beings-involved, trading. Boxes trading with/against other boxes.

It it robust? Not over the long term, in the John Maynard Keynes sense of the expression ("In the long term, we are all dead."). The trick, if there is one, is to include some adaptation in the 100% mechanical trading programming. Make the portfolio fluid, dynamic, adaptable. Make the betsizing fluid, dynamic, adaptable. Make the composition of the multiple-systems-being-traded-simultaneously, adaptable. It doesn't guarantee success; nothing does. (Consider the expression: "The Risk Free Investment" ... the investment whose return is guaranteed. This investment is T-Bills). But adaptability does allow your trading bot to smoothly transition from good offense to good defense when your pre-programmed scenarios believe it is warranted.

What if an ECN fails or has latency, or you lose internet connectivity, or your feed has bad or missing ticks, or you get a partial fill and can't cancel the remainder, or your real portfolio goes out of sync with the model... A good metaphor is PayPal, which lost over $100 million to fraud as a necessary step to learning how to stop it.

 

[Bitstream]

how do you backtest a purely discreet method?

 

[man]

Quote from Bitstream:

how do you backtest a purely discreet method?

since this is the AT section i guess it is all about pure mechanics.

 

[Bitstream]

Quote from man:

since this is the AT section i guess it is all about pure mechanics.

i was referring to the op gann post where he seemed to suggest or question testing both discretionary and mechanical systems.

 

[man]

Quote from Bitstream:

i was referring to the op gann post where he seemed to suggest or question testing both discretionary and mechanical systems.

right.

 

[TraderMojo]

Are there two questions here?

1) Robustness in terms of dealing with practical issues such as physical failures of hardware, connectivity, exchanges, data feeds etc.

2) Robustness in terms of the trading strategy, algorithm or method as applied to the markets.

I suppose, depending on what level you view things from you could look upon these two groups as one. For example, perhaps your system adapts to changes in the physical environment using the same technologies and techniques to adapt to changes in the market?

Is it possible to make a system 100% robust or are there always going to be unknown unknowns?