backtest for 3 years, blow up in 3 days,

Quote from Alex55:

Then we both agree that automated trading doesn't work.
As all automated-trading engines are geared to exploit one rule (call it an edge) which works generally in the past and future.
More...

This is interesting, because I've found some different conclusions.

First off, not all automated strategies exploit a static parameter based rule.
Some adapt dynamically.

2nd, I find it hard to believe you stumbled upon 80% success rate (is that hit rate only? what was theil and rmse) in training/validation/test windows, and it was zero profitable due to slippage/commision:confused: If what you say is true, I assure you, you are miles ahead of the some of the smartest minds in finance. Somehow I doubt it, but I want to give you the benefit of the doubt. Some comments you made were astute, from my experience.

Since the strategy was unprofitable, I'm sure you don't mind sharing a losing strategy then? Because I'd really like to see under the hood as to why this system failed in practice.
Also how much data did you use for your in sample set?
 
Quote from Alex55:

Then we both agree that automated trading doesn't work.
As all automated-trading engines are geared to exploit one rule (call it an edge) which works generally in the past and future.
More...

"Then"?

With all respect, you have peculiar sense of logic relations and of what a logical implication is.

Where do you draw the conclusion that I support the idea that "automated trading doesn't work" when, instead I have been even indicating in my signature my autotrading journal with detailed tests of different algos ?

Like in:

"
___________________
Tom
My <a href="http://www.datatime.eu/public/gbot/2009Oct22/default.htm" target="_blank">autotrading</a> journal
"

Your statement "As all automated-trading engines are geared to exploit one rule" does not mean much to me.

You may say that an automatic systems may implement one or more class of algorithms, each with its own parameter space.

"geared to exploit one rule" is an expression rather vague that, to me, does not tell much. In addition, for what I understand, it's not generally true.

Strict logic is essential, or else it would be impossible to make progress in developing effective strategies.

And, in my personal experience (and I may be wrong), the best algos are not driven by an underlying "rule" which can be possibly "learned" from past data (in short: no prevision).


Tom
 
Not having read the whole thread, I apologize should I repeat something said already.
However, I went through this process, a big eye opener was one of the reading I had to do for the CMT exam of the MTA (certified market technician), "Evidenced-based Technical Analysis" by Aronson.
Statistical concepts such as
Null Hypothesis
Population
Sample
Population Parameter
Sample Statistics
Inference
Reliability of Inference
Significance of Observation

Descriptive Statistics, such as Frequency Distribution, Measure of Central Tendency, Measure of Variation

Inferential Statistics, such as Probability, Probability Density, Degree of Certainty

Hypothesis Testing with Detrended Data

etc. etc. ..... are all covered.

Psychological points from behavioral finance, such as hindsight bias, hedonic framic, anchorin, heuristics, illusion of validity all show themselves in your backtesting are also covered, albeit in some other readings more.

Highly recommended all, a big eye opener at the time.

Once all information is covered, which software can help you with it?

Lots to ponder.

Cheers.
 
Quote from dtrader98:

First off, not all automated strategies exploit a static parameter based rule.
Some adapt dynamically.
More...

Indeed that is the way to go: adapt dynamically to the "new future"". And that's the way I did as well.

I programmed an (quite large) environment of a Genetic-algorithm (self learning) with a special kind of bell-curve logic (Sorry forgot the scientific name).

Genetic-systems get feed with new data, so adapt.
 
Quote from dtrader98:

2nd, I find it hard to believe you stumbled upon 80% success rate
More...

Actually the succes rate for the learn-phase data was much higher. The test-data (not used in the learn-phase) was about 80%.
System goal was to decide if to take a shortterm scalp-trade, and if so, predict the target (in ticks). In 80% of all cases it took a correct trade (direction was ok). The Target-prediction was much harder, and the cause why it wasn't profitable to be used in real trading.
 
Quote from dtrader98:

Since the strategy was unprofitable, I'm sure you don't mind sharing a losing strategy then? Because I'd really like to see under the hood as to why this system failed in practice.
Also how much data did you use for your in sample set?
More...

Sample set was several years of Time & Sales data of most common Futures.

I might tell more later (have to go now).
 
Quote from Alex55:

Then we both agree that automated trading doesn't work.
As all automated-trading engines are geared to exploit one rule (call it an edge) which works generally in the past and future.
More...

There are two major dangers in dealing with automated trading.
First, the idea of buying a fully automated system that you can just turn on and make money is ludicrous. No one would sell a product that makes more money then it is selling for. That's like selling $1000 for $100. doesn't happen. Be very very wary of fully automated black box systems.

Secondly, automating an strategy that you have created is a much better idea but it has pitfalls too. Programming/scripting a strategy is very time consuming and once it's done even if it works for a while it is unlikely that the same thing will work indefinitely in a dynamic marketplace. Constantly revamping your code or worse having your programmer revamp your code is far too slow and costly to likely result in profits.

However, automation is certainly the future of trading and it is too pervasive to ignore.
Cyborg Trading offers an alternative for Sterling and Laser users allowing users to program strategies in minutes using a point and click interface. Cyborg also offers automated spread trading, Signal execution (like Trade-Ideas), and canned algos. like the shake. http://www.youtube.com/cyborgtrading#p/u/0/ngwpcJy7CTA
Automating is a viable option, you just have to be careful how you do it.
 
What would you think about a mechanical trading system that in historical testing:

1. Has been in the market either long or short since SP inception.

2. Has been profitable every year since SP inception.

3. Averages about 34 trades per year.

4. Has only 1 entry rule which is mirror opposite for shorts and longs.

5. Has only 2 exit rules.

6. None of the parameters specify anything to do with date or time.

7. Shows hypothetical gains on 1 contract with no money management of $1M.


Is this possible? Is it possible to over optimize given the above characteristics?
 
Quote from Muskoka Joe:

What would you think about a mechanical trading system that in historical testing:

1. Has been in the market either long or short since SP inception.

2. Has been profitable every year since SP inception.

3. Averages about 34 trades per year.

4. Has only 1 entry rule which is mirror opposite for shorts and longs.

5. Has only 2 exit rules.

6. None of the parameters specify anything to do with date or time.

7. Shows hypothetical gains on 1 contract with no money management of $1M.

Is this possible? Is it possible to over optimize given the above characteristics?
More...

>3. Averages about 34 trades per year.

34 trades per year will make the system essentially untestable and lead to immediate overfit (false profitability).

It' like trying to measure a grain of dust with a meter.

>4. Has only 1 entry rule which is mirror opposite for shorts and longs.

Market does no offer a clue of symmetrical behavior. In fact, it's quite the opposite. The fact that symmetrical entries are possible does not mean that market will reward symmetrical strategies.


____________________
Tom
My <a href="http://www.datatime.eu/public/gbot/2009Oct19/default.htm" target="_blank">autotrading</a> journal
 
You must log in or register to reply here.
Back
Top