The Flaw in Trading System Reasoning...?

[Remiraz]

Pure mechanical or system traders will tell us these two things:

1. No system will last forever. It'll break down as the market changes.

2. Mechanical/System traders have to modify or come up with new set of rules/systems as the market changes and their current systems loses their edge.

3. Backtest should best be done over price data that encompass the entire history of the market u're planning to trade the system. (With anti-optimisation measures like seperating data into forward and backward of course)

All traders who backtest will agree that we must backtest as far as as we can for a market.

But as systems which performed well over historical data "stopped working", wouldn't there be lesser systems/edges that can perform well both over historical and new data?

Say there are 10000 systems (both discovered and undiscovered) which works over the entire history of ES.
5000 failed as the market changes. There will be 5000 left.
The original 10000 that works doesn't increase as historical data is static.

Sooner or later....we're gonna run out of edges that performs well in backtest over historical data!

 

[science_trader]

Quote from Remiraz:

Pure mechanical or system traders will tell us these two things:

1. No system will last forever. It'll break down as the market changes.

2. Mechanical/System traders have to modify or come up with new set of rules/systems as the market changes and their current systems loses their edge.

3. Backtest should best be done over price data that encompass the entire history of the market u're planning to trade the system. (With anti-optimisation measures like seperating data into forward and backward of course)

All traders who backtest will agree that we must backtest as far as as we can for a market.

But as systems which performed well over historical data "stopped working", wouldn't there be lesser systems/edges that can perform well both over historical and new data?

Say there are 10000 systems (both discovered and undiscovered) which works over the entire history of ES.
5000 failed as the market changes. There will be 5000 left.
The original 10000 that works doesn't increase as historical data is static.

Sooner or later....we're gonna run out of edges that performs well in backtest over historical data!

They are compensated by 5000 new systems that wouldn't have showed up before. And so on...
There were a lot of systems that, tested on the 95-00 period, wouldn't have been good compared to a buy&hold strategy. But if you consider the 00-04 period, they could show a far better performance.

 

[GSCO]

certain systems work during certain times.
Its up to the trader to find out what is current.

 

[science_trader]

Quote from GSCO:

certain systems work during certain times.
Its up to the trader to find out what is current.

Don't forget that a system 'works' with respect to its underlying market.

Can we say a system works if it has a performance of -10% while its underlying market makes -50% ?

Can we say a system doesn't work if its performance is +30% in a +90% year for its underlying market ?

 

[jonnysharp]

un-correlated systems targeting various market behaviours with a central indictor for asset allocation.

 

[Maverick1]

Quote from Remiraz:

Pure mechanical or system traders will tell us these two things:

1. No system will last forever. It'll break down as the market changes.

2. Mechanical/System traders have to modify or come up with new set of rules/systems as the market changes and their current systems loses their edge.

3. Backtest should best be done over price data that encompass the entire history of the market u're planning to trade the system. (With anti-optimisation measures like seperating data into forward and backward of course)

All traders who backtest will agree that we must backtest as far as as we can for a market.

But as systems which performed well over historical data "stopped working", wouldn't there be lesser systems/edges that can perform well both over historical and new data?

Say there are 10000 systems (both discovered and undiscovered) which works over the entire history of ES.
5000 failed as the market changes. There will be 5000 left.
The original 10000 that works doesn't increase as historical data is static.

Sooner or later....we're gonna run out of edges that performs well in backtest over historical data!

Not all edges are born equal

 

[OddTrader]

Yes, apparently there is a flaw, too many assumptions.

 

[indahook]

it is my belief that a few systems stand the test of time...it is simply the expectancy that changes. The traders using one of these systems will believe it has petered out because the profits dwindle when the underlying volatility or the mkt dynamics have simply changed. That is why it is so important to stay flexible in your approach.

 

[mogul]

setup and entry edges may come and go, but money management principles remain the same

altough money management won't make a negative expectancy profitable, it can exponentially increase the profitability of even the smallest edge

 

[gwb-trading]

This shows the need to trade multiple uncorrelated instruments. When the system stops working one then it most likely to start showing setups on another. This premise holds true irrespective of the type of trading system; trend following, dip-buying, etc.

The other concept implied is the need to find an edge, even a small edge, and utilize appropriate money management in order to maintain long term success.

- Greg

Quote from Remiraz:

Pure mechanical or system traders will tell us these two things:

1. No system will last forever. It'll break down as the market changes.

2. Mechanical/System traders have to modify or come up with new set of rules/systems as the market changes and their current systems loses their edge.

3. Backtest should best be done over price data that encompass the entire history of the market u're planning to trade the system. (With anti-optimisation measures like seperating data into forward and backward of course)

All traders who backtest will agree that we must backtest as far as as we can for a market.

But as systems which performed well over historical data "stopped working", wouldn't there be lesser systems/edges that can perform well both over historical and new data?

Say there are 10000 systems (both discovered and undiscovered) which works over the entire history of ES.
5000 failed as the market changes. There will be 5000 left.
The original 10000 that works doesn't increase as historical data is static.

Sooner or later....we're gonna run out of edges that performs well in backtest over historical data!