backtest for 3 years, blow up in 3 days,

May I just address the first one first...

1. So one entry rule in the market long or short for 25 years (98.5% actually has a position). How would you curve fit that one rule if it is in the market all the time and almost always reverses on exit?

Would a system in the market 98.5% of the time with 34 trades a year, be optinmized but a system in the market 10% of the time and 300 trades a year likely less so?

So time in the market and # rules is irrelevant to over optimization? Appreciate examples to help me understand.
 
Quote from Muskoka Joe:

May I just address the first one first...

1. So one entry rule in the market long or short for 25 years (98.5% actually has a position). How would you curve fit that one rule if it is in the market all the time and almost always reverses on exit?

Would a system in the market 98.5% of the time with 34 trades a year, be optinmized but a system in the market 10% of the time and 300 trades a year likely less so?

So time in the market and # rules is irrelevant to over optimization? Appreciate examples to help me understand.
More...


It's true that "the market all the time and almost always reverses", if we talk in terms of event frequency.

The problem is another. First, all long and short entries are often characterized by different
reversion dynamics.

Second, the frequency of reversals is not so important if most of them will lead to losses
due to the fact that some drawdown are practically unsustainable for most accounts.
(In presence of unlimited capital it's clearly always possible to devise strategies that will
never lose.)

In addition to that, most importantly, the amount of reversals in any case may be insufficient to cover the
drawdown expenses which have occurred to maintain (or grow) the position until reversal. (And this, as
matter of fact, is statistically more likely to happen with short entries.)

Percentages are not much meaningful as measure of performances and can be deadly misleading.

And yes, in general a higher avg number of daily entries will make a possible good result more reliable.


If you have 2 strategies and P some "suitable" (net) performance Index:

- One with positive performance P and 1 avg entry per day
- Another with positive performance only P/10, but with 10 avg entries per day,

then, without hesitation, choose the one with performance P/10 ! ;-)


____________________
Tom
My <a href="http://www.datatime.eu/public/gbot/2009Oct22/default.htm" target="_blank">autotrading</a> journal
 
I am pretty simple and don't understand much of what you wrote. Perhaps you could clarify a bit more?

I take it from your last statement that a system that cuts out the 90% of the action to be in the market 10% of the time on 10 trades, is less likely to be curve fit then to one that has 10% of the trades but is in the market 100% of the time?

Here is the equity curve of this strategy. I do not trade with it by the way as drawdown can be a bit much though recovery is swift. More of a discussion on strategy development.
 

Attachments

Quote from Muskoka Joe:

I am pretty simple and don't understand much of what you wrote. Perhaps you could clarify a bit more?

I take it from your last statement that a system that cuts out the 90% of the action to be in the market 10% of the time on 10 trades, is less likely to be curve fit then to one that has 10% of the trades but is in the market 100% of the time?

Here is the equity curve of this strategy. I do not trade with it by the way as drawdown can be a bit much though recovery is swift. More of a discussion on strategy development.
More...

>I take it from your last statement that
a system that cuts out the 90% of the action to be in the market 10% of the time on 10 trades,
is less likely to be curve fit then to one that has 10% of the trades but is in the market 100% of the time?

i don't recognize exactly what I was saying in your statement above. I am just saying that
the less the avg number of daily entries, more easily a strategy will be subject to a computer optimization
which will reveal to be *specific of the past data* and therefore useless for forward (real or paper) trading.

We all are able to produce fantastic equity curves, fitting past data. It's, then, forward testing that can offer
you hints about possible issues.

Along with backtesting results you need to show also forward testing results. If they are "aligned" then a more meaningful discussion can take place.


____________________
Tom
My <a href="http://www.datatime.eu/public/gbot/2009Oct22/default.htm" target="_blank">autotrading</a> journal
 
Quote from jacksmith:

One common story I had heard about trading system is that,

Successfully backtest for 3 year's data, yet the system blows up in 3 days,

what causes this ?

Thanks.
More...

The3 problem here is the fact that the public is always behind the form ande out of phase with a cycle. Something will work, until it stops and nobody will know when the cycle is going to change.
 
Quote from Alex55:

3 full years I tested the most advanced strategies (special kind of Neural-nets included), and only thing I learned was that:

a) noice in real-time historical data makes it hard to filter "general working strategies".

b) the found "working strategies" gave nice statistics (more than 80% winners) but in real world trading results were nearly zero (because of Slippage and commissions, and technical issues like an internet-stop).

c) the trained history, doesn't repeat in future

After I learned this I went to discretionary trading, and doing quite well.
More...


Hi Alex,

I am also interesting in NN. Could you tell what were your enters and exits? What do you predict with NN?

Thanks
 
What accounts for this is market manipulation, you can apply optimization and backtesting, you can fit the best model but at the end of the day market makers and banks have the enfluence over price and they will "blow you up" as you say...guess I would call it ...shake you down.
 
Quote from maninjapan:

Muskoka, dont suppose you would care to elaborate any more on the strategy behind the equity curve?
More...

The last time anyone here posted usable information was back in 1997.
 
You must log in or register to reply here.
Back
Top