Bset Number of Trades to Judge System Performance

[vikana]

It's entirely a function of the complexity of the system. If you have just few entry filters, then 100 trades are probably enough. If you have more, then very quickly you'll need 1000s of trades.

In my experience, the only realistic way to obtain good statistical coverage is to run the same system on a large portfolio.

On little example: on one of my intraday systems, i have 450 trades the last 3 months. That's enough for me to trade this particular system. It's a relatively simple system with few rules.

 

[dima777]

Quote from Businessman:

If your system only gives 10 signals a year.

Test it going back 10 years across 10 markets.
Or 20 years with 5 markets.

The more the better. But 1000 is the minimum, ideally you want several thousand instances in your back test.

thanks..but 1k is 10 times larger than 100 cited above..I wonder why so huge a difference...

 

[dima777]

Quote from phattails:

Should your system work in previous years (based on the fundamental idea driving the system).

If yes, and if you have enough sample trades from past data then test over as many years as needed.

Otherwise look into developing synthetic data.

does this mean that you belie only fundamentals-driven systems can work for more than a year?

 

[dima777]

Quote from vikana:
In my experience, the only realistic way to obtain good statistical coverage is to run the same system on a large portfolio.
[/B]

is it better to have instruments of different types in portfolio? - eg indexes, currencies, commodities...

 

[vikana]

Quote from dima777:

is it better to have instruments of different types in portfolio? - eg indexes, currencies, commodities...

It's ideal if you can have different instruments as well.

I've found, however, that systems that work well on stocks, rarely work well on currencies. So while theoretically a good idea to test across all asset classes, my experience is that this rarely works.

 

[intradaybill]

Quote from dima777:

Hi!
I am interested to know what is the optimum number of trades that a system should generate on any single market - that can be used to judge its performance. Another question - what is the best average number of bars that should be between the signals - i.e. what is the maximum level of signal clutter that the system can exhibit. I remember the term - "statistical significance" - I wonder what relationship this concept has to the two questions above.
Thanking in advance!
Dima

Are you talking about intraday, position or trend following systems. The difference is importnat.

A trend following system with good filter for sideways markets may generate one or two traders per year. In 5 years you get 10 trades max or less. Does that mean it is not statistical significant result?

On the other hand, an intraday system may generate 500 trades per year. Does that mean it is statistical significant? Most of that may be noise trading. What you care is the confidence level not significance.

Just recall what statistical significance means. It means that the result could not have been obtained by chance alone.

So what? If I have a trading system that generates one trade per 10 years but that is a sure-fire thing because I have an edge, why should I care about significance at all?

Read this to find out why statistical significance tests are useless:

http://www.sciencedirect.com/scienc...serid=10&md5=b94f5bc8cd7ef2888c54b50b8d183187

 

[Pippi436]

I have read several times now that 30 is the minimum sample size to be of any significance. Can anyone tell me where this number comes from?

 

[dima777]

Quote from intradaybill:

Are you talking about intraday, position or trend following systems. The difference is importnat.

A trend following system with good filter for sideways markets may generate one or two traders per year. In 5 years you get 10 trades max or less. Does that mean it is not statistical significant result?

On the other hand, an intraday system may generate 500 trades per year. Does that mean it is statistical significant? Most of that may be noise trading. What you care is the confidence level not significance.

Just recall what statistical significance means. It means that the result could not have been obtained by chance alone.

So what? If I have a trading system that generates one trade per 10 years but that is a sure-fire thing because I have an edge, why should I care about significance at all?

Read this to find out why statistical significance tests are useless:

http://www.sciencedirect.com/scienc...serid=10&md5=b94f5bc8cd7ef2888c54b50b8d183187

thanks...very interesting opinion that captures some of my own feelings towards this subject....the fact is my system is of the first type - producing only a few signals per year per instrument...btw...do you have full access to this site? i was only able to read the outline...