Bset Number of Trades to Judge System Performance

D

dima777

Hi!
I am interested to know what is the optimum number of trades that a system should generate on any single market - that can be used to judge its performance. Another question - what is the best average number of bars that should be between the signals - i.e. what is the maximum level of signal clutter that the system can exhibit. I remember the term - "statistical significance" - I wonder what relationship this concept has to the two questions above.
Thanking in advance!
Dima
 
a standard answer is >100 trades. of course, 100 trades do not preclude a curve-fit.

will be interested in hearing what metrics people use to adjust for clustering.
 
You have to consider both the time and trades.

1. For something to be statistically viable, academically, you need at least 30+ data samples. I personally like to have more, though.

2. Markets change with time. You can 1000 trades in a day, but that data only considers the market condition of 1 day.

3. It's not only about the frequency of the system you use but type of the system you are testing that matters.

If you're system is parametric then you'll be needing to test in a longer timeframe to test the robustness of the system.

If you are trying to extract a specific tendency or an edge, and testing for it, then you'll be needing more trade samples rather than time. (Time is not weighed larger because edges always fade. To explain it further... You expose to maximize the profit and most importantly you're testing to find the cut off point. You're already considering to cut it out when the edge is gone so testing for robustness of the edge is weighed smaller. Simply, edge is an edge because the tendency being exposed is already defined.)
 
From a practical stand point, you should have enough trades so that your historical sample test is a good approximation for the population. When applying statistics, which in your case is the sample error you must identify the assumptions-- one of which is independence of samples.

If they were independent then you would choose the sample error that suits your confidence requirement. =1/sqrt(n)). So, for a 10% error -- n=10... for a 5% error n=400..

Try to divide your data into subsets of data that you could classify as, for example, nonvolatilie trending, volatile trending, etc. Then you could weight the results of each sample test to reflect the actual results. Don't forget to rule out data that is no longer useful for representation, or create a work around.
 
whoops, i meant n=100 for a 10% error. However, you must consider the fact that we will never know what the population of all possible trades are. As Taleb expressed, the average behavior in the market is not nearly as important as the "improbable" behavior.

This topic, extensively answered, would lead to identifying the how-to's for estimating the distribution of your trade population.

To be perfectly accurate, there is no such thing as an optimal number of trades in a socioeconomic driven market. You can impose mathematics and all of its sharp edges on this sort of thing. If you lose sight of the forest for the trees, you're screwed.
 
It depends on what you want your margin of error to be.

You can use the preisdental polls as a guide, they get a field of about 800 to get their error rate between 2-3%
 
Quote from shortie:

a standard answer is >100 trades. of course, 100 trades do not preclude a curve-fit.

will be interested in hearing what metrics people use to adjust for clustering.
More...

thanks....the answer is the more the better? or is there a ceiling number of trades beyond which adding more trades won't add anything to the picture?
 
You must log in or register to reply here.
Back
Top