New to elitetrader- have traded the last 2 years. Now focusing on a mechanical strategy I've developed following intra-day trends of the ETFs. I wrote this a while back while doing some research on trading strategies and putting some ideas on paper, and wanted to share to see what you guys think.
Basically on over-filtering a trading strategy, why I'd rather have 100 trade signals over 10 (if I'm following a mechanical system)- and why so many back tested strategies that look great fail due to curve fitting. Probably nothing new to those of you who've been trading a while though.
Note- While I am good w/ stats, etc. I am certainly not an expert! These are basically just my thoughts on the matter.
---------
Insurance companies do not want to simply hand out insurance to the entire population. Excluding obvious, easily proven, outliers such as current cancer and AIDS sufferers can drastically reduce the odds of an insured developing these conditions. However, when a group of similar risks are lumped together (such as a large corporation of financial workers) the Law of Large Numbers comes into play; the reason insurance companies demand that (in the case of group insurance) 100% of workers sign up with the plan in order to reduce adverse selection.
Why would insuring the entire working population of a company, rather than try to cherry pick group of people with a less than average chance of getting sick, be more desirable from an insurance standpoint? The Law of Large Numbers is the reason. This law of statistics states that:
âThe average of a large number of independent measurements, with a similar probability of success, of a random quantity tends toward the theoretical average of that quantity.â
The more samples you take of a population with a similar risk profile the more likely you are to come close to the average value of the entire population. For example, say that out of a population of working professionals, 100 out of every 1000 will develop a grave disease over the course of the upcoming year. Picture what could happen in an insurance company tried to hand-pick the workers it would insure, and instead of insuring the entire population of 1000 they narrowed down the number of people to receive insurance to only 100. Given that 100 people (10% of the 1000) are statistically destined to become gravely ill, there is a chance (albeit a small one) that all 100 (or 100%) of the hand-picked population, out of the original 1000, will be the unfortunate statistical victims!
Given this scenario, there are a number of possible outcomes.
1. 100% of the 100 chosen people are disease free
2. Only a fraction of the chosen population will be stricken with a disease
3. 100% of the 100 chosen will also be members of the statistical probability group destined to fall gravely ill.
Out of these three scenarios, only the first one would really be acceptable to an insurance company. The second scenario could potentially yield disastrous results, as each member of that population to fall ill is makes up much larger percentage than if the entire population were to have been taken into account. Even if only ten of the cherry-picked population became sick (10%), the percentage of sick workers would now equal the same percentage of ill workers that the entire population had as a whole. What happens if 20 become ill? More? It is because of these risks that insurance companies seek to include as many people, of similar risk profiles, into their group insurance plans. Typically it is mandatory that 100% of a company seeking these plans is to be covered under these blanket plans. Narrowing down the population to be insured even more would continue to increase the risk of having a disproportionate number of insured that become ill. The fewer who are included in a population of similar risk profiles, the greater the risk that there are more adverse selections (in this case gravely ill workers) within this group- better to spread the risk out over the entire population of similar risks. The larger the number of insured, who all have similar risk profiles, the further the risk is distributed.
It is only when an entire population of similar risk profiles are taken into consideration that these numbers become acceptable. As the Law of Large Number states, the larger your sample size, the more accurate your predictions will be. In the above example, if the insurance company decided to cover the entire population of 1000 workers and the statistical 100 became ill, the impact of those who became ill would be drastically reduced. While there is now an almost guaranteed statistical risk that 10% of the workers will result in a loss for the insurance company, there is close to a 0% risk that 100% of the workers will become ill, thus keeping the overall risk profile of the insured group to a minimum.
Take the same theory, that a larger number of samples from a similar risk profile will produce more stable, predictable results then a smaller sample, and now move the focus away from insurance and towards trading the markets. The results are very much the same.
While analyzing a certain market, an investor/trader can attempt any number of strategies- most utilizing certain filters in an attempt to narrow down the total universe of stocks/futures/etc. into a smaller population that meet certain predefined criteria. These criteria are chosen with the belief that they will produce a list of potential trading/investing opportunities that will be profitable.
-need to split here to keep post length down...
Basically on over-filtering a trading strategy, why I'd rather have 100 trade signals over 10 (if I'm following a mechanical system)- and why so many back tested strategies that look great fail due to curve fitting. Probably nothing new to those of you who've been trading a while though.
Note- While I am good w/ stats, etc. I am certainly not an expert! These are basically just my thoughts on the matter.
---------
Insurance companies do not want to simply hand out insurance to the entire population. Excluding obvious, easily proven, outliers such as current cancer and AIDS sufferers can drastically reduce the odds of an insured developing these conditions. However, when a group of similar risks are lumped together (such as a large corporation of financial workers) the Law of Large Numbers comes into play; the reason insurance companies demand that (in the case of group insurance) 100% of workers sign up with the plan in order to reduce adverse selection.
Why would insuring the entire working population of a company, rather than try to cherry pick group of people with a less than average chance of getting sick, be more desirable from an insurance standpoint? The Law of Large Numbers is the reason. This law of statistics states that:
âThe average of a large number of independent measurements, with a similar probability of success, of a random quantity tends toward the theoretical average of that quantity.â
The more samples you take of a population with a similar risk profile the more likely you are to come close to the average value of the entire population. For example, say that out of a population of working professionals, 100 out of every 1000 will develop a grave disease over the course of the upcoming year. Picture what could happen in an insurance company tried to hand-pick the workers it would insure, and instead of insuring the entire population of 1000 they narrowed down the number of people to receive insurance to only 100. Given that 100 people (10% of the 1000) are statistically destined to become gravely ill, there is a chance (albeit a small one) that all 100 (or 100%) of the hand-picked population, out of the original 1000, will be the unfortunate statistical victims!
Given this scenario, there are a number of possible outcomes.
1. 100% of the 100 chosen people are disease free
2. Only a fraction of the chosen population will be stricken with a disease
3. 100% of the 100 chosen will also be members of the statistical probability group destined to fall gravely ill.
Out of these three scenarios, only the first one would really be acceptable to an insurance company. The second scenario could potentially yield disastrous results, as each member of that population to fall ill is makes up much larger percentage than if the entire population were to have been taken into account. Even if only ten of the cherry-picked population became sick (10%), the percentage of sick workers would now equal the same percentage of ill workers that the entire population had as a whole. What happens if 20 become ill? More? It is because of these risks that insurance companies seek to include as many people, of similar risk profiles, into their group insurance plans. Typically it is mandatory that 100% of a company seeking these plans is to be covered under these blanket plans. Narrowing down the population to be insured even more would continue to increase the risk of having a disproportionate number of insured that become ill. The fewer who are included in a population of similar risk profiles, the greater the risk that there are more adverse selections (in this case gravely ill workers) within this group- better to spread the risk out over the entire population of similar risks. The larger the number of insured, who all have similar risk profiles, the further the risk is distributed.
It is only when an entire population of similar risk profiles are taken into consideration that these numbers become acceptable. As the Law of Large Number states, the larger your sample size, the more accurate your predictions will be. In the above example, if the insurance company decided to cover the entire population of 1000 workers and the statistical 100 became ill, the impact of those who became ill would be drastically reduced. While there is now an almost guaranteed statistical risk that 10% of the workers will result in a loss for the insurance company, there is close to a 0% risk that 100% of the workers will become ill, thus keeping the overall risk profile of the insured group to a minimum.
Take the same theory, that a larger number of samples from a similar risk profile will produce more stable, predictable results then a smaller sample, and now move the focus away from insurance and towards trading the markets. The results are very much the same.
While analyzing a certain market, an investor/trader can attempt any number of strategies- most utilizing certain filters in an attempt to narrow down the total universe of stocks/futures/etc. into a smaller population that meet certain predefined criteria. These criteria are chosen with the belief that they will produce a list of potential trading/investing opportunities that will be profitable.
-need to split here to keep post length down...