You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
Probability of success due to chance alone?
- Thread starter anesthesiaman
- Start date
Quote from logic_man:
Secondly, let's say you've reached a point where you have a large enough sample of trades to have some confidence in the statistical validity, the metric is designed to answer the question, "What is the probability these results are the product of chance?"
The "system quality measure" you made reference to is essentially the signal-to-noise ratio multiplied by a number that depends on the sample size.
You should know that the signal-to-noise ratio is only useful for positive random variables. This is basic fact for statistics students. Whoever does not know that flunks the course
http://en.wikipedia.org/wiki/Signal-to-noise_ratio
As far as a test for testing for randomness, it is known to mathematicians that there is not a definitive way of achieving that.
I agree with the last poster who suggested measuring the performance with respect to that of an index. This makes more sense.
The "system quality measure" is a useless test as far as randomness and also not applicable because trade outcomes can be negative.
Probability of success due to chance alone?
about 50-50 duh
about 50-50 duh
Quote from alexandermerwe:
The "system quality measure" you made reference to is essentially the signal-to-noise ratio multiplied by a number that depends on the sample size.
You should know that the signal-to-noise ratio is only useful for positive random variables. This is basic fact for statistics students. Whoever does not know that flunks the course
http://en.wikipedia.org/wiki/Signal-to-noise_ratio
As far as a test for testing for randomness, it is known to mathematicians that there is not a definitive way of achieving that.
I agree with the last poster who suggested measuring the performance with respect to that of an index. This makes more sense.
The "system quality measure" is a useless test as far as randomness and also not applicable because trade outcomes can be negative.
You're not testing each individual trade, you're testing the set of trades, presumably only if they show a positive average outcome. If they have a negative average outcome, surely you'd reject the system unless you thought there was some value in taking the contrary of the system's signals, in which case you could use the absolute value of the average outcome as your mean. Knowing that a method produces a negative outcome over time at the 95th percentile confidence interval would not be bad information to have.
I never said this was a "definitive" test for randomness, so I guess I agree with mathematicians.
Testing against an index is fine, unless you're trading that index.
Do you guys put as much effort into developing your trading strategies as you do to trying to debunk a fairly random comment on an uncontroversial component of probability theory on an internet chat board? Wow.
Quote from logic_man:
You're not testing each individual trade, you're testing the set of trades, presumably only if they show a positive average outcome.
I think you totally missed the point/message. The measure you suggested does not apply at all to random variables that have negative values.
Do you understand what a random variable is? Do you understand what we mean by the value of a random variable?
Trade P/L can be positive or negative. Therefore, if you average trade P/L to get a mean value, as you do with that measure, you have both positive and negative numbers there. The measure you presented does not apply then.
Quote from logic_man:
Do you guys put as much effort into developing your trading strategies as you do to trying to debunk a fairly random comment on an uncontroversial component of probability theory on an internet chat board? Wow.
I was trying to help you see your mistakes. If you need no help you should announce that and I am sure people will not bother you. Why are you getting so defensive? I have hard time understanding your reaction. I will continue developing my systems and I promise you will never hear from me anymore.
Quote from alexandermerwe:
I think you totally missed the point/message. The measure you suggested does not apply at all to random variables that have negative values.
Do you understand what a random variable is? Do you understand what we mean by the value of a random variable?
Trade P/L can be positive or negative. Therefore, if you average trade P/L to get a mean value, as you do with that measure, you have both positive and negative numbers there. The measure you presented does not apply then.
I was trying to help you see your mistakes. If you need no help you should announce that and I am sure people will not bother you. Why are you getting so defensive? I have hard time understanding your reaction. I will continue developing my systems and I promise you will never hear from me anymore.
The measure is clearly a proxy for something, even if it doesn't meet the strict definition you are putting forth as a requirement. Say what you want about Van Tharp (I'm not a follower), but the man clearly has a lot of data on trading and traders and when he says that the higher a trading method scores on this particular metric, the better a method it is and that the best traders he knows of score highly on it, it's going to carry more weight from a practical perspective than knowing whether or not it meets the textbook definition of "signal to noise ratio".
Since trading is an endeavor without a lot of clearly-defined probability distributions, of course you're not going to be able to measure trading results the way you would the sound quality of a stereo speaker, with properties obeying the laws of physics. Have some perspective.
Quote from logic_man:
The measure is clearly a proxy for something, even if it doesn't meet the strict definition you are putting forth as a requirement. Say what you want about Van Tharp (I'm not a follower), but the man clearly has a lot of data on trading and traders and when he says that the higher a trading method scores on this particular metric, the better a method it is and that the best traders he knows of score highly on it, it's going to carry more weight from a practical perspective than knowing whether or not it meets the textbook definition of "signal to noise ratio".
Since trading is an endeavor without a lot of clearly-defined probability distributions, of course you're not going to be able to measure trading results the way you would the sound quality of a stereo speaker, with properties obeying the laws of physics. Have some perspective.
Van Tharp does not have a real trading record to show I have heard. As a matter of fact, there are rumors he has never traded. I agree with the posters who debunked his quality measure. Most of the stuff he pushes are worthless.
Now, FYI, a non-random system is one that after a large sample of real trades has win rate > 50% and R:R >= 1, thus better than coin flipping.
Quote from tim888:
Van Tharp does not have a real trading record to show I have heard. As a matter of fact, there are rumors he has never traded. I agree with the posters who debunked his quality measure. Most of the stuff he pushes are worthless.
Now, FYI, a non-random system is one that after a large sample of real trades has win rate > 50% and R:R >= 1, thus better than coin flipping.
I'm not sure that it matters whether he trades or not in this instance, because we are only talking about a metric, not a trading method. Someone doesn't have to be a good baseball player to be good at analyzing baseball statistics.
I do keep track of this number (among other metrics I keep track of) and it is definitely true that when my method is performing well, it goes up and vice versa.
I am pretty sure my results are not due to chance, but maybe they are. If you worry about that too much, you get analysis paralysis and don't take your signals when they come, in which case you won't have any results to worry about because you'll just be sitting in cash.
Just wondering, would you say that applies to the inverse as well -- that a non-random system would also be one with a loss rate > 50%?Quote from tim888:
Now, FYI, a non-random system is one that after a large sample of real trades has win rate > 50% and R:R >= 1, thus better than coin flipping. [/B]