Binomial hypothesis tests and z score sample size. I have a program that computes them
Last edited:
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
backtesting vs real trades
- Thread starter DTorSwing
- Start date
Binomial hypothesis tests and z score sample size. I have a program that computes them
In the other thread you used 30 trades. State your formula. You know no statistics. This is not coin tossing.
I did not say 30 trades....someone else did, and I produced a graph showing the problem with only 30 trades. Are you just thick, or do you struggle with English?
No one said anything about flipping a coin...no one said anything about hypothesis tests guaranteeing profits. I prefer to have some level of confidence that my test results weren't produced by random chance...and your prefer to be an ass... To each their own
Last edited:
Binomial hypothesis tests and z score sample size. I have a program that computes them
You wrote this:
If you wanted to reduce the margin or error to 1%, than you would need 16641 trades, with 8470 profitable to reject the null hypothesis.
It does not make any sense. You have a program and I don't doubt that. But do you know what it does exactly? Do you have a link? Does the program explain its calculations?
The numbers you are suggesting allude to a test of a fair coin. Why do you think this has any relation to trading systems?
Can you state clearly the null hypothesis H0 you are testing and the alternative?
You wrote this:
It does not make any sense. You have a program and I don't doubt that. But do you know what it does exactly? Do you have a link? Does the program explain its calculations?
The numbers you are suggesting allude to a test of a fair coin. Why do you think this has any relation to trading systems?
Can you state clearly the null hypothesis H0 you are testing and the alternative?
I wrote the program, so yeah,, i know what its doing. Just google sample size determination and hypothesis testing. No sense reproducing what is all over the internet.
The numbers i referenced are for a simple pass fail/test binomial test, which as i stated in a previous post may be an oversimplification. But the result of every trade is usually a profit or a loss so the numbers for significance should be in the right ballpark. Im not making any claims of guaranteeing profitability , just claims that these tests help with establishing confidence that the results of the study were not produced by random chance. The study is still prone to all of the usual biases
I wrote the program, so yeah,, i know what its doing. Just google sample size determination and hypothesis testing. No sense reproducing what is all over the internet.
The numbers i referenced are for a simple pass fail/test binomial test, which as i stated in a previous post may be an oversimplification. But the result of every trade is usually a profit or a loss so the numbers for significance should be in the right ballpark. Im not making any claims of guaranteeing profitability , just claims that these tests help with establishing confidence that the results of the study were not produced by random chance. The study is still prone to all of the usual biases
Are you a joker or statistics illiterate? The null hypothesis of the test with the 16000+ trade requirement for 1% error at the 95% confidence level is that the win rate of the system is 50%. That is what you are testing, the presence of a fair coin. Win rate is irrelevant in trading. Your system can have 25% win rate and still make a lot of money. Many CTAs can attest to that. You have the null hypothesis all messed up. The hypothesis you are testing is:
H0: win rate is 50% (fair coin)
H1: win rate is not 50% (biased coin)
This null hypothesis is irrelevant to trading results. You are using a tool you made yourself but without proper understanding of statistics. The proper hypothesis for trading systems is:
H0: The system mean trade return is 0 (no alpha)
H1: The system mean trade return is not 0 (delivers alpha)
This is a difficult test to perform in Monte Carlo. You must sample period returns that correspond to the lowest time frame used by your system. Then perform some detrending and then determine the sampling distribution. Then shift the distribution by the system mean return and do the ranking to determine the p-value for one-sided test of significance.
What you are doing with your tool is kids staff. It is not even an approximation. It is wrong. You are testing the wrong null hypothesis.
Are you a joker or statistics illiterate? The null hypothesis of the test with the 16000+ trade requirement for 1% error at the 95% confidence level is that the win rate of the system is 50%. That is what you are testing, the presence of a fair coin. Win rate is irrelevant in trading. Your system can have 25% win rate and still make a lot of money. Many CTAs can attest to that. You have the null hypothesis all messed up. The hypothesis you are testing is:
H0: win rate is 50% (fair coin)
H1: win rate is not 50% (biased coin)
This null hypothesis is irrelevant to trading results. You are using a tool you made yourself but without proper understanding of statistics. The proper hypothesis for trading systems is:
H0: The system mean trade return is 0 (no alpha)
H1: The system mean trade return is not 0 (delivers alpha)
This is a difficult test to perform in Monte Carlo. You must sample period returns that correspond to the lowest time frame used by your system. Then perform some detrending and then determine the sampling distribution. Then shift the distribution by the system mean return and do the ranking to determine the p-value for one-sided test of significance.
What you are doing with your tool is kids staff. It is not even an approximation. It is wrong. You are testing the wrong null hypothesis.
Yes, the numbers I quoted are for a 50% win rate, but they can be easily adjusted for expected win rate... 50% win rate results in the most pessimistic sample size requirements, which is why I quoted them.
The only thing I have stated is that you need X number of trades to claim your backtest is statistically significant....I've made no claims of profitability or anything further than that....Excuse me for attempting to help someone who had a question.
Once it sunk into to my widget brain long ago, I bought data, sample size I use for daily/weekly stock trading is 25 years, so I wanted to know on markets that went back that far what I can expect. Based on my own knowledge, going down to minute based entries robs me of longer term sustained trends with the exception of fine tuning entries using volume and two minute charts. BUT if I can't make profits based on a given price entry like last weeks low minus 10% to buy, getting in better by 50 cents not work trading to me. I want time to be on my side after I am in it. And when you staying in longer, you want to be trading dividend stocks that have options, that alone once you get the hang of the selling covered options, can get you another 20% with the dividends. Doing way too many trades in stocks really is tough over long haul, at least for me.I did some backtesting of my strategy of few stocks for 18 months and also tried each 6months. backtesting was done in Ninja trader and Multicharts at for $1000/ stock trade
my strategy is ATR based strategy
1st 6 months $3351
2nd 6 months -$4250
3rd 6months $1452
and I am testing this in sim now and have NO IDEA what would be for the next 6 months. it could be even -PnL
so how do you guys use backtest results? without knowing it can reproduce a +PnL in the future.
I dont expect a exact replica of backtest in real trades, so should be closer?
many stocks have like 100 shares..you cannot trade or daytrade big.. newbies actually think they can buy 200,000 shares of google at $100 and sell it for $101 in the same day everyday and make $200,000 in one hour and do that everyday. well if you buy $100 the stock will drop one dollar and your down $100,000 and it won't go up til you sell at loss.