how do u determine if something is statistically significant?
- Thread starter Gordon Gekko
- Start date
And yes.
nitro
nitro
Quote from harrytrader:
This is book's math at school. It is always true because they suppose all conditions are ideal. But in real world these conditions are not filled automatically, that makes the big difference between theory and practice. z score is just a normalisation procedure, a protocol for making things comparable. You can always make any statistical calculation on any sample that doesn't mean that YOU CAN USE THE INFERENTIAL CONCLUSION IF FOR EXAMPLE HOMOSCEDASCICITY OF VARIANCE(I explained the other day the contrary term heteroscedascicity which means instability of variance that is the official term since instability is too fuzzy and could be confused with variance itself !) IS NOT CHECKED. That's why one sample of 30, 100 1000 or even 10000000 can have any sense in PRACTICAL STATISTICAL INFERENCE ! In theory a sample of 1000000 is great but in reality a sample of 1000000 has a huge probability to contain heteroscedascicity more than a small sample that is why in practice it is better to have multiple samples with few items than a single sample with huge collection of items. It is only when you have assured that premisces of inferential theory are good that you can use at extreme one sample only.
That's a general problem at school: they learn you how to calculate not really how to think and feel things relating in fact to common sense. It is when you get some experiences in real world indutries that you will know that what's in book is 1/4th of what you should know.
30 trades... ain't enough...
Let's say, I make a system just out of my head. I think it's very robust and confident because I haven't done any parameter curve-fitting optimization.
With 1 day of E-mini Data, I get 30 trades and 25 are profitable making about $1400 a day.
I forward test this with paper trading the next day, and I've won all trades...
Also, let's add that this is system is counter-trend system.
Will you trade this with real money???
Let's say, I make a system just out of my head. I think it's very robust and confident because I haven't done any parameter curve-fitting optimization.
With 1 day of E-mini Data, I get 30 trades and 25 are profitable making about $1400 a day.
I forward test this with paper trading the next day, and I've won all trades...
Also, let's add that this is system is counter-trend system.
Will you trade this with real money???
Use this as you change the sample size. If you get a statistically significant results with small random sample sizes then you are on to something.
On the other hand if you use a standard good size (meaning as large as is convenient) and you vary factors in the approach, you can then see the relative importance of each factor in the steps of how you vary each of them.
When data is being processed by statisticulators, they follow many size conventions in order that they are playing in the same ball barks and leagues as others.
Not to say anything that is an affront to you, but you colud pick upa used copy of Stats for dummies. It is a very clear read and abrviated which are two things that willeasily get you in the ball park.
We all know you are ranging about and not focusing on central issues as yet, but when you do get exhausted, someday you are going to nail it and your real potential is really going to show up.
Gordo, I would say that your trading history as revealed in your thousands of posts is a statistically significant measure that you will continue to lose your ass and need to get a life.
As with most things, it isn't as simple as any of us would like, and every decision you make regarding statistics involves compromise and trade-offs. When you use statistics to determine significance, first, all that is telling you is that the result is (most likely, but never absolutely) not occuring simply by chance. The 5% error acceptability used in social sciences is simply convention agreed upon by consensus, it is that for no other reason, and may be greater or less than you wish to use. Stats will NOT give you any predictive powers, but can help to determine if your outcomes are more than just random occurance, and while those two things sound similar, they are really very different. Second, simply adding larger numbers of trades to your sample will potentially result in both better AND WORSE results, it is definitely not a case of more-is-better. What I mean by that is that the statistical power achieved by using very large samples will allow you to detect statistically significant events that are for all practical purposes meaningless. An example of that in trading might be that a very large sample could tell you that a system that had a win:loss ratio of 1:1.000001 was, with a 95+% probability, a statistically significant winning system. That would be true, but it wouldn't be of much use in practice, and that would be all the information that a significance test alone would give you. Statistical power is a trade off with effect size. If you achieve significance with a small n, that can demonstrate a larger effect size than the same test with a larger n, and might be much more important. You really need to do a power analysis to determine the optimal paramters for exactly what you are doing. HOWEVER, the design of your experiment/study will determine the validity and reliability of the results in any case. Another caveat is that if you are using correlational tests of significance, you may well fall into the correlation/causation trap. Just because you discover a statistical relationship between time frames and results, or intermarket spreads, does not mean that the relationship is causal in any way, and the design of your study must take that into account, or you will just be curve-fitting your indicators to your data, which doesn't work, but looks GREAT in backtesting. Anyhow, suffice it to say that you can't just pick a certain number of instances, then run a t-test to see if it will make money. I wish.........
Good luck & good trading!
Jessie
P.S. And pay attention to everything harrytrader said too!
Good luck & good trading!
Jessie
P.S. And pay attention to everything harrytrader said too!
Gordo, I would say that your trading history as revealed in your thousands of posts is a statistically significant measure that you will continue to lose your ass and need to get a life. --TruthTeller
Good entries, patient exits, strict discipline on stops, keep costs down, go both long and short - BUT ABOVE ALL, YOU GOTTA DO SIZE -- nitro
Gordon. If these are actual trades and not backtested trades, yes, they are significant. Backtesting only shows, imo, what is possible with a trading system. Actually trading, whether on paper or for real, will tell you if you can face the naked facts of the market as they exist and prevail.
If I spend a whole day trading on paper then I want to have the best results possible and I want to keep those results. If i do 20 trades a day, on average, then the results of those trades, when compared to my backtested system will tell me how effective I've been at facing the market. Eventually, this will empower me to trade with real money. It is a process of facing the truth. No one is born a doctor, lawyer, nurse or teacher. They become all these people. Just as traders become traders, through study, testing, practice, and finally, by working at it day in and day out.
Think about what I've said. You are on the edge of making it. If it takes 20 trades to make 4 ES points, then thats what it takes. If you can do that for 10 days, that's $1000. Don't ever forget with great opportunity comes great responsibility. You have the opportunity to make a lot of money. It is your responsibility to husband your resources so you can continue to seek it.
Bruce
lol ok dick
thx everyone for your replies....
Mr Gekko,
I have decided to post an idea just to try a help you in your quest to find significance in trading. Tons of smart people posted replies on this forum but in my opinion their WAY over complicating the issue, as are you. I don't spend to much time here on ET but I understand that you aren't making money in the market correct? Then stop making it difficult on yourself. I will show you something I am working on right now. (Before I continue I will let you know that my girlfriend is out of town so I am very bored. If you don't want my ideas please disregard this post immediately.) I trade the Dow mini. As such I will show you a chart of the Dow mini but the information I will present will work on just about anything with just a little tweaking.
Awhile back I was going over my 60 min YM chart. This is the chart I use for swing trading the YM's. Something that I have started to notice is that the 60 min RSI has a major resistance level at 85 and a major support level at around 23. (I first thought the support level would be 25 but that turned out to be incorrect.) Another thing I noticed is that when the market is going up on my 60 min chart the downward pull backs will tend to draw the market down to the 40 level on my RSI, and when the market is going down on my 60 min chart the upward pullbacks will tend to draw the market up to the 60 level on my RSI. (This is all marked on the chart.) With that said while watching my 30 day 60 min chart I watched the market going up and drive up my RSI to the 85 resistance level twice and both times it pulled back, bringing the RSI to the 40 level. So what did I do? The next time the market drove the RSI to the 85 level I jumped in short and I had never even back tested the strategy over the last 1,000 years!
:eek: (collective gasp):eek:
I was short just after the open at 9330. I had planed to get out once the RSI receded to the 40 level. Unfortunately I became worried and bailed out to soon. My exit price was 9288. Total profit was 42 points per contract. After this the market headed south and broke both my trend line and the 40 RSI level so I started looking for places to get long in a down trending market.
Like I said I thought the 25 RSI level would work but the market went right through it so I looked for other levels. I saw support and around 20 and then change my support level to 23. Twice the market hit this level and both times the market was able to move higher off the level and move the RSI back up to the 60 level. So the next time the market drove down to the 23 level I pounced on the longs, (no back testing needed).
:eek: (collective gasp):eek:
The market went up and crashed through my 60 level so I waited for a down bar and jumped out. Price in 8880. Price out 9039. Profit 159 points per contract. After that the market headed up again but never reached my short level of 85 on the RSI so I just waited. (It should also be noted here that I trade about 7 different set-ups so these trades are by no means my only trades over the last 30 days.)
Then just last Thursday the market drove down to the 23 RSI level again. So I jumped on the long at 8985 and rode it up to the 60 level again. After we hit the 60 level I waited for a down bar and got out at 9090 profit of 105 points per contract. The total in profit on these three trades is 306 points per contract or 1,530 dollars per contract less commissions.
What's the point of all my babbling? There is no point, I already told you I'm just bored. But my other point is who cares about back testing all this crap for the last 100 years and bla... bla... bla... If I did stuff like I would never get to take any trades. I know some very successful traders who just watch for something to happen a couple times and their all over it. That's what stops are for. To get you out of trades that don't work. What do you think guys did back in the 70 or 80? What about the 30? They didn't have computers where they could back test everything for years and years. They traded set-ups they noticed would happen every now and then. And some of these guys made huge money.
Speaking of stops maybe you want to know where I placed my stops on these trades. I would have got out of the trades on an extreme move in my RSI. Like a move from 85 up to 90. I would have bailed out at 90 and waited for a move back down below 85 and jumped back in. Same the other way. A move down to the 15-16 RSI level and I would have bailed out and waited for a move back above 22 or so and then jumped back in.
I am not recommending that any body take my trades. Everyone trades in their own style and finds what works for them. I'm willing to take big losses on trades like these just because I like the risk/reward level. Some people couldn't handled it and that's fine. The point is I saw an opportunity and I took. I didn't need to back test it over 50 years to see if it always worked, I don't care.
So, El Gordo, if this market drives back to the 85 level I'm all over the short. But if we head back down to the 23 level I'll be all over the long. I have all the statistical evidence I need. And I didn't even need to fry my brain.
Good luck.
I have decided to post an idea just to try a help you in your quest to find significance in trading. Tons of smart people posted replies on this forum but in my opinion their WAY over complicating the issue, as are you. I don't spend to much time here on ET but I understand that you aren't making money in the market correct? Then stop making it difficult on yourself. I will show you something I am working on right now. (Before I continue I will let you know that my girlfriend is out of town so I am very bored. If you don't want my ideas please disregard this post immediately.) I trade the Dow mini. As such I will show you a chart of the Dow mini but the information I will present will work on just about anything with just a little tweaking.
Awhile back I was going over my 60 min YM chart. This is the chart I use for swing trading the YM's. Something that I have started to notice is that the 60 min RSI has a major resistance level at 85 and a major support level at around 23. (I first thought the support level would be 25 but that turned out to be incorrect.) Another thing I noticed is that when the market is going up on my 60 min chart the downward pull backs will tend to draw the market down to the 40 level on my RSI, and when the market is going down on my 60 min chart the upward pullbacks will tend to draw the market up to the 60 level on my RSI. (This is all marked on the chart.) With that said while watching my 30 day 60 min chart I watched the market going up and drive up my RSI to the 85 resistance level twice and both times it pulled back, bringing the RSI to the 40 level. So what did I do? The next time the market drove the RSI to the 85 level I jumped in short and I had never even back tested the strategy over the last 1,000 years!
:eek: (collective gasp):eek:
I was short just after the open at 9330. I had planed to get out once the RSI receded to the 40 level. Unfortunately I became worried and bailed out to soon. My exit price was 9288. Total profit was 42 points per contract. After this the market headed south and broke both my trend line and the 40 RSI level so I started looking for places to get long in a down trending market.
Like I said I thought the 25 RSI level would work but the market went right through it so I looked for other levels. I saw support and around 20 and then change my support level to 23. Twice the market hit this level and both times the market was able to move higher off the level and move the RSI back up to the 60 level. So the next time the market drove down to the 23 level I pounced on the longs, (no back testing needed).
:eek: (collective gasp):eek:
The market went up and crashed through my 60 level so I waited for a down bar and jumped out. Price in 8880. Price out 9039. Profit 159 points per contract. After that the market headed up again but never reached my short level of 85 on the RSI so I just waited. (It should also be noted here that I trade about 7 different set-ups so these trades are by no means my only trades over the last 30 days.)
Then just last Thursday the market drove down to the 23 RSI level again. So I jumped on the long at 8985 and rode it up to the 60 level again. After we hit the 60 level I waited for a down bar and got out at 9090 profit of 105 points per contract. The total in profit on these three trades is 306 points per contract or 1,530 dollars per contract less commissions.
What's the point of all my babbling? There is no point, I already told you I'm just bored. But my other point is who cares about back testing all this crap for the last 100 years and bla... bla... bla... If I did stuff like I would never get to take any trades. I know some very successful traders who just watch for something to happen a couple times and their all over it. That's what stops are for. To get you out of trades that don't work. What do you think guys did back in the 70 or 80? What about the 30? They didn't have computers where they could back test everything for years and years. They traded set-ups they noticed would happen every now and then. And some of these guys made huge money.
Speaking of stops maybe you want to know where I placed my stops on these trades. I would have got out of the trades on an extreme move in my RSI. Like a move from 85 up to 90. I would have bailed out at 90 and waited for a move back down below 85 and jumped back in. Same the other way. A move down to the 15-16 RSI level and I would have bailed out and waited for a move back above 22 or so and then jumped back in.
I am not recommending that any body take my trades. Everyone trades in their own style and finds what works for them. I'm willing to take big losses on trades like these just because I like the risk/reward level. Some people couldn't handled it and that's fine. The point is I saw an opportunity and I took. I didn't need to back test it over 50 years to see if it always worked, I don't care.
So, El Gordo, if this market drives back to the 85 level I'm all over the short. But if we head back down to the 23 level I'll be all over the long. I have all the statistical evidence I need. And I didn't even need to fry my brain.
Good luck.