The backtest has 29 trades so far, but you're right - it's fairly small...and although I put "20 days" into the field, it's showing 30 days of data: from 2010/11/08 to present.
My back-testing is too good to be true. What am I missing?
- Thread starter TradeSparrow
- Start date
The backtest has 29 trades so far, but you're right - it's fairly small...and although I put "20 days" into the field, it's showing 30 days of data: from 2010/11/08 to present.
One thing that definitely seems wrong is how you are handling rollovers. Did you do this backtest on a unadjusted continuous contract?
For example:
Buy 2/11/05 Sell 3/1/05
This trade is held through a rollover and it's pretty clear from the prices quoted you have allowed yourself to buy the March and sell the May and capture the spread for free.
This would only affect a few trades a year so I doubt it will make a huge impact, but it is something to think about.
Two other things, take them for what they are worth, only you know you rules and your system:
I only looked at a handful of trades, but I saw that many (even most) buys and sells are at the days closing price. If you are using limit orders to get in and out of trades I'm not sure how you are placing limit orders during the day at what turns out to be the exact close price of the day? Are you sure you haven't allowed your yourself to "look ahead" in some manner? It doesn't feel right, but again, only you know the specifics.
And last, ATR in wheat the last couple years is easily more than double what it was is 5-6 years ago. You need to make sure your 2 cent stop still works.
For example:
Buy 2/11/05 Sell 3/1/05
This trade is held through a rollover and it's pretty clear from the prices quoted you have allowed yourself to buy the March and sell the May and capture the spread for free.
This would only affect a few trades a year so I doubt it will make a huge impact, but it is something to think about.
Two other things, take them for what they are worth, only you know you rules and your system:
I only looked at a handful of trades, but I saw that many (even most) buys and sells are at the days closing price. If you are using limit orders to get in and out of trades I'm not sure how you are placing limit orders during the day at what turns out to be the exact close price of the day? Are you sure you haven't allowed your yourself to "look ahead" in some manner? It doesn't feel right, but again, only you know the specifics.
And last, ATR in wheat the last couple years is easily more than double what it was is 5-6 years ago. You need to make sure your 2 cent stop still works.
Screenshot of a live March 11 Wheat Order Book taken from my CTS trading platform at 9:56 am Central. The volume traded electronically at that point was 25,633. The price volumes in quotations includes exchange-supported implieds, which is fine for your purposes because it is indeed flat price liquidity.
When I am swing trading, I enter a GTC order at my profit level, and for the stop-loss I use a GTC stop limit order with a payup tic level of 3.
Hope this adds some color.
When I am swing trading, I enter a GTC order at my profit level, and for the stop-loss I use a GTC stop limit order with a payup tic level of 3.
Hope this adds some color.
Jordanf, that's really helpful.
1. You're right - I'm not looking at rollover at all and that is something to think about. My original goal was 40% profit per year, though, so with that in mind, the system is more than okay.
2. That's also true. If I want to short a green candle, but my entry trailing stop limit is not getting hit, I'll place an order at 14:12 or so, which is pretty much the closing price. I don't know how realistic that is, but I'll find out as I continue to forward test the system and when I finally start using real money.
3. I've been forward testing the entry strategy, and you're right, it tends to get hit more often, but it's nothing prohibitive so far.
If I need to, I will reduce the contracts I buy to one per $10,000 capital, thus doubling my stop distance to 4 cents. Hopefully, that'll be enough because I only have $10,000.
Bone:
That looks really cool, and I especially like where you say, "which is fine for your purposes", but I don't understand anything else.
- Are the green and blue prices the number of ticks off the Last price?
- What are exchange-supported implieds?
- What is flat-price liquidity?
- I also use a GTC for the profit level and a GTC stop limit for the stop loss. What is a payup tic level of three though?
1. You're right - I'm not looking at rollover at all and that is something to think about. My original goal was 40% profit per year, though, so with that in mind, the system is more than okay.
2. That's also true. If I want to short a green candle, but my entry trailing stop limit is not getting hit, I'll place an order at 14:12 or so, which is pretty much the closing price. I don't know how realistic that is, but I'll find out as I continue to forward test the system and when I finally start using real money.
3. I've been forward testing the entry strategy, and you're right, it tends to get hit more often, but it's nothing prohibitive so far.
If I need to, I will reduce the contracts I buy to one per $10,000 capital, thus doubling my stop distance to 4 cents. Hopefully, that'll be enough because I only have $10,000.
Bone:
That looks really cool, and I especially like where you say, "which is fine for your purposes", but I don't understand anything else.
- Are the green and blue prices the number of ticks off the Last price?
- What are exchange-supported implieds?
- What is flat-price liquidity?
- I also use a GTC for the profit level and a GTC stop limit for the stop loss. What is a payup tic level of three though?
I can say with greater than 99% certainty that this system will not work. My posts are blunt, but they are honest and backed by an actual understanding of math. Please do not take offense to what I am about to say; I just want to save you from losing money.
This system uses indicators in a way that test for correlation (basically, searches for patterns). However, the methods you used do not test for how long these correlations are valid, nor do they test for causality. For example, with enough data mining, you can probably create a system that profitability predicts the S&P during back-tests based on the eating habits of elephants. Just because there is a correlation, it does not mean that the correlation will last. Furthermore, since their isn't a causality relationship between the two, it is just "random correlation". The elephant system will fail.
It may seem strange, but mathematically, your system is no more valid than the elephant based system. That is because in both causes correlation longevity/strength and causality were not assessed. Your post reminds me of my own trading approach when I first started. I thought that just because I was taking a systematic approach that looked mathematically sound, the systems would work. After several years of rigorous study, I have learned that such an approach is not enough.
Furthermore, the data you used is not nearly extensive enough to make any statistically significant deductions regarding correlation.
If you are serious about improving your knowledge of statistics, spending more time studying before trading would be the most profitable course of action. Check out the book at the link below:
http://www.amazon.com/Introduction-...=sr_1_1?s=books&ie=UTF8&qid=1288640834&sr=1-1
That book is not the definitive guide to stats, and in all likelihood, much further reading will be required before you are ready to system analysis the proper way. However, this book is a good start. (Just as a disclaimer, I DO NOT make anything from the sales of that book).
I tend make posts similar to this quite often. After I get some more time, I will make a more in-depth thread regarding the information in this post.
This system uses indicators in a way that test for correlation (basically, searches for patterns). However, the methods you used do not test for how long these correlations are valid, nor do they test for causality. For example, with enough data mining, you can probably create a system that profitability predicts the S&P during back-tests based on the eating habits of elephants. Just because there is a correlation, it does not mean that the correlation will last. Furthermore, since their isn't a causality relationship between the two, it is just "random correlation". The elephant system will fail.
It may seem strange, but mathematically, your system is no more valid than the elephant based system. That is because in both causes correlation longevity/strength and causality were not assessed. Your post reminds me of my own trading approach when I first started. I thought that just because I was taking a systematic approach that looked mathematically sound, the systems would work. After several years of rigorous study, I have learned that such an approach is not enough.
Furthermore, the data you used is not nearly extensive enough to make any statistically significant deductions regarding correlation.
If you are serious about improving your knowledge of statistics, spending more time studying before trading would be the most profitable course of action. Check out the book at the link below:
http://www.amazon.com/Introduction-...=sr_1_1?s=books&ie=UTF8&qid=1288640834&sr=1-1
That book is not the definitive guide to stats, and in all likelihood, much further reading will be required before you are ready to system analysis the proper way. However, this book is a good start. (Just as a disclaimer, I DO NOT make anything from the sales of that book).
I tend make posts similar to this quite often. After I get some more time, I will make a more in-depth thread regarding the information in this post.
There are all kinds of features you can set up with a good execution platform like CTS or TT for futures trading. The column on the far left is the volume traded at that price (electronically). The chartreuse colored price is the last print with quantity. The bright yellow is an average price for the trading session. The history column histogram is set for 2 minutes as I recall. You can also link the order book to a CTS charting package. With the CQG Integrated Client package, you can trigger trades right off the platform into the execution order ticket. I realize that there are many other platforms that do that, but the ECN infrastructure (speed) for CTS, TT, and CQG IC is very robust with co-located servers and dedicated order servers. Call it 'poor man's gray box trading' or something to that effect. The order book quantities in parentheses are part of spread market orders (exchange implied spread order matching) - because you are trading just one contract month on a directional flat price basis they provide liquidity and should be of no concern to you, in fact they help tighten up the market. For your purposes they are there for the taking. I always put a tic value into my stop limit (SL) orders - otherwise, they just go to the next available price (which under market stress or timeframe considerations can be alot). Just a good practice.
Again, hope it helps.
Again, hope it helps.
"With the CQG Integrated Client package, you can trigger trades right off the platform into the execution order ticket. "
Point of clarification: you can set trade triggers right off of chart technical studys or chart price levels. You can also link an order book to an Excel spreadsheet - when the conditions or rules within your macros are satisfied, that can be enabled to shoot orders directly into the execution order ticket as well. Lots of possibilities, not as slick or functional or fast as a custom black box system, but it will do the job depending upon the level of sophistication you desire.
Point of clarification: you can set trade triggers right off of chart technical studys or chart price levels. You can also link an order book to an Excel spreadsheet - when the conditions or rules within your macros are satisfied, that can be enabled to shoot orders directly into the execution order ticket as well. Lots of possibilities, not as slick or functional or fast as a custom black box system, but it will do the job depending upon the level of sophistication you desire.
But this is not what he has done. So your point is irrelevant. He has used mainstream TA. There is a high corellation and relevance between TA patterns and market moves because the latter cause the former beyond any doubt. Furthermore, there is nothing wrong with data mining. There are many ways to minimize data mining effects including but not limited to forward testings. My resampling analysis showed his results are highly significant. I cannot discount forward looking though, he has to investigate that.
NaifWonder, I appreciate these kinds of posts. If you have time for a in-depth criticism, I would be very happy. My intention is not to defend my current system, but to create a system that works.
In response to your post, I can't claim the system will work in the future. I think a system has to evolve over time.
Once I have a working system, I will continue to look for patterns and indicators that hint at the market condition. I am sure that as the market changes, the patterns we see will mean different things. Here's an example:
For most of the test with wheat, there were absolutely no doji. Then, I think in 2005, I started seeing them. I looked for a pattern and once I found one, I traded it successfully.
My approach has been to write down indicator and chart patterns, looking for anything helpful. As you've seen, the approach is not statistically rigorous the way yours is. It's more like...very basic technical analysis with an heaping spoonful of hand-waving for good measure.
Correlation longevity:
I try to assume that any correlation I find may quickly and will eventually become meaningless. However, if a pattern does arise that seems to hold true year after year...well, I suppose that is my measure of correlation longevity.
Like i said, I have no rigorous statistical support. Once I am done with The Futures Game by Teweles and Jones, I'll read the book you recommended! I never studied statistics, and seeing as I studied math, I really ought to take a look!
Causality:
By this, do you mean to say that a pattern I find has to make sense logically in terms of market psychology/fundamentals/etc.? When I see a candle doing something, I do ask myself what the amateurs are doing, what the public is doing, what the experienced traders are doing, and what the professionals are doing.
Of course, I don't have much understanding of these concepts, so I'm happy you've recommended some reading material.
In response to your post, I can't claim the system will work in the future. I think a system has to evolve over time.
Once I have a working system, I will continue to look for patterns and indicators that hint at the market condition. I am sure that as the market changes, the patterns we see will mean different things. Here's an example:
For most of the test with wheat, there were absolutely no doji. Then, I think in 2005, I started seeing them. I looked for a pattern and once I found one, I traded it successfully.
My approach has been to write down indicator and chart patterns, looking for anything helpful. As you've seen, the approach is not statistically rigorous the way yours is. It's more like...very basic technical analysis with an heaping spoonful of hand-waving for good measure.
Correlation longevity:
I try to assume that any correlation I find may quickly and will eventually become meaningless. However, if a pattern does arise that seems to hold true year after year...well, I suppose that is my measure of correlation longevity.
Like i said, I have no rigorous statistical support. Once I am done with The Futures Game by Teweles and Jones, I'll read the book you recommended! I never studied statistics, and seeing as I studied math, I really ought to take a look!
Causality:
By this, do you mean to say that a pattern I find has to make sense logically in terms of market psychology/fundamentals/etc.? When I see a candle doing something, I do ask myself what the amateurs are doing, what the public is doing, what the experienced traders are doing, and what the professionals are doing.
Of course, I don't have much understanding of these concepts, so I'm happy you've recommended some reading material.
That's too bad, because it's selling for $155 US, on sale from $213! :eek: Ahahaha.
Thanks. Yeah, I try to keep that in mind. If the rise/fall hit my stops during testing, I have to be careful.