In Forward Testing, I've always watched for "setup", and when I can say, "that's it", I write it down, note my stop and go from there. I'm not actually looking to see how well it works, but how badly it's flawed. I WANT trades to break in test. This is much easier that writing code for just about all of us. Comment?
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
What's the catch?
- Thread starter jboydston
- Start date
In Forward Testing, I've always watched for "setup", and when I can say, "that's it", I write it down, note my stop and go from there. I'm not actually looking to see how well it works, but how badly it's flawed. I WANT trades to break in test. This is much easier that writing code for just about all of us. Comment?
OK, then it's research. It's good. We use this to enhance and debug systems while researching them. my remark about forward testing was only to advise against trading based solely on a forward test.In Forward Testing, I've always watched for "setup", and when I can say, "that's it", I write it down, note my stop and go from there. I'm not actually looking to see how well it works, but how badly it's flawed. I WANT trades to break in test. This is much easier that writing code for just about all of us. Comment?
An idea may work for a month and seem the greatest thing since sliced bread. Then you go live with actual money and although the forward test was positive (and well done) the market change and you lose your account.
Don't get me wrong : YOU NEED FORWARD TESTING, but only after extensive backtesting with enough data and many backtest trades (5 years of data and 10 trades is obviously not good !).
PS : of course, outthinking the system is a big flaw. maybe the biggest indeed, because it's sneaky and kills a good system easily.
however, my other points should not be underestimated.
disciplined, backtested trading often fails because the proper design process is not followed.
tntneo
Originally posted by tntneo
OK, then it's research. It's good. We use this to enhance and debug systems while researching them. my remark about forward testing was only to advise against trading based solely on a forward test.
An idea may work for a month and seem the greatest thing since sliced bread. Then you go live with actual money and although the forward test was positive (and well done) the market change and you lose your account.
Don't get me wrong : YOU NEED FORWARD TESTING, but only after extensive backtesting with enough data and many backtest trades (5 years of data and 10 trades is obviously not good !).
One caution on backtesting... some programs come with a backtesting feature. Bad idea to trust it. (Particularly eWave programs.) One should always do their own testing before risking capital and not take anyone's word for it.
Originally posted by tntneo
it seems silly, but that's the one of the most common mistake. one code a system with a few indicators calculated on close value, but enters within the bar.
during backtesting, both the close and total bar is known. in real life you don't have all this data. if your system counts on it and enter in the same bar, you will lose your shirt.
What else of this idea? Is there a fundamental flaw in a system that backtests using the bar's close? Is it more likely to incur slippage?
Using the close is fine. in fact you can use either the close or the open to enter. The danger is to use indicators based on close and enter within the same bar. that's a bug !What else of this idea? Is there a fundamental flaw in a system that backtests using the bar's close? Is it more likely to incur slippage?
nothing against the close itself actually.
But consider open also (if you can build indicators based on the open value) because sometimes it gives you less lag on signals.
We have systems based on opens and on closes (not same system).
This is usually easy to avoid with tradestation, since you enter next bar. But I hate it. I prefer the flexibility of wealth lab. but then you need to be careful with the extra flexibility to avoid coding things which are impossible to match in reality (the forward testing easily detect these however).
tntneo
I'll go with "nothing's that easy." I've had a lot of trouble coming up with a simple method that works consistently.
I'm currently trading a very simple system that I spent several months developing. It looked promising during development, backtesting and forward testing, and even worked pretty well when I first started trading it (close to 70% winners).
However, it completely broke down in mid-August...currently 10 losers in a row. I am following my rules and taking all the trades I see. (I did miss a couple that would have been winners, but I'd still be behind).
Here is the dilemma: I believe the problem may be just that the markets have changed for the month of August. But is it just that? Do I change my rules, or stop trading for the rest of the month, or try to trade through it and hope my system starts working again? If I change the rules or stop trading it, then I'm not sticking with my system and "lack discipline". If I try to trade through it in spite of the dismal recent performance, then I'm ignoring reality and refusing to go with the flow and change my behavior relative to the markets. This is a constant problem for me: the discipline vs. discretion question.
It's easy to blame poor performance on a discipline problem, but then you can get into problems of not taking responsibility. If you lose money, but have great discipline, then you can say, "but I followed the rules". In essence, you are shifting blame to the markets because you are unwilling to change the rules. Again it gets back to the same dilemma: if you change the rules, it can be said that you lack discipline, but if you follow the rules and they stop working, and you don't change them, then you aren't accepting personal responsibility for your results.
A simple change in the stop placement would probably help, but according to the statistics I collected, my stop was at an optimal place. Now it appears to be too close. Do I change my system, or do I maintain my "discipline" and keep getting bitch slapped by the market?
This isn't the first time this has happened. I find that systems tend to break down very quickly. It seems that the market is too fluid and changes too much for anything to work consistently over time (at least that has been my experience). I'd love to find something that has an edge that works consistently over time, but I haven't found it yet.
I'm currently trading a very simple system that I spent several months developing. It looked promising during development, backtesting and forward testing, and even worked pretty well when I first started trading it (close to 70% winners).
However, it completely broke down in mid-August...currently 10 losers in a row. I am following my rules and taking all the trades I see. (I did miss a couple that would have been winners, but I'd still be behind).
Here is the dilemma: I believe the problem may be just that the markets have changed for the month of August. But is it just that? Do I change my rules, or stop trading for the rest of the month, or try to trade through it and hope my system starts working again? If I change the rules or stop trading it, then I'm not sticking with my system and "lack discipline". If I try to trade through it in spite of the dismal recent performance, then I'm ignoring reality and refusing to go with the flow and change my behavior relative to the markets. This is a constant problem for me: the discipline vs. discretion question.
It's easy to blame poor performance on a discipline problem, but then you can get into problems of not taking responsibility. If you lose money, but have great discipline, then you can say, "but I followed the rules". In essence, you are shifting blame to the markets because you are unwilling to change the rules. Again it gets back to the same dilemma: if you change the rules, it can be said that you lack discipline, but if you follow the rules and they stop working, and you don't change them, then you aren't accepting personal responsibility for your results.
A simple change in the stop placement would probably help, but according to the statistics I collected, my stop was at an optimal place. Now it appears to be too close. Do I change my system, or do I maintain my "discipline" and keep getting bitch slapped by the market?
This isn't the first time this has happened. I find that systems tend to break down very quickly. It seems that the market is too fluid and changes too much for anything to work consistently over time (at least that has been my experience). I'd love to find something that has an edge that works consistently over time, but I haven't found it yet.
"I have spent many years developing and evaluating technical trading systems. Although I have found systems that make nearly as much as Trout does (based on average annualized return), these systems invariably exhibit much greater volatility. Drawdowns of 25 percent in these systems are commonplace, with worst-case drawdowns even exceeding 50 percent. Certainly, the volatility of these systems could be reduced by cutting back the leverage (i.e., the number of contracts traded per $100,000). Doing so, however, would lower the returns down to mediocre levels.
I have never found any systems that could remotely approach Trout's performance in terms of return/risk measurements. In fact, every trader I interviewed who displayed a combination of high return and very low risk invariably proved to be a discretionary trader (ie, a trader who relies on his own internal synthesis of market information to make trading decisions, as opposed to using computer-generated trading signals). How, then, does Trout do it?
I got the answer to that question in this interview. Part of it has to do with his reliance on systems that are based primarily on statistical analysis as opposed to more standard, trend-following approaches. However, perhaps the major factor is that Trout's exceptional skill in timing the entry and exit of his positions, by his own estimate, accounts for fully half of his return. "I could give ten CTA's the exact systems we use, and some of them still wouldn't make any money," he says. Thus, once again, we're talking about synthesis of information that can't be computerized (e.g., the noise level on the floors) accounting for the superior performance. In other words, Trout may reach his trading decisions in a similar fashion to many traders, but he executes those decisions like a discretionary trader."
-Jack Schwager, recap of Monroe Trout interview
New Market Wizards (1992)
I have never found any systems that could remotely approach Trout's performance in terms of return/risk measurements. In fact, every trader I interviewed who displayed a combination of high return and very low risk invariably proved to be a discretionary trader (ie, a trader who relies on his own internal synthesis of market information to make trading decisions, as opposed to using computer-generated trading signals). How, then, does Trout do it?
I got the answer to that question in this interview. Part of it has to do with his reliance on systems that are based primarily on statistical analysis as opposed to more standard, trend-following approaches. However, perhaps the major factor is that Trout's exceptional skill in timing the entry and exit of his positions, by his own estimate, accounts for fully half of his return. "I could give ten CTA's the exact systems we use, and some of them still wouldn't make any money," he says. Thus, once again, we're talking about synthesis of information that can't be computerized (e.g., the noise level on the floors) accounting for the superior performance. In other words, Trout may reach his trading decisions in a similar fashion to many traders, but he executes those decisions like a discretionary trader."
-Jack Schwager, recap of Monroe Trout interview
New Market Wizards (1992)