Black Diamond -
thanks for this. a lot of info... give me a couple days to look at it before answering.
thanks for this. a lot of info... give me a couple days to look at it before answering.
How to research and verify trading ideas
- Thread starter talontrading
- Start date
- Status
It is (sort of) a mean reversion approach. If you trade only the members of the S&P 500 with it, then I would expect that portfolio selection to be the best performing in the world in terms of standard indexes. The S&P is so arbed up / over traded that (currently) it makes for a damn fine mean reversion trading portfolio.
My first concern is actually if people are using point-in-time data. I actually suspect (based on what I see so far in this thread) that most of you may think you are, but probably aren't (or data quality is... questionable). If it isn't coming from Reuters or S&P Comstock, I would be particularly concerned about the results you are seeing.
Short-side is going to give you (maybe) better risk-adjusted returns. Building a durable short strat that provides decent alpha is sort of a mirage for just about all fully automated guys. Inflation is a bitch (gives new meaning to "don't fight the fed" right?)
On your second point, I suggest researching it. I personally haven't found a ton of gold in volume analysis other than it indicating liquidity and thus potentially indicating costs of doing business (slippage). Which brings me to your 3rd comment.
Be careful of costs of doing business. Particularly doing spreads on options. At least the S&P components are somewhat liquid in the options market, but this thing even trading with the underlying might suffer significantly from commissions/slippage.
-TD80
My first concern is actually if people are using point-in-time data. I actually suspect (based on what I see so far in this thread) that most of you may think you are, but probably aren't (or data quality is... questionable). If it isn't coming from Reuters or S&P Comstock, I would be particularly concerned about the results you are seeing.
Short-side is going to give you (maybe) better risk-adjusted returns. Building a durable short strat that provides decent alpha is sort of a mirage for just about all fully automated guys. Inflation is a bitch (gives new meaning to "don't fight the fed" right?)
On your second point, I suggest researching it. I personally haven't found a ton of gold in volume analysis other than it indicating liquidity and thus potentially indicating costs of doing business (slippage). Which brings me to your 3rd comment.
Be careful of costs of doing business. Particularly doing spreads on options. At least the S&P components are somewhat liquid in the options market, but this thing even trading with the underlying might suffer significantly from commissions/slippage.
-TD80
First of all forgive me for my poor english cause this is not my mother tongue.
Using Tradedision (www.tradedision.com) I ran a backtest for this strategy on all SP500 stocks for the period 2000-2009.
As my skills on programming are limited and I use a common desktop pc I had to make some concessions.
Setup :
-I replaced the 50e percententiel by a sma(close, 100), this was really necessary to free up cpu capacity.
-Each position is 10% of a 200.000 $ account for each stock.
-2.5$ cost/transaction
-0.04% slippage
- data is Yahoo. I know some of you have doubts (me to
) but don't forget this strategy only needs one datapoint a day : the closing and maybe the closing data are correct on Yahoo ??
- opening/closing position 'on this bar close' (also the stop !!, don't forget)
The results in attach.
Don't look at the absolute figure of the total account, it only needs to cover the total open positions of 10% for each stock started with 200K/stock so it's not relevant.
Using Tradedision (www.tradedision.com) I ran a backtest for this strategy on all SP500 stocks for the period 2000-2009.
As my skills on programming are limited and I use a common desktop pc I had to make some concessions.
Setup :
-I replaced the 50e percententiel by a sma(close, 100), this was really necessary to free up cpu capacity.
-Each position is 10% of a 200.000 $ account for each stock.
-2.5$ cost/transaction
-0.04% slippage
- data is Yahoo. I know some of you have doubts (me to
) but don't forget this strategy only needs one datapoint a day : the closing and maybe the closing data are correct on Yahoo ??- opening/closing position 'on this bar close' (also the stop !!, don't forget)
The results in attach.
Don't look at the absolute figure of the total account, it only needs to cover the total open positions of 10% for each stock started with 200K/stock so it's not relevant.
as you can see in the report, average days in losing trades are double than in winning trades
I've tried some things and as mentioned here before tightening the closing is not an option.
Altough the profit/loss ratio is +/- 1.20.
I really think the exit needs some extra attention but .... ???
I've tried some things and as mentioned here before tightening the closing is not an option.
Altough the profit/loss ratio is +/- 1.20.
I really think the exit needs some extra attention but .... ???
Maybe of more use would be to take an actual money making strategy (that no longer works so you're not giving away the golden goose) and use that to lead us from dev to profit.
Any of us can come up with a strategy that is nominally profitable on a backtest but doesn't actually beat the vigorish going forward (commission and slippage).
Now comes the time when you can show us how to spot the gold in the pan on our own. What do we do with our strategy from this point: Discard it? Optimize it? Use it as a component in a larger meta-strategy?
Don't get me wrong, I still like this thread. The Talon/BoWo interaction was worth the price of admission. But it's right now where we can really learn something. Nudge us in the right direction, what do we do next?
Any of us can come up with a strategy that is nominally profitable on a backtest but doesn't actually beat the vigorish going forward (commission and slippage).
Now comes the time when you can show us how to spot the gold in the pan on our own. What do we do with our strategy from this point: Discard it? Optimize it? Use it as a component in a larger meta-strategy?
Don't get me wrong, I still like this thread. The Talon/BoWo interaction was worth the price of admission. But it's right now where we can really learn something. Nudge us in the right direction, what do we do next?
Ideas come from being emersed in a system developmemt process. The blank screen in Strategy Testing is a tough one. You just have to start and test something. Anything. Exit when its X% away from a moving average for example. Then look at the signals on a chart. Try something else. Look at the chart with the signals; the visual can really help stimulate the idea process.
Sometimes you discover something by accident because what you intended to code and what you coded, are not the same thing. Sometimes you discover something because what you coded gave you the exact opposite of the results you expected. That leads to a new path. Sometimes an idea you had is even a good one, though this is rare in the early stages.
If you get stuck on price exits or entry modifications go to time and see what that uncovers. Eventually you stumble into stuff that completes the system for you. And there are so, so, so many ways to improve and complete on the incredible information Talon has provided here.
He has really given those who continue to work on it a great XMas present. It's really all there. It just needs a little sweat to flesh it out. And it is nothing you need to be a quant to figure out.
Trading experience helps, but the best substitute for that is to look at the chart signals for everything you try. Anyone who will to put the time into working and refining this general framework will be rewarded with a very good system.
Sometimes you discover something by accident because what you intended to code and what you coded, are not the same thing. Sometimes you discover something because what you coded gave you the exact opposite of the results you expected. That leads to a new path. Sometimes an idea you had is even a good one, though this is rare in the early stages.
If you get stuck on price exits or entry modifications go to time and see what that uncovers. Eventually you stumble into stuff that completes the system for you. And there are so, so, so many ways to improve and complete on the incredible information Talon has provided here.
He has really given those who continue to work on it a great XMas present. It's really all there. It just needs a little sweat to flesh it out. And it is nothing you need to be a quant to figure out.
Trading experience helps, but the best substitute for that is to look at the chart signals for everything you try. Anyone who will to put the time into working and refining this general framework will be rewarded with a very good system.
There have been a lot of good posts in this thread and I confess I haven't been able to carefully go through them all. I know it sucks to ask people to do more work and then not carefully evaluate it, but if you have posted anything here you can bet some people are carefully going through it.
This is a tough time of year. Holidays take time, but also very important to spend a lot of time evaluating the past year's trading, especially the mistakes we made. I do a lot of writing and going back through old system sheets, etc. It takes an incredible amount of time and this has been my focus for the past week and will be for the rest of the year. Also important to come up with a written guide for the coming year, focusing more on our strengths and things we want to do more of.
Anyway, let's look at this little strategy. None of the testing we have done so far is a finished system, but I think we have evaluated some of the pieces well enough to know there's an edge.
What we know:
-There seems to be an edge to buying near the bottom of a short term lookback window (say 10-15 days).
-We have paired this with a simple exit of selling (to get flat) when price comes to the middle of an even shorter lookback period.
**Remember that any test like this with multiple conditions is a joint test of all of these conditions. It is possible that the pieces may not show an edge separately (that the whole may be greater than the sum of the parts.) It is also possible that the pieces may have a greater edge apart than together... that an exit and an entry may not work together. Just something to keep in mind going forward.
- It seems that the system's performance could be improved by avoiding signals when volatility is low.
Questions:
-We are not sure at all that long / short is symmetrical. We need to pay attention this and consider the implications. It is also possible (quite likely) that the long/short performance will become more balanced in future versions of the system.
-Does the longer term position matter? As a simple starting point, what if we only took signals above the 100 period moving average? Is 100 the right length? How do we avoid overoptimizing this?
-One way we know we are going to get hurt is buying as the market comes off (what turns out to be) a longer term high. Could we avoid this by, say, only taking buys below the 90% percentile of the past 100 days?
-Should the system have a stop?
-The exit is admittedly crude. Can we do better? Sometimes crude is more robust....
-Some systems benefit from only allowing X (say X=3) long entries in a row. Is this one of those?
-The idea of buying near a short term low is that we think the market is stretched to the downside. This system is a form of a mean reversion system. Could we improve this by adding more conditions to confirm the "oversold" condition? Short-term RSI, number of down closes in a row, large close down (maybe % based)... would any ideas like that help?
At this point we are still thinking conceptually... we know we have a valid core idea... before trying 30 different things out and keeping what works (which = a form of curve fitting) we should come up with a better plan from a combination of conceptual thinking and maybe testing other building blocks before putting them together.
the process is the whole point here imo. i think there have been several versions of this system posted here that actually are probably tradable as-is, but there's still more to learn.
i will continue to check in on this thread and offer what advice i can, but i apologize in advance -- i'll be pretty scarce until the new year. sorry!
This is a tough time of year. Holidays take time, but also very important to spend a lot of time evaluating the past year's trading, especially the mistakes we made. I do a lot of writing and going back through old system sheets, etc. It takes an incredible amount of time and this has been my focus for the past week and will be for the rest of the year. Also important to come up with a written guide for the coming year, focusing more on our strengths and things we want to do more of.
Anyway, let's look at this little strategy. None of the testing we have done so far is a finished system, but I think we have evaluated some of the pieces well enough to know there's an edge.
What we know:
-There seems to be an edge to buying near the bottom of a short term lookback window (say 10-15 days).
-We have paired this with a simple exit of selling (to get flat) when price comes to the middle of an even shorter lookback period.
**Remember that any test like this with multiple conditions is a joint test of all of these conditions. It is possible that the pieces may not show an edge separately (that the whole may be greater than the sum of the parts.) It is also possible that the pieces may have a greater edge apart than together... that an exit and an entry may not work together. Just something to keep in mind going forward.
- It seems that the system's performance could be improved by avoiding signals when volatility is low.
Questions:
-We are not sure at all that long / short is symmetrical. We need to pay attention this and consider the implications. It is also possible (quite likely) that the long/short performance will become more balanced in future versions of the system.
-Does the longer term position matter? As a simple starting point, what if we only took signals above the 100 period moving average? Is 100 the right length? How do we avoid overoptimizing this?
-One way we know we are going to get hurt is buying as the market comes off (what turns out to be) a longer term high. Could we avoid this by, say, only taking buys below the 90% percentile of the past 100 days?
-Should the system have a stop?
-The exit is admittedly crude. Can we do better? Sometimes crude is more robust....
-Some systems benefit from only allowing X (say X=3) long entries in a row. Is this one of those?
-The idea of buying near a short term low is that we think the market is stretched to the downside. This system is a form of a mean reversion system. Could we improve this by adding more conditions to confirm the "oversold" condition? Short-term RSI, number of down closes in a row, large close down (maybe % based)... would any ideas like that help?
At this point we are still thinking conceptually... we know we have a valid core idea... before trying 30 different things out and keeping what works (which = a form of curve fitting) we should come up with a better plan from a combination of conceptual thinking and maybe testing other building blocks before putting them together.
the process is the whole point here imo. i think there have been several versions of this system posted here that actually are probably tradable as-is, but there's still more to learn.
i will continue to check in on this thread and offer what advice i can, but i apologize in advance -- i'll be pretty scarce until the new year. sorry!
I'm not going to takeover this thread from talon and I wouldn't want to divert any questions from him to me, considering that it's his thread and I'm not the most "nicest" person on ET. I was planning on replying a lot earlier but I figured that I should be run a few tests on my end about the "mean reversion" system before I start commenting on this. Now that I've done my research...
Please understand that my development process is different from talon. I do run different development paradigms and most of my trading is not within a retail access (I have an advantage over commish).
Finally... this post intended for talon (and his pupil) to dig deeper into the "system" he provided and run more tests regarding the models they provided. I know they can do better than what they provided. As I mentioned... I'm not going to bother replying to most of the questions I may get from posting this...
That said...
1. The concept of the edge is still sketchy. All you guys know so far is that it works. When a viable edge exists, the scalability of the exposure can be expanded. The provided signal should be taken as an entry point to finding out what is happening. You need to come up with an hypothesis of this and get closer to the core. One suggesting hypothesis to consider is:
"The market reverts back due to some institutional trading affecting the market. Mean reversion within a long term is used by traders who:
Trade Options Expiration (Theta).
Statistical Index Arbitrage.
Etc. Etc. Etc. ()
Can this strategy be a single leg exposure of the impact of the above?"
Then you design a test to test the shit out.
2. Actually, I've given 1/2 the answer above but once you find a conditional "correlation". The next step is to simplify the "confirmed concept". Is the provided signal an "optimal" signal to expose the "core concept"? To give you the answer... the answer is NO (but that's my opinion), it's not enough largely because of risks.
What needs to be thought out is to come up with a signal that will provide the trader, with the appropriate risks levels of the current status of the "concept". All signals do not last forever, markets change... So where would you cut it? Where would you re-evaluate (calibrate... optimized... etc. etc.)?
Simply... you want the signals to be pro-actively managable to reduce the impact for the worst case scenario.
3. Once you have a viable signal... then comes the little tweaks that all you guys like to with stops and etc. What's missing is the "risk management concept" that you should have derived from the previous step of exposing an applicable signal. Without the RM concept, you're simply "curve fitting" the equity curve which makes the equity vulnerable to larger losses on the worst case scenario. The worst thing you can do is applying useless stops and rules that can blind the trader during times that the system has stopped working (or requries re-evaluation...). There's plenty of ways to kill a good signal with the wrong stops / position sizing.
---------------------------------------------------
And please...
Don't start telling me something like, "You do all that shit because you don't have an edge!!! If you have an edge, like talon, all your shit is useless..." or "There's no curve-fitting if you deal with an edge...".
I don't like semantical debates. (And again... I've done my own set of research before posting this... I have a better idea of what I'm than most of you may think...)
Please understand that my development process is different from talon. I do run different development paradigms and most of my trading is not within a retail access (I have an advantage over commish).
Finally... this post intended for talon (and his pupil) to dig deeper into the "system" he provided and run more tests regarding the models they provided. I know they can do better than what they provided. As I mentioned... I'm not going to bother replying to most of the questions I may get from posting this...
That said...
1. The concept of the edge is still sketchy. All you guys know so far is that it works. When a viable edge exists, the scalability of the exposure can be expanded. The provided signal should be taken as an entry point to finding out what is happening. You need to come up with an hypothesis of this and get closer to the core. One suggesting hypothesis to consider is:
"The market reverts back due to some institutional trading affecting the market. Mean reversion within a long term is used by traders who:
Trade Options Expiration (Theta).
Statistical Index Arbitrage.
Etc. Etc. Etc. ()
Can this strategy be a single leg exposure of the impact of the above?"
Then you design a test to test the shit out.
2. Actually, I've given 1/2 the answer above but once you find a conditional "correlation". The next step is to simplify the "confirmed concept". Is the provided signal an "optimal" signal to expose the "core concept"? To give you the answer... the answer is NO (but that's my opinion), it's not enough largely because of risks.
What needs to be thought out is to come up with a signal that will provide the trader, with the appropriate risks levels of the current status of the "concept". All signals do not last forever, markets change... So where would you cut it? Where would you re-evaluate (calibrate... optimized... etc. etc.)?
Simply... you want the signals to be pro-actively managable to reduce the impact for the worst case scenario.
3. Once you have a viable signal... then comes the little tweaks that all you guys like to with stops and etc. What's missing is the "risk management concept" that you should have derived from the previous step of exposing an applicable signal. Without the RM concept, you're simply "curve fitting" the equity curve which makes the equity vulnerable to larger losses on the worst case scenario. The worst thing you can do is applying useless stops and rules that can blind the trader during times that the system has stopped working (or requries re-evaluation...). There's plenty of ways to kill a good signal with the wrong stops / position sizing.
---------------------------------------------------
And please...
Don't start telling me something like, "You do all that shit because you don't have an edge!!! If you have an edge, like talon, all your shit is useless..." or "There's no curve-fitting if you deal with an edge...".
I don't like semantical debates. (And again... I've done my own set of research before posting this... I have a better idea of what I'm than most of you may think...)
Hi,
I have benefited a lot from the thread and would like to thank talon and others who posted here.
@TSGG
"Then you design a test to test the shit out."
I am wondering how you go about designing a test for such hypothesis. Can you suggest a framework? An example will be of immense help.
I have benefited a lot from the thread and would like to thank talon and others who posted here.
@TSGG
"Then you design a test to test the shit out."
I am wondering how you go about designing a test for such hypothesis. Can you suggest a framework? An example will be of immense help.
- Status