kubilai:
Thanks for the encouragement, Jerry.
***
- You're welcome.
***
I can certainly see such returns to be possible, though the capacity for such systems would be fairly low, correct? Strong market inefficiencies must be small/niche/limited in scope...
***
- No, I find it to be the reverse. A really sophisticated system
(a lot of work to do this) is much more robust than an indicator based linear rule system. We've all heard of these where they trade markets in a trading range but get killed in trending markets...and you end up knowing youâre in a trend after a few huge losses.
***
Data mining has a bad name among traders.
***
- This is true for several reasons:
1) Traders without the necessary background in statistics, computer science and system theory try to do what they think is data mining and fail so they announce that it doesn't work.
2) The software companies that exploit gung-ho traders with a few thousand to spend on software use the buzz word to sell units of software. The software in many cases is so poorly designed and stripped down for novice use that even someone with the necessary skills couldnât do much with it.
3) Those who have discovered how to do it want to keep others from the same discovery. So there is a certain amount of deliberate mis-information put out by "experts". Many markets like futures are zero sum games...for every dollar of profit somebody has to have a dollar of loss. So the concept is put out that the best system is simple, with a few rules that works on all markets, avoid complexity and sophistication. This insures that there will always be a fresh crop of losers who take this advice and prove it doesn't work with their losses.
***
So you must have some pre-conditions for applying these techniques successfully to the trading world. The book I'm focusing on these days: Design, Testing and Optimization of Trading Systems, suggests that two conditions must be in place to ensure successful optimization: logical basis to the strategy, and out-of-sample testing. Do you apply these?
***
- My experience has been otherwise in terms of the logical part.
I don't assume or look for visible logic I can understand as the models I use are "black boxes" automatically generated by the modeling application.
I couldnât understand why the connections between the neurons are weighted the way they are if I wanted to. The only logic is does it work in the market.
Out of sample, yes, that is critical and must be extensive. A typical model uses 10,000 bars to train, another 2000 to develop trade strategy from the model output and then another 2000 in OOS to see if it works once everything is frozen. The common mistake is to look at the training output and if that looks good, to start trading. If learning stopped in a local minima you'll see a lot of loosing trades very quickly.
***
Is anything else needed to ensure the resulting model is predictive of the future?
***
An R2 function above 0.6 is good along with 30 to 60 live paper trades that have a distribution pattern of wins and losses that match the OOS result.
****
Do you use data mining only for optimization or is it useful for coming up with the initial strategy too?
***
- Both work. I let the modeling system create the trading system to point of an entry signal then create a trading strategy using a combination of manual effort and genetic algorithms.
Jerry
**
Cheers,
Kubilai
Thanks for the encouragement, Jerry.
***
- You're welcome.
***
I can certainly see such returns to be possible, though the capacity for such systems would be fairly low, correct? Strong market inefficiencies must be small/niche/limited in scope...
***
- No, I find it to be the reverse. A really sophisticated system
(a lot of work to do this) is much more robust than an indicator based linear rule system. We've all heard of these where they trade markets in a trading range but get killed in trending markets...and you end up knowing youâre in a trend after a few huge losses.
***
Data mining has a bad name among traders.
***
- This is true for several reasons:
1) Traders without the necessary background in statistics, computer science and system theory try to do what they think is data mining and fail so they announce that it doesn't work.
2) The software companies that exploit gung-ho traders with a few thousand to spend on software use the buzz word to sell units of software. The software in many cases is so poorly designed and stripped down for novice use that even someone with the necessary skills couldnât do much with it.
3) Those who have discovered how to do it want to keep others from the same discovery. So there is a certain amount of deliberate mis-information put out by "experts". Many markets like futures are zero sum games...for every dollar of profit somebody has to have a dollar of loss. So the concept is put out that the best system is simple, with a few rules that works on all markets, avoid complexity and sophistication. This insures that there will always be a fresh crop of losers who take this advice and prove it doesn't work with their losses.
***
So you must have some pre-conditions for applying these techniques successfully to the trading world. The book I'm focusing on these days: Design, Testing and Optimization of Trading Systems, suggests that two conditions must be in place to ensure successful optimization: logical basis to the strategy, and out-of-sample testing. Do you apply these?
***
- My experience has been otherwise in terms of the logical part.
I don't assume or look for visible logic I can understand as the models I use are "black boxes" automatically generated by the modeling application.
I couldnât understand why the connections between the neurons are weighted the way they are if I wanted to. The only logic is does it work in the market.
Out of sample, yes, that is critical and must be extensive. A typical model uses 10,000 bars to train, another 2000 to develop trade strategy from the model output and then another 2000 in OOS to see if it works once everything is frozen. The common mistake is to look at the training output and if that looks good, to start trading. If learning stopped in a local minima you'll see a lot of loosing trades very quickly.
***
Is anything else needed to ensure the resulting model is predictive of the future?
***
An R2 function above 0.6 is good along with 30 to 60 live paper trades that have a distribution pattern of wins and losses that match the OOS result.
****
Do you use data mining only for optimization or is it useful for coming up with the initial strategy too?
***
- Both work. I let the modeling system create the trading system to point of an entry signal then create a trading strategy using a combination of manual effort and genetic algorithms.
Jerry
**
Cheers,
Kubilai
