How should you optimize a trading system?
- Thread starter travis
- Start date
No, don't worry, I am not offended. However, I hope you are wrong, and I think you could be wrong.
Just because someone is not that good at Math and shies away from formulas, that doesn't mean he is in no position to make any money from trading. Maybe I am getting everything else right. And maybe someone else who loves formulas is getting everything else wrong.
You don't have the absolute need to understand the Sharpe ratio to be able to make money with an automated system. Probably I could even understand the Sharpe ratio, but I don't feel like it's worth the effort to get into all these formulas.
For example, the RSI and most other indicators are too complex for me to get into all their implications, so I only use HIGH, LOW, CLOSE and one moving average. I use time ("what time is it in the day?"). All simple things that I completely and thoroughly understand.
For the same reason, people who write here and seem almost illiterate seem to me more reliable than people who write in a very good and proper English, because I am sure they are not wasting time and get to the point of making money. Otherwise all Computer Science students by now would be millionaires. Instead, I think a lot of science people are getting lost in the technicalities. I don't want to get lost in the technicalities and I think the Sharpe ratio is a technicality (and so are most indicators in my opinion).
But don't get defensive either, and if you want to explain something to me about optimization, please do it and thank you in advance. You can do it with simple terms, as I am sure it can be done. I have been able to understand threads on this forum for years without understanding the Sharpe ratio.
Just because someone is not that good at Math and shies away from formulas, that doesn't mean he is in no position to make any money from trading. Maybe I am getting everything else right. And maybe someone else who loves formulas is getting everything else wrong.
You don't have the absolute need to understand the Sharpe ratio to be able to make money with an automated system. Probably I could even understand the Sharpe ratio, but I don't feel like it's worth the effort to get into all these formulas.
For example, the RSI and most other indicators are too complex for me to get into all their implications, so I only use HIGH, LOW, CLOSE and one moving average. I use time ("what time is it in the day?"). All simple things that I completely and thoroughly understand.
For the same reason, people who write here and seem almost illiterate seem to me more reliable than people who write in a very good and proper English, because I am sure they are not wasting time and get to the point of making money. Otherwise all Computer Science students by now would be millionaires. Instead, I think a lot of science people are getting lost in the technicalities. I don't want to get lost in the technicalities and I think the Sharpe ratio is a technicality (and so are most indicators in my opinion).
But don't get defensive either, and if you want to explain something to me about optimization, please do it and thank you in advance. You can do it with simple terms, as I am sure it can be done. I have been able to understand threads on this forum for years without understanding the Sharpe ratio.
Some of the approaches in the area of statistics and machine learning might be useful to look into.
For example, cross-validation is a common method to mitigate the problems of curve-fitting to some extend. Walk-forward testing can be considered a particular implementation of cross-validation, but many other approaches are possible as well.
Perturbation is another possibility: You add a small random number to your model inputs, and see to what extend this changes the outcome, i.e. how well your optimization algorithm can deal with this.
For example, cross-validation is a common method to mitigate the problems of curve-fitting to some extend. Walk-forward testing can be considered a particular implementation of cross-validation, but many other approaches are possible as well.
Perturbation is another possibility: You add a small random number to your model inputs, and see to what extend this changes the outcome, i.e. how well your optimization algorithm can deal with this.
Ah yes, I see. Well, perturbation is similar to what I already did by taking the system tailored for the EURO FX (USD/EUR) and using it on the USD/GBP and the YM (future on the Dow Jones). Both systems work on those two other markets almost as well as on the USD/EUR and without changing any settings.
I assume you really mean to avoid over fitting not optimization, at least as I understand the term.
To me optimization means taking the basic elements of the system design and finding the best way to use them to trade a particular market in a particular time granularity. A given reversal trading system might work great on 15 minute bars but fail on daily bars. Optimization is good if it can adapt the general system to work on both by creating subcomponents for the specific time granularity.
As to the question of how to make it work in the future when the market permutes and evolves, the answer depends on the sophisticaton of your development process. Mine is very automated and based on neural network models in contrast to the more common linear rules. So I don't know if my approach will work in the linear world, but here it is:
1) Take a measure of general market conditions over the long term, say trending or trading range and separate all data, historical and live on this basis. Develop a separate sub system to trade each. This avoids having to create one general purpose
edifice to handle all conditions.
2) Take a measure of market short term market conditions over the last few bars and do the same,
3) Abandon the traditional development perspective on data segregation, which is something like this, assuming one is trading daily bars and has historical data form 1980 to 2008:
System development and learning: 1980 to 1998
Testing and refinement: 1999 to 2003
Paper trading: 2004 to 2008
A much better approach is to use a "stripped" data segregation approach:
System development and learning: 1980, 1983, ....2006
Testing and refinement: 1981,1984,... 2007
Paper trading: 1982, 1987, .... 2008
As I said this approach works well with non-linear models, often increasing the Profit Factor by a significant amount. I can't speak
to the results in linear rule based systems.
Jerry030
Your testing should be done on in sample and out of sample data to avoid curve fitting. Always save some data and never look at it until your ready. The ideal program should be robust and run across a multitude of markets. Even if the program does not perform profitable on a certain market IE gold, it should have respectable results where the amount of trades are considerably lower than average and the amount lost should not be devastating. As far as optimization methods such as genetic algorithms, you should always run the tests and look for statistically relevant results. If you can pinpoint an area that holds true across multiple markets it may be trying to tell you that a certain parameter can be adjusted to the next common setting. IE an SMA of 20 performs better across everything at "43", you can try rounding it up to the nearest commonly used number and try an SMA of 50 on your untouched data.