You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
Lessons Learned From Profitable System Development
- Thread starter TriPack
- Start date
To start the thread off, let me share a few things I've learned from system development. This is from developing a counter trend system.
1) Filtering out smaller moves is important. This does two positive things for the system, reduces the # of whipsaws (entry in a flat market) and increases the % profitable. One side-effect is that whenever you filter you introduce lag. See:
http://www.elitetrader.com/vb/showthread.php?s=&threadid=21739
2) Only taking entry signals on bar close, rather than accepting intra-bar signals. This has the same positive and negative effects as Filtering moves, and is in fact a type of filter. But it is such an effective filter that it bears mentioning by itself.
3) Timing the trade: Place buy order on weakness, and sell order on strength. Even with a method like a breakout method, it is possible to implement this strategy by only entering on a pullback after the breakout has occurred. For a countertrend system, a breakout is the pullback.
4) Filtering the raw data, then applying simple analysis to the filtered data is superior to using raw data and applying complex analysis to the raw data.
1) Filtering out smaller moves is important. This does two positive things for the system, reduces the # of whipsaws (entry in a flat market) and increases the % profitable. One side-effect is that whenever you filter you introduce lag. See:
http://www.elitetrader.com/vb/showthread.php?s=&threadid=21739
2) Only taking entry signals on bar close, rather than accepting intra-bar signals. This has the same positive and negative effects as Filtering moves, and is in fact a type of filter. But it is such an effective filter that it bears mentioning by itself.
3) Timing the trade: Place buy order on weakness, and sell order on strength. Even with a method like a breakout method, it is possible to implement this strategy by only entering on a pullback after the breakout has occurred. For a countertrend system, a breakout is the pullback.
4) Filtering the raw data, then applying simple analysis to the filtered data is superior to using raw data and applying complex analysis to the raw data.
NinjaTrader_Dierk
ET Sponsor
Quote from TriPack:
4) Filtering the raw data, then applying simple analysis to the filtered data is superior to using raw data and applying complex analysis to the raw data.
Could you please elaborate on this issue? What are your experiences about quality of known data providers (.e.g. Trackdata) ?
Your comments will be appreciated.
Dierk
I'd suggest doing a search for the data provider you wish to know more about on this site, since this thread isn't really about any specific data vendor (best left to other threads to keep things on topic).
As for the filtering issue, you have an issue of filtering bad ticks, and of filtering price into price bars.
For more info on filtering for bad ticks see:
http://www.interactivebrokers.com/discus/messages/2/8742.html
For some of my thoughts on filtering price data into price bars, see:
http://www.elitetrader.com/vb/showthread.php?s=&threadid=21739
As for the filtering issue, you have an issue of filtering bad ticks, and of filtering price into price bars.
For more info on filtering for bad ticks see:
http://www.interactivebrokers.com/discus/messages/2/8742.html
For some of my thoughts on filtering price data into price bars, see:
http://www.elitetrader.com/vb/showthread.php?s=&threadid=21739
Quote from droth:
Could you please elaborate on this issue? What are your experiences about quality of known data providers (.e.g. Trackdata) ?
Your comments will be appreciated.
Dierk
Quote from TriPack:
I'd suggest doing a search for the data provider you wish to know more about on this site, since this thread isn't really about any specific data vendor (best left to other threads to keep things on topic).
As for the filtering issue, you have an issue of filtering bad ticks, and of filtering price into price bars.
For more info on filtering for bad ticks see:
http://www.interactivebrokers.com/discus/messages/2/8742.html
For some of my thoughts on filtering price data into price bars, see:
http://www.elitetrader.com/vb/showthread.php?s=&threadid=21739
I heard some people said first pre-processed (e.g. remove spikes) the data before applying any TA to it. Is it possible for a day trader that holding positions for minutes to do that?
Quote from 0008:
Is it possible for a day trader that holding positions for minutes to do that?
To do what? If you're asking if you can apply filtering and still create a daytrading system, the answer is yes. The filtering will tend to smooth your raw data out so that your trading technique has a better chance to work without getting false signals.
My own view is that systems are really pretty stupid - they will do things that rational people simply won't do. This is why they go through periods of whipsaws, and ironically this is also why when a system works, it really works well.
- Ah saw you edited. Possible to do, but your datafeed or charting program must do it for you (built in options), or you need to programattically do it and then feed it into your system, which may or may not run off from a charting program like TradeStation. It's easier to filter the data than it is to deal with data spikes in your system.
It doesn't have to be very time-consuming. I do data filtering on a tick by tick basis before using the data in my programs and the data delay is negligable. I don't do fancy filtering, just simple % change from a baseline "good" price. Take a look at the thread link I posted on the IB forum two posts up. In a nutshell, if I get more than a certain # of bad ticks (say 10-20) at a price level, then I move the baseline to that price level. It's fast, pretty effective (still lets some bad ticks through) but surely better than getting all those bad ticks.
Are your methods quite subjective? It is hard to determine which one is a bad data. And some spikes are huge but some are not. I know some professors of statistics did research in this topics. But I remember that they couldn't make a very great improvment of the accuracy of the forecasting (it was not about trading).
I agree "Filtering out smaller moves is important". But I think this is very very hard, no matter what methods you use. Otherwise we could get a 100% winning system! I tried many methods, but no one could remove all the wipesaws. And you?
I agree "Filtering out smaller moves is important". But I think this is very very hard, no matter what methods you use. Otherwise we could get a 100% winning system! I tried many methods, but no one could remove all the wipesaws. And you?
My take on going down the systems development road:
1. Simpler is better. Have to watch myself on this one as the love for computers & math will lead me astray. The really cool stuff like neural nets and signal filtering algorithms can help a little, but don't make the mistake of equating higher tech with higher performance in trading. There seams to be an inverse relationship here. Some folks seem to make this work, but they're usually so deep in it that its hard for others to understand what the heck they're doing (see HarryTrader!).
2. Get yourself a solid backtesting platform and use the hell out of it! You'll be amazed at how much of the stuff out there DOESN'T work. IMHO that's the most important feature of a backtesting platform - demonstrating that something doesn't work. This will also provide much needed exposure to the market's workings.
3. Search out and read all posts here and on other similar websites that are put up be systems traders. Other good websites for this include the message areas of the backtesting software maker's websites. You'll discover the serious systems traders by just browsing or searching the site. These posts provide more direct and timely information than most books I've read.
My systems: Have two countertrend and one trend follower. The countertrends only trade about 25 times a year for 1 or 2 days holding - they're my bear / choppy market winners. The trend follower is designed to sense a major trend and ride it as far as possible given its parameters - its a bull market trader.
Just my opinion,
Smitty
1. Simpler is better. Have to watch myself on this one as the love for computers & math will lead me astray. The really cool stuff like neural nets and signal filtering algorithms can help a little, but don't make the mistake of equating higher tech with higher performance in trading. There seams to be an inverse relationship here. Some folks seem to make this work, but they're usually so deep in it that its hard for others to understand what the heck they're doing (see HarryTrader!).
2. Get yourself a solid backtesting platform and use the hell out of it! You'll be amazed at how much of the stuff out there DOESN'T work. IMHO that's the most important feature of a backtesting platform - demonstrating that something doesn't work. This will also provide much needed exposure to the market's workings.
3. Search out and read all posts here and on other similar websites that are put up be systems traders. Other good websites for this include the message areas of the backtesting software maker's websites. You'll discover the serious systems traders by just browsing or searching the site. These posts provide more direct and timely information than most books I've read.
My systems: Have two countertrend and one trend follower. The countertrends only trade about 25 times a year for 1 or 2 days holding - they're my bear / choppy market winners. The trend follower is designed to sense a major trend and ride it as far as possible given its parameters - its a bull market trader.
Just my opinion,
Smitty