need advise - system trading

R

radolym

I trade intraday system that I developed and it's making money. I've been trading it for about 6 months now. When I was developing and testing my system I had 6 months of data.

Recently I got intraday data for 2003 and 2004 and I backtested the same system on new data. And it didn't work nearly as good, still made money in 03 and 04 but much less then I'm making now.

Here is my dilemma: I am making money now, should I care about the fact that it's been working for only last 6 months and didn't work in 2003 and 2004? It's a day-trading system and goes long and short.

Also, in your experience at what point you think your system stopped working?
 
It looks like your system is slightly over-optimized with the newer data. However, since it still backtested with a profit with out-of-sample data from '03 and '04, I would guess that the system is reasonably robust. If I were you, I'd try to see how I could tweak the system parameters slightly in order to increase the theoretical performance for '03 & '04. You will most likely have to sacrifice some level of current performance in order to achieve this.

I'd also try to see if it can be adjusted to get '03 & '04 performance to be on par with the current performance that you are getting. Then I'd try to correlate the differences between these 2 sets of parameters with some general characteristic of the market that has changed during this time. This characteristic would be something that changed very slowly over the past few years. If you can quantify this with something measureable, then you'd have additional data on how you might fine-tune your system to accomodate these slow changes in the general market.

Having this information would give you a better understanding of how your system might react to future changes in the market, and how you might need to tweak in in order for it to remain profitable. Since the system is doing well now, I would not stop trading it in its present state until its performance degrades over a period of at least 6 months.
 
Do you trade individual stocks? If so, how do you pick them? Have you changed your selection criteria? If you've changed your selection based on "market experience" then you may have selected stocks which work for your system. Nothing wrong with that at all, you just must be aware of it.

If you are backtesting the same issue(s) you are now trading then JJJ's comments are rather valid.

Jack
 
Quote from JackR:

Do you trade individual stocks? If so, how do you pick them? Have you changed your selection criteria? If you've changed your selection based on "market experience" then you may have selected stocks which work for your system. Nothing wrong with that at all, you just must be aware of it.

If you are backtesting the same issue(s) you are now trading then JJJ's comments are rather valid.

Jack
More...

Thanks for the comments.

I trade individual stocks and didn't change my selection criteria.
I understand market is changing all the time, the question is how to change with the market before going to larch drawdowns?
 
how many stocks are in your list, and how many trades in your backtesting?

creating a system on 6 months of data, then running that on 'out of sample' data is one way of having some confidence that the results aren't curve fit. did you do that also when backtesting the first six months of data (ie split the data for creating and testing) ?

depending on these there may be a possibility of some curve fitting. you could use at acrary's 'edge test' to determine that.

in what way are the results from 2003-4 worse? is it purely in profit, or is it dd, sharpe, number of trades etc. if its just profit but other factors are still high then i'd be less concerned, you'll work out over time why the change. but if there are decent draw downs in the systems past and if there is curve fitting, then the draw downs could return.

the simplest form of good protection is a consistently high sharpe ratio (or similar risk/reward indicator). the higher the sharpe is initially the more warning you will have if it drops, that something is changing, and you may even be able to stop trading while you're still making a profit.
 
sounds like some form of over-optimization is going on..does your system use any fixed number parameters, eg it needs a 17-period moving average to give good results on the 6 month data but if you use a 15, 16 or 18, 19-period moving average then the results are much worse? that could indicate curve-fitting
 
Since you are trading individual stocks, I would suspect that you are in sectors that maybe were not as hot in those earlier years. It could also depend on the type of strategy. If you are fading the extremes, ie using some sort of OB/OS method, then the tighter range of the past year and lack of trending may be helping you.

You might want to backtest on an index to see what affect that has. Lot of variables that can affect this.
 
You must log in or register to reply here.
Back
Top