System testing questions.
- Thread starter jim c
- Start date
Here are the full results with the new filter of vix > 20. It seemed to help alot. I have checked once again for possible errors but havent found any yet. It seems trade worthy. I am still concerned about the small # of trades. I will do some testing on an additonal 5 years and see what I come up with. Any one have any thoughts on this? Jim
Hey Jim,
Which Trading software are u using?
How do you get the Fair Value of the market? Do u calculate it on fly.
I have made my own strategies on TD Ameritrade Strategy desk and have been back testing for sometime.
Even I am planning to trade only those stocks which have been profitable during my backtesting.
By the way adding VIX or even choppiness factor or lets say BB with SD=1 was a good idea. I am planning to incorporate that in my Strategy. This will reduce the trades that happen due to noise factor.
One more idea - dont trade when -DI > +DI and -DI has been rising and ADX > 20. This will remove trades that happen when stock is gradually declining. In your case it will be reverse.
What does PF stand for? I know pretty lame qusn. What is the time frame u r working on such as daily bars or 60 min or just min by min ticks?
What is your suggestion?
Sumeet
Which Trading software are u using?
How do you get the Fair Value of the market? Do u calculate it on fly.
I have made my own strategies on TD Ameritrade Strategy desk and have been back testing for sometime.
Even I am planning to trade only those stocks which have been profitable during my backtesting.
By the way adding VIX or even choppiness factor or lets say BB with SD=1 was a good idea. I am planning to incorporate that in my Strategy. This will reduce the trades that happen due to noise factor.
One more idea - dont trade when -DI > +DI and -DI has been rising and ADX > 20. This will remove trades that happen when stock is gradually declining. In your case it will be reverse.
What does PF stand for? I know pretty lame qusn. What is the time frame u r working on such as daily bars or 60 min or just min by min ticks?
What is your suggestion?
Sumeet
Short positions and no stops = disaster.
Hello! Thx for taking a look. I have used excel for backtesting my stuff. I use IB for trading. I am not an expert on backtesting and I can only test daily data for now. I do not have data for fair value. No idea where I might get it. Nitro suggested I backtest with vix>20 as a filter and I posted with the new results.
With one suggestion I improved the system considerably. (I dont know if it qualifies as a "system" just yet) This is the type of discussion I was trying to start. I have not tried any of the ADX, BB, or DI stuff but I willing to take a look for sure.
PF stands for profit factor. It is simply gross profits/gross losses. If your PF is a 2 you basically make 2$ on your wins and 1$ on your losses. Just a basic way to measure performance.
One of my questions was should you just trade the stocks that perform well? Or is this a form of curvefitting? It is not very active on one particular stock. Spy only had 33 trades over the 5 year test, and only 22 with the vix filter. I use trade ideas to scan for the setup in real time. ( I have not traded this "system"at all)
Again, I only use daily data at this point to get a rough idea if this would be profitable. I will look into some of your suggestions. Thx again for the reply. Its good stuff to think about.
Jim
Thanks Hook. I agree with you on this. At this point I dont have a great way to backtest using stops. I backtest on daily data with no stops and see how it goes. When I find something that seems workable I will then look into the smaller time frames and get some real time signals with Trade Ideas and see how it looks. It seems with some systems stops actually hurt the performance..esp for mean reverting type stuff.
I have been trading a simple breakout system or a couple of months now. The farther away I put my stops the better it has performed. I have really struggled with this as I love to use a tight stop. Anyway thx agin for chimming in. Jim
It really doesn't matter how great it looks with no stop. That is the definition of curve-fitting. Could you really trade without a stop? How would you feel if you shorted S&P 660 or whatever it was now that it has rallied 200+ points? Would you still be holding? I doubt it.
ANYONE can design a system that looks good with no stop knowing the performance of that market over the test period. For instance, do a test on EURUSD from beginning of 2002 to beginning of 2008. Long-only with no stop. How many losers do you think you'd have? That's right, 0. How would you have done through the 3000 pip drop that followed? A system with no stop and no mechanism to reverse isn't a system but rather a quick route to financial ruin.
I apprectiate the response. This is just a rough way to see of the system would work at all. I would like to think that using stops with this system would actually improve it. ( I deff think it would, esp after looking at the MAE) Maybe you could talk a little more about how you think using no stops would be considered curve fitting. Again, I really apprectiate the response! Jim
Jim,
Curve fitting has a strong correlation to number of parameters. Most system traders aren't aware of all the hidden parameters that a system has. For example, using a particular date range to test a system against is a hidden variable. Or how about which symbol to test a system against is a hidden variable. Basically, any decision that you make is potentially a variable, that while not necessarily explicit in your actual system code, nevertheless exists as a constraint on it's performance, and therefore could be selecting a curve fit. The act of not having stops is clearly a hidden variable.
If you said something like,
Code:
VIXVALUE = 15
if VIX < VIXVALUE then no stops
else
stops = ATR(oversomeperiod)That would make the variable explicit, and should probably be done for anyone that doesn't have infinite resouces.