Strategy Desk backtesting results

O

Oliver84

I'm fairly new to trading. I've been messing around with strategies with TD Ameritrade's software and was wondering how accurate these backtesting results are.

I know backtesting can never be right on because in back testing the closes interval is 1 minute but in real time it checks the market every second so since there isn't second-by-second data feeds available, you'll never get right on. But with the 1 minute intra bar setting checked, I was told you can get pretty close.

Anyway, the reason I ask is because I came up with a strategy that is giving me amazing results. I've backtested it till November of last year and every month is seems to give me profits averaging 400%+ per stock. Some times even 1000%+.

The weird thing is that I haven't seen any loss in any of the trades... I know this can't be but I've check a ton of trades and the ones that don't show a profit yet have always been because the formula is still long on the position and it's still up.

My question is, has anyone seen something like this and how close to reality can this be? In backtesting it's given me consistent results for almost a year. I just started testing it live these past couple of days and so far it's been right now. But obviously a couple of days isn't enough to really know. I just wanted to get your opinions before put some real money into this. Thanks!
 
The most likely cause is a programming error in which your script uses some type of unknowable (future) data to generate its trades. Such a system will only buy when it sees the price rising a certain amount in the future. This will generate impressive results but cannot be replicated in real life since nobody knows future securities prices. So that "buy" signal that always results in a profit cannot be generated.

Without seeing your script :D I can't know for sure but this is a likely cause. I wouldn't be trading real money with it until you are sure no future data is being used. Also the recent runup (assuming you were back testing recent data) could cause a long only system to show a 100% win rate. But in order for those results to continue this stock market would have to continue rising at the astounding pace of the past months.

Finally, if you want to do serious back testing I'd suggest not using SD. It is great software for generating signals, but horrible for backtesting. WealthLab for example is much, much better.
 
Forgive my ignorance, but how can your strategy use future data? I'm using mainly stochastic and MACD in my entry strategy with most of them looking a few bars back.

Also, I've heard a lot about strategies and keeping up with the market movement. Using these indicators that are usually just present when the price is on an upward trend, why would market movement matter that much? If the market goes south, wouldn't the formula just not alert any more buy? Do certain indicators "indicate" the opposite when the market turns around?

The reason I'm using SD right now is because it's free and I know it's OTCBB compatible. I'm not sure if wealthlabs or which other ones are OTCBB compatible and since I'm dealing mainly with OTC stocks right now, I don't want to pay that money for something that won't do me any good right now.
 
Oliver:

Take a look at all the standard index ETFs (SPY, QQQQ, etc) as well as the Dow, NASDAQ, and Russell from November 2008 to date. Up!. Now go back to Nov 2007 and look at the same symbols - Down! Backtest your strategy against the same issues from November 2007 forward and see how it does.

If you are doing OTCBB issues be aware that many of them are very thinly traded and backtested results, which do not include your trading, are highly suspect.

How did you pick the issues? With an automated screener? If so, you have done a form of data-mining. Nothing wrong with data-mining but it must be done correctly. Use caution and read up on data-mining, survivor bias, optimization, and curve fitting. If you selected the issues based on specific targeted research then you may be on to something.

Jack
 
Quote from Oliver84:

I'm fairly new to trading. I've been messing around with strategies with TD Ameritrade's software and was wondering how accurate these backtesting results are.
More...

The problem is either

1. The software
2. The data
3. Your coding of the parameters

In backtesting, you can be certain that whenever the results appear to be too good to be true, you have errors that need to be tracked down.

One place to start is in looking at the reported results and comparing them to charts of the past data and actual trades. That can often help you to visually track down where you're making your errors in entries and exits. Because somewhere, you do have errors.

It's easy for someone new to backtesting to think that he's stumbled upon the holy grail. Sorry to bust your bubble, but it isn't going to happen. :(

To quote my dear mother, "Sweetie, money doesn't grow on trees."
 
Example of newbie commom backtesting error that results in extraordinary performance on paper:

If open of today > close of yesterday then
buy at the open at market;
sell at the high of today - 1 points
 
haha, yea I was afraid of that...

I have, however, looked at the charts and it seems to be good. Not sure what you're talking about there or what I'm suppose to find but it seems like it's buying and selling at the correct times.

As far as volume of these stocks, they are usually between 5 Mil - 300 Mil and the price range is usually .001-.1

The problem with backtesting from Nov 2007 is that many of these stocks either did not exist or were in a different market or were out of my price range so they would not have worked. If I could find historical data to test stocks that meet these requirements back in 2007, that would be great to test and see what happened.

So what would be a good place to start looking for errors? The formula isn't very complicated so I don't think it would be too hard to find the errors. I just don't know what I'm looking for...
 
Oliver, if you don't have any losses among your trades then you have a problem. :) When you find the problem with your code, your strategy will most likely turn into a losing strategy. Try not to despair though that's what searching for a decent automated system is all about. It's hard to look for a problem with your strategy when you WISH so much that the results were real.
Good luck.
 
Can't you extend your backtest to cover the preceding two years? If it still shows huge gains and those gains also occurred during the decline you almost certainly have a future leak (peeking ahead).

Find some stocks which have not recovered from the decline. Look at financials like PMI or C, or a consumer like LIZ. See what your system shows. All three are off their lows but they certainly should have lost money over the past year. Unless your earlier post did not provide enough detail about the stocks with the huge gains and when your system bought them, the performance is too good to be true. What is the raw winning trading to losing trade ratio?. I once had a system that showed a 9:1 win/loss ratio. It once took me a week to find a simple error because I was using new software and was unaware of a subtle difference between it and another vendor's package. Once I corrected the error the system was mediocre and I discarded it.

Jack
 
You must log in or register to reply here.
Back
Top