Strategy Desk backtesting results

If I test any stock that is above my price range of .001-.1 the strategy will not work as good.

I tested LIZ from May 2007-present and it didn't buy anything until Dec 2008 and after that it had 2 profits, one for 40% and one for 30% and then a loss of 3%.

PMI didn't buy in till July 2008 with an average of 15% gain, no loss.

C bought in on March 2009 at 1.32 and sold off in Aug 2008 at 4.92 for a 262% gain.

So I guess I do need to clarify that. The formula only does this well with certain stocks. Mainly stocks from the OTCBB with a Volume of at least 1 Mil and price range between .001-.01 per share. Anything out of this and it will have some moderate loss and moderate gains.
 
BB stocks have terrific moves.. I researched this years ago.. you actually won't get filled on those BB stocks though.. the rules are different, your limit order can sit there while price trades right through it, it can sit there for weeks after that actually.. the MM that is holding it doesn't have to fill it unless he wants to trade through it with his own money, it's not up to the aggregate market... so toss those BB stocks out of the testing...

I would also search my code for use of future knowledge.. like does an indicator calculate on the close of the current bar but you are buying during the current bar... if all your orders are on the open of the next bar you are ok on that one but many backtesters allow you to use future knowledge...
 
Quote from Oliver84:

haha, yea I was afraid of that...



As far as volume of these stocks, they are usually between 5 Mil - 300 Mil and the price range is usually .001-.1
More...

LOL. You're looking at penny stocks. You're probably catching something at the bid and something and ask and thinking you can sell at those prices. You can't. Backtest on the NAZ100, and then come back to us when you found something that works.
 
Quote from bwolinsky:

LOL. You're looking at penny stocks. You're probably catching something at the bid and something and ask and thinking you can sell at those prices. You can't. Backtest on the NAZ100, and then come back to us when you found something that works.
More...

ok, I figured I'd get flamed on about the penny stock thing. So I was expecting that. I did, however, backtest the NAZ100 from May until now and got unexpected results. I was expecting to get some loss but it actually did pretty good. I got 0% loss and 100% wins, again and all I did was adjust the formula due to the price difference of these stocks. These stocks are much more expensive so I was buying them at $1,000 per trade instead of $200 per trade that I did with my penny stocks. The other adjustment was in my exit strategy.

Anyways, I posted the results of the top gainers so you can see. Gains average about 30 or 35%. I really don't understand what's going on. I tried to find the future errors but most of the indicators are looking at a few bars ago in the past, precisely to avoid those errors. I'm also looking at the bid and the close. I tried to make it so it buys at the close, not the bid or the ask.

Granted with penny stocks my percent gains were much much higher but the case still remains that all the triggers where wins even when testing the NAZ100
 

Attachments

Almost all of the "Sell" times shown in your attachment seem to be intraday, not the close time. Are these limit orders or market orders? How does the TD Ameritrade simulator handle each type?

Jack
 
I'll give some other ways one can introduce errors into backtesting such that the results are too good to be true.

Awhile back I was reading The Encyclopedia of Technical Indicators, looking for some general knowledge about indicators. Each one was testing on the S&P500 over something like the past century. For each one, it would buy/sell at the day's closing price given a condition at the closing price. This cannot be; the market is closed when the closing price emerges. If you ranked the indicators, the ones that most closely approximated the real closing price were the best, since that was the closest to best information you could get for that fictitious scenario. If you try to apply the indicators using closing price, but at opening the next day, you'd find they'd fail terribly.

I had written my own backtester for an automated swing trading tool on which I tested this all out. I eventually tried to salvage something from the indicators be combining the least losing ones in a neural network. This network did best with inputs in the range 0-1, so imagine trying to map all those inputs to fit the range.

For performance, I'd do all my calculations up front in arrays. The downside is that it made it real easy to incorporate future data. I have two examples. In the first, I was calculating slopes of points using a polynomial formula. I'd get to the last data point, and switch to a different formula since I had no more points to make a slope. Just by incorporating a future x+1 point in the slope was an error since I'd never have that info, but then I switched formulas right at the end. So my neural network got trained using a lot of points with slope formula A, but every day I'd be using the last data points, which were calculated with formula B. Needless to say, forward-tested results didn't match back tested results.

For the 0-1 mapping strategy, I was mapping a window of points to a normal distribution to fit the various indicators into that range. I had made a mistake and was using the full test window to calculate the mean and standard deviation, instead of using a rolling window only up to the target point in question. So in backtesting, it basically had hints of the future based on that false mean and standard deviation. It seems subtle, but this was the difference from turning $12,000 to $2,000,000 in a year and turning $12,000 to $200 in a year.

I would recommend some forward testing and you should see your problems expose themselves over a few days.
 
I use StrategyDesk. My backtesting results are consistant with my forward testing and actual trading. However for me, it takes some work.

My trade strategy is in multiple timeframes (5m, 60m, and Daily).
This presents a cause for gross errors, since StrategyDesk uses closing bar prices in the computation. To overcome this, you can set the backtester to detect intra bar signals, however I couldn't get this to work as precisely, so for backtesting I have rewritten my code in all 5m periods.

I have mathamatically converted MACD, Stochastics, etc. into 5m periods. The programming code size expands quite a bit, however the results are consistant with actual trading. You also need to set the backtester to 'buy the open'.

borland
 
You must log in or register to reply here.
Back
Top