anatomy of a backtest report

Quote from tjrz:



I think the latest version of Metastock (v. 8.0) has portfolio back testing now.
More...

I've used Metastock for years and gave up on it as a system development tool. I looked at the info on the new version and see they still haven't created a really useful testing tool. Better than the gawdawful POS the earlier versions were, but still not nearly as good as TechnifilterPlus, WealthLab, TS, etc.

gs
 
The basic problem I have with backtest results is that none of us trade like a computer. A computer has no emotions--how would your state of mind be after 9 straight losers--would you be able to pull the trigger? the computer can.
 
Along that same line, the confidence required to pull the trigger after 9 straight losers can be gained from having backtested a strategy and reviewed the results thoroughly enough to know that it will work.

I agree with your point however.

:)
 
I realize that some of this has already be mentioned, but here is what I look for before trading a system

1. num trades > 50
2. smooth equity curve. Run regression test of equity curve and look at noise levels
3. average trade > 5 * (commission + slip)
4. average trade > 2 * risk (as defined by your stops)
5. average trade > 0.5% of capital risked
6. % Wins > 45 (preferably > 50)
7. No too many losing trades in a row (e.g 3 to 5 tops)

Hope this is of help
 
Hallo All.

Here are a few thing I would like to add.

@Gordon Gekko
If you see a result in TS or any other software you have to know whther this has been tested WITH a decent money managment rule. Comparing aplles with apples, means to trade 4 S&P contracts when the S&P was at 300 and only one in times when it was at 900. Most important is also parameter stability. I can show you a perfect equity ion the S&P based on two moving averages (value = zero).

@wdbaker
One of the main feature in WLD2 is the REAL portfolio level backtesting. All other windows software only do basket testing. That is NOT portfolio level backtesting. Once you got to the stage where you use this tool then you know what I am talking about.

@ges
I have never used technifilter but I found WLD2 very fast. Most important is that if you change your money managment strategy on the portfolio you dont have to rerun the whole system again. ALso I think you must have presses some alignment buttons in the simulator that are really slowing down the software a lot. I hardly use them since they do not change the result that much. Also the way the code is written can slow down things.

In general, parameter stability, portfolio equity curve, under water quity curve, longest flat period and average profit per trade are the figures I look at. But not for ONE market but for many, related or not!

Volker
 
@ges Thinking about the speed, TechniFilter must be extremly fast I did a quick test of the code below on the 30 Dow Jones stocks. I tested 1000 bars of each stock, it took me in the 'slow' mode 12 seconds. That is using money managment on the portfolio level. The system was doing very good. I post the results here too. The system was published in February 2002, so it has already 9 month out of sample running.


All Trades Long Trades Short Trades Buy && Hold
100.000,00 $ 100.000,00 $ 100.000,00 $ 100.000,00 $
182.990,42 $ 157.736,66 $ 125.253,66 $ 109.897,23 $


var atrprofit, atrstop, nextatrprofit, temp, thisatr, thisrsi, buyprice,
limitprice, exitprice, maxrisk, maxriskprice : float;
var atrrange,srsi,lrsi,rsiperiod,minprice,minadx,adxperiod,
Bar, buybar, p : integer;
var str : string;

{ setup variables }
atrprofit := 200 / 100; { percentage of the ATR as a target}
nextatrprofit := 110 / 100; { target for day 2, calculated from the open }
atrstop := 100 / 100; { percentage of the ATR as a stop }
maxrisk := 15; { never more than this, even on big atr stocks }
atrrange := 21; { need lots of bars to establish TRUE range }
lrsi := 18;
srsi := 82;
rsiperiod := 5; { this seems to suit the short term hold period }
minprice := 0; { play with price levels here }
minadx := 22; { filter out rangebound stocks }
adxperiod := 14;


for Bar := 41 to BarCount() - 1 do
begin

{ let's calc this for later }
thisrsi := RSI(Bar,#close,rsiperiod);
thisatr := ATR(Bar,atrrange);

for p := 0 to PositionCount() - 1 do
begin

if PositionActive(p) then
begin

buybar := PositionEntryBar(p);
buyprice := PositionEntryPrice(p);

if PositionLong( p ) then
begin

{ Use our wide ATR exits on day one }
if ( Bar = buybar) then
begin
limitprice := buyprice + (atrprofit * thisatr);
exitprice := buyprice - (atrstop * thisatr);
maxriskprice := buyprice * (1 - (maxrisk / 100));
if (exitprice < maxriskprice) then exitprice := maxriskprice;

if not SellAtStop( Bar, exitprice, p, 'atrstop') then
if (PriceClose(Bar) > limitprice) then
SellAtClose( Bar, p, 'day1close');
end;

{ Tighten the target, and get out at the close if we don't get it }
if ((Bar - buybar) = 1)
AND (PositionActive(p)) then
begin
limitprice := PriceOpen(Bar) + (nextatrprofit * thisatr);
exitprice := PriceLow(Bar-1) * 0.995;
maxriskprice := buyprice * (1 - (maxrisk / 100));

if (exitprice < maxriskprice) then exitprice := maxriskprice;

if not SellAtStop( Bar, exitprice, p, 'day2stop') then
if not SellAtLimit( Bar, limitprice, p, 'day2limit') then
SellAtClose(Bar,p,'timeout');
end;

end
else
begin

if ( Bar = buybar ) then
begin
buyprice := PositionEntryPrice(p);
limitprice := buyprice - (atrprofit * thisatr);

exitprice := buyprice + (atrstop * thisatr);
maxriskprice := buyprice * (1 + (maxrisk / 100));
if (exitprice > maxriskprice) then exitprice := maxriskprice;

if not CoverAtStop( Bar, exitprice, p, 'atrstop') then
if (PriceClose(Bar) < limitprice) then
CoverAtClose( Bar, p, 'day1close');
end;

if ((Bar - buybar) = 1)
AND (PositionActive(p)) then
begin
limitprice := PriceOpen(Bar) - (nextatrprofit * thisatr);
exitprice := PriceHigh(Bar-1) * 1.005;
maxriskprice := buyprice * (1 + (maxrisk / 100));
if (exitprice > maxriskprice) then exitprice := maxriskprice;

if not CoverAtStop( Bar, exitprice, p, 'day2stop') then
if not CoverAtLimit( Bar, limitprice, p, 'day2limit') then
CoverAtClose(Bar,p,'timeout');
end;
end;
end;
end; { for position loop }

{ entry rules... }

if (ADX(bar,adxperiod) > minadx)
AND (thisrsi < lrsi)
AND (Pricelow(bar) > minprice)
then
begin
BuyAtMarket( Bar+1,'');
{ tell simulator to enter more oversold stocks first }
SetPositionData(LastPosition, (100 - thisrsi));
end;


if ((ADX(bar,adxperiod) > minadx)
AND (thisrsi > srsi)
AND (PriceLow(bar) > minprice))
then
begin
ShortAtMarket( Bar+1, '');
{ tell simulator to enter more overbought stocks first }
SetPositionData(LastPosition, thisrsi);
end;
end;

var INDPane: integer;
INDPane := CreatePane( 50, true, true );
PlotSeries( RSISeries( #Close,rsiperiod ), INDPane, 009, #Thin );
str := 'ATR(' + FloatToStr(atrrange) + ') = ' + FormatFloat('##.##',thisatr);
DrawLabel( str, INDPane );
str := 'RSI(5): ' + FormatFloat('##.##',thisrsi);
DrawLabel( str, INDPane );

Changing the risk to 20% per trade took less then 2 seconds and here is the result:

All Trades Long Trades Short Trades Buy && Hold
100.000,00 $ 100.000,00 $ 100.000,00 $ 100.000,00 $
316.819,00 $ 258.984,98 $ 157.833,91 $ 109.897,23 $


I think this is pretty fast?

Volker
 
Quote from Volker Knapp:



@ges
I have never used technifilter but I found WLD2 very fast. Most important is that if you change your money managment strategy on the portfolio you dont have to rerun the whole system again. ALso I think you must have presses some alignment buttons in the simulator that are really slowing down the software a lot. I hardly use them since they do not change the result that much. Also the way the code is written can slow down things.

Volker
More...

The real speed hit seemed to result from using WL's 'basket' trading ability, picking the 'best' signal from among many with various filters...it got to be agonizingly slow. Same tests in TF were many times faster.

Otherwise, I am very impressed with WL.

gs
 
Quote from Volker Knapp:

@ges Thinking about the speed, TechniFilter must be extremly fast I did a quick test of the code below on the 30 Dow Jones stocks. I tested 1000 bars of each stock, it took me in the 'slow' mode 12 seconds. That is using money managment on the portfolio level. The system was doing very good. I post the results here too. The system was published in February 2002, so it has already 9 month out of sample running.

Volker
More...

It may have been the way I was trying to use WL.

But in TF I am running complicated tests over 2500 stock symbols over a 10 year period and the test takes 6 minutes on my old 700mzh systemax laptop.

This way I can do many iterations.

Still...I do think WL is a great tool.

gs
 
One of the most important to consider when reviewing backtested results is in-sample vs. out-sample results. If a trading system is created based on a data from a certain period of time, and it is also tested on the same time period, the results are not very useful. For the results to be valid, they should be based on out of sample data...ie data that was not used to build the model.

Now the question becomes when the in-sample/out-sample periods should be.
 
You must log in or register to reply here.
Back
Top