Rol's Trading Journal
- Thread starter Rol
- Start date
I have looked at the backtest results and followed the discussion on what I would call the 'averaging problem'. After a lot of work in this area I have concluded that end-of-day tests of this type can be biased and in some cases completely unreliable.
To keep matters simple say I have a MR strategy (SMR) which goes long if the last close is breached (low of the day < prior close). SMR can be tested by doing the following. For every day check whether high > prior close and low < prior close. This means that the price has crossed the target price at least once intraday. For most purposes this is precise enough. So the trade returns for each stock indivually can easily be tested.
Now the number of signals is directly proportional to the change of the market in %. If the market is down 2% usually 95% of all stocks generate a signal. If the market is up 2% usually 5% of the stocks generate a signal. If I backtest this strategy by averaging all signals this will generate outsized returns, because of a lookahead underlying the test. I don't know how many signals (N) occur before the day. If I knew I could allocate 1/N of my capital to each signal and generate the returns, which I get by average(all signals).
The correct way to do this test is to use intraday data and assign capital by occurrence of the signal, e.g. 10% to signal 1 ... 10% to signal 10. then all my capital is allocated, and no slot is free if new lows occur.
To show how biased end-of-day tests can be I have tested a simple strategy similar to SMR. Even if I put 100% of equity in one stock only it generates outsized returns with only a 30% drawdown. This is very unrealistic and most of the profit comes from the implicit lookahead.
It is true that the problem is bigger for a higher number of signals, however, based on my experience I would strongly suggest that you verify your results on intraday data. The best case is that intraday results match EoD results. The worst case scenario would be that the EoD backtest has little resemblence to a real-life performance especially in drawdown periods. As I understand your life results matches your backtest, so a complete lookahead is not in question. However the problem might be more pronounced in drawdowns.
DTN gives you 3 years worth of minute data for 50$/month for all NYSE stocks + the live feed. If you want I can give you python code and some data to get you started.
The test goes like this:
For every stock doTest:
For every minutebar:
checkEntrycondition
If hour == 9 and minute == 30:
do Beginng of Day calculations
If hour == 16 and minute == 0:
do End of Day calculations
For all stocks:
signals = doTest(stock)
makePortfolio( signals) # sort by arrival
My second recommendation would be that you use % position sizing. I can't imagine that the additional benefit of reduced costs due to lot trading outweighs the negative impact due to unlinear scaling. In effect trading 100 shares lots is like assigning a random position size. From my experience position size has a significant impact on returns and sharpe ratio.
Strategies that have something I call alpha factors can be very valuable, that is if the input factors are linear to the alpha of the trade. Say you have a input value (IV) of 1 to 10 and the alpha increases linearly with 1 to 10 from 10bps alpha to 100bps alpha. Then according to Kelly Sizing the position size should be much higher for higher values of IV.
To keep matters simple say I have a MR strategy (SMR) which goes long if the last close is breached (low of the day < prior close). SMR can be tested by doing the following. For every day check whether high > prior close and low < prior close. This means that the price has crossed the target price at least once intraday. For most purposes this is precise enough. So the trade returns for each stock indivually can easily be tested.
Now the number of signals is directly proportional to the change of the market in %. If the market is down 2% usually 95% of all stocks generate a signal. If the market is up 2% usually 5% of the stocks generate a signal. If I backtest this strategy by averaging all signals this will generate outsized returns, because of a lookahead underlying the test. I don't know how many signals (N) occur before the day. If I knew I could allocate 1/N of my capital to each signal and generate the returns, which I get by average(all signals).
The correct way to do this test is to use intraday data and assign capital by occurrence of the signal, e.g. 10% to signal 1 ... 10% to signal 10. then all my capital is allocated, and no slot is free if new lows occur.
To show how biased end-of-day tests can be I have tested a simple strategy similar to SMR. Even if I put 100% of equity in one stock only it generates outsized returns with only a 30% drawdown. This is very unrealistic and most of the profit comes from the implicit lookahead.
It is true that the problem is bigger for a higher number of signals, however, based on my experience I would strongly suggest that you verify your results on intraday data. The best case is that intraday results match EoD results. The worst case scenario would be that the EoD backtest has little resemblence to a real-life performance especially in drawdown periods. As I understand your life results matches your backtest, so a complete lookahead is not in question. However the problem might be more pronounced in drawdowns.
DTN gives you 3 years worth of minute data for 50$/month for all NYSE stocks + the live feed. If you want I can give you python code and some data to get you started.
The test goes like this:
For every stock doTest:
For every minutebar:
checkEntrycondition
If hour == 9 and minute == 30:
do Beginng of Day calculations
If hour == 16 and minute == 0:
do End of Day calculations
For all stocks:
signals = doTest(stock)
makePortfolio( signals) # sort by arrival
My second recommendation would be that you use % position sizing. I can't imagine that the additional benefit of reduced costs due to lot trading outweighs the negative impact due to unlinear scaling. In effect trading 100 shares lots is like assigning a random position size. From my experience position size has a significant impact on returns and sharpe ratio.
Strategies that have something I call alpha factors can be very valuable, that is if the input factors are linear to the alpha of the trade. Say you have a input value (IV) of 1 to 10 and the alpha increases linearly with 1 to 10 from 10bps alpha to 100bps alpha. Then according to Kelly Sizing the position size should be much higher for higher values of IV.
Regarding the two recent posts about my strategy, I am well aware that portfolio back testing skews the results by not looking at intraday data. I don't need go to the trouble and expense to do intraday testing to show me what I already know. The back testing only laid the groundwork for further strategy refinements, which came later when taken live. I wonder what will be said when I am able to take every signal I get.
At first, I was reluctant to display the back test results because I knew they were outrageous. I got feedback I needed to make live adjustments, so it was not in vain. Now I very rarely refer to the paper trading results. I do, however, still like to see the trades it takes each day.
I used to use % position sizing, but to reduce comm. and simplify orders I went per share. Sometimes a slightly larger position size has helped with profits, so I would expect randomness to cancel out any harmful consequences.
Going forward, I will no longer refer to "my strategy." It will be just "me trading" by the seat of my pants. I may even make some "live calls" taking all the credit for the entries myself. If anyone asks, I will just say, "I have an assistant." I was getting tired of typing "my strategy" anyway. Now I can just type "I." Others can talk freely about the shortcomings of my strategy if they desire, but I no longer feel any need to defend it.
I am thinking of creating a poll to vote on whether I will succeed, or I will blow up.
At first, I was reluctant to display the back test results because I knew they were outrageous. I got feedback I needed to make live adjustments, so it was not in vain. Now I very rarely refer to the paper trading results. I do, however, still like to see the trades it takes each day.
I used to use % position sizing, but to reduce comm. and simplify orders I went per share. Sometimes a slightly larger position size has helped with profits, so I would expect randomness to cancel out any harmful consequences.
Going forward, I will no longer refer to "my strategy." It will be just "me trading" by the seat of my pants. I may even make some "live calls" taking all the credit for the entries myself. If anyone asks, I will just say, "I have an assistant." I was getting tired of typing "my strategy" anyway. Now I can just type "I." Others can talk freely about the shortcomings of my strategy if they desire, but I no longer feel any need to defend it.
I am thinking of creating a poll to vote on whether I will succeed, or I will blow up.
The bulk of todayâs profits came from an opening fade in QUAD. I placed a stop to capture a minimum $300 profit which soon got hit, because I feared the volatility and volume spike. Around high noon I placed several trades in oil related names that I will hold overnight. I am up $250 on my 700 share EWI iShares MSCI Italy Index ETF, and would like to continue holding for more profit. I have my eye on SLV iShares Silver Trust. It is still too soon to try and dig myself out of a $90 hole on that one. I got killed with UFPI at the open. I need to pull up the news on that one tonight, probably a downgrade. I have a few financials, but donât expect any miracles out of them as the sector is the poster child of bearishness right now. Current exposure is 82%, which is a nice zone I like to work in.
Code:
Current Holdings:
UFPI ($174.00) -6.82% 25.50 100 $2,550 Univl Forest Products
SLV ($90.00) -2.68% 33.54 100 $3,354 iShares Silver Trust
KBW ($66.00) -3.46% 19.07 100 $1,907 KBW Inc
AIXG ($61.00) -0.62% 32.89 300 $9,868 Aixtron SE ADS
GRMN ($24.70) -0.76% 32.36 100 $3,236 Garmin Ltd
AU ($2.18) -0.05% 40.05 100 $4,005 Anglogold Ashanti Ltd
DWA ($15.00) -0.74% 20.28 100 $2,028 DreamWorks Animation SKG Inc
ATPG ($9.53) -0.65% 14.75 100 $1,475 ATP Oil & Gas
BK ($5.00) -0.20% 24.70 100 $2,470 Bank Of New York
BCS ($1.00) -0.06% 15.40 100 $1,540 Barclays Plc ADS
GEOI $4.00 0.20% 20.06 100 $2,006 GeoResources Inc
EXH $13.89 0.72% 19.38 100 $1,938 Exterran Hldgs Inc
FST $31.00 1.26% 24.62 100 $2,462 Forest Oil Corporation
STRI $34.00 2.45% 13.86 100 $1,386 STR Holdings Inc
PETD $42.00 1.46% 28.71 100 $2,871 Petroleum Development
CWEI $175.00 3.24% 53.98 100 $5,398 Clayton Williams Energy
EWI $250.00 2.19% 16.30 700 $11,412 iShares MSCI Italy Index Fd
Real-time Account Net Worth $72,479.75
Beginning Day Account Net Worth $71,848.77
Real-time Unrealized P/L $101.48
Real-time Realized P/L (Today) $346.09At 14:44:12 New York time I panic sold all of my positions ( Ctrl+F12 for TS users). It was purely impulsive behavior as I observed my NW reach a nice round number of 73K. I knew Mr. Market was aware of how fond I am of nice round numbers, and refused to let Him steal my joy.
Code:
Real-time Account Net Worth $73,077.99
Beginning Day Account Net Worth $72,468.78
Real-time Unrealized P/L $0.00
Real-time Realized P/L (6/28/11) $700.21Hi Rol
Maybe you already addressed this question somewhere in the thread, but I noticed you also trade ETFs from time to time. What % of your trades occur in ETFs and how did you determine which ETFs were suitable for trading, in terms of liquidity, bid-ask spreads, etc? Do you have a short list of 50-100 ETFs, say, and does this include commodity ETFs or bond ETFs?
It would be interesting to seperate the results from regular stock trades and ETF trades. Of course, long regular stock is long the averages, but some ETFs are short the averages in the sense that they were created to gain from stock price declines.
If the results for ETFs and stocks were similar you might be able to reduce overall stock directional bias but including a higher proportion of ETFs that are directionally short?
I would suspect that many ETFs would behave differently to regular stocks...
Is that something you have considered?
Maybe you already addressed this question somewhere in the thread, but I noticed you also trade ETFs from time to time. What % of your trades occur in ETFs and how did you determine which ETFs were suitable for trading, in terms of liquidity, bid-ask spreads, etc? Do you have a short list of 50-100 ETFs, say, and does this include commodity ETFs or bond ETFs?
It would be interesting to seperate the results from regular stock trades and ETF trades. Of course, long regular stock is long the averages, but some ETFs are short the averages in the sense that they were created to gain from stock price declines.
If the results for ETFs and stocks were similar you might be able to reduce overall stock directional bias but including a higher proportion of ETFs that are directionally short?
I would suspect that many ETFs would behave differently to regular stocks...
Is that something you have considered?
Yes, I considered it here:
http://www.elitetrader.com/vb/showthread.php?s=&postid=3199681#post3199681
ETFs have an equal chance of getting my attention as stocks. However, in general I have avoided commodity ETFs for rtm trading due to their strong directional bias. Spreads are too big with low volume ETFs. Bond ETFs are not volatile enough so are a waste of BP. I have had success with country ETFs, but they may not be as safe anymore as countries can go bankrupt too.
I managed a healthy comeback for June, coming back from an 8% DD, and ending the month with a 3.56% return on initial capital of $65,035. I am now 100% in cash. It was a turbulent month to say the least, but I am stronger from it.
I plan to post results for the 2nd quarter and YTD this weekend.
I plan to post results for the 2nd quarter and YTD this weekend.
Code:
[color=green][b]
Total Net Profit $2,314.80
(Per Share) $0.09
Gross Profit $9,963.12
Gross Loss ($7,648.31)
Profit Factor 1.3
Total Number of Trades 208
Percent Profitable 62.50%
Winning Trades 130
Losing Trades 78
Avg. Trade Net Profit $11.13
Avg. Winning Trade $76.64
Avg. Losing Trade ($98.06)
Ratio Avg. Win:Avg. Loss 0.78
Expectancy 0.11
Largest Winning Trade $566.00
Largest Losing Trade ($576.25)
Max. Consecutive Winning Trades 18
Max. Consecutive Losing Trades 7
Total Shares/Contracts Held 24700
Total Commission $479.80
Return on Initial Capital 3.56%
Annual Rate of Return 42.58%
Buy & Hold Return -0.23%
Trading Period 30 Dys
Max. Equity Run-up(Daily) $7,098.33
Date of Max. Equity Run-up 6/29/2011 15:00
Max. Drawdown(Daily)
Value ($5,101.59)
Date 6/8/2011 15:00
as % of Initial Capital 7.84%
Max. Trade Drawdown ($662.00)
[/b][/color]Here is a summary of my trading YTD. I would say the numbers are minimally acceptable. I want to keep the APR over 72%, to double account size each year. If anything, this journal is making me understand I am in this for the long haul. With patience, I can be triumphant. I wish to commit to keeping this journal going until I reach 1 million. At my current rate that would occur sometime in 2015. I still plan to migrate over to IB to ramp up buying power in the 4X range, so that should help speed things along. I know there will be many naysayers along the way, which is to be expected. It never hurts to have a plan.
Code:
[color=green][b]
Initial Capital $47,853
Total Net Profit $21,261.48
(Per Share) $0.14
Gross Profit $53,744.09
Gross Loss ($32,482.61)
Profit Factor 1.65
Total Number of Trades 1269
Percent Profitable 62.96%
Winning Trades 799
Losing Trades 469
Avg. Trade Net Profit $16.75
Avg. Winning Trade $67.26
Avg. Losing Trade ($69.26)
Ratio Avg. Win:Avg. Loss 0.97
Expectancy 0.24
Largest Winning Trade $1,936.09
Largest Losing Trade ($796.50)
Max. Consecutive Winning Trades 29
Max. Consecutive Losing Trades 38
Total Shares/Contracts Held 148539
Total Commission $3,121.34
Return on Initial Capital 44.43%
Annual Rate of Return 73.38%
Buy & Hold Return -1.69%
Return Retracement Ratio 2.53
Trading Period 5 Mths, 29 Dys, 23 Hrs, 59 Mins
Max. Equity Run-up(Daily) $21,628.32
Date of Max. Equity Run-up 6/29/2011 15:00
Max. Drawdown(Daily)
Value ($5,498.87)
Date 6/8/2011 15:00
as % of Initial Capital 11.49%
Max. Trade Drawdown ($1,511.50)
[/b][/color]