What is the correct way to think analyse this?

A

abattia

I am investigating a systematic strategy (stock swing trades).

Over 10 years, the most the strategy trades is 100 times for any given NASDAQ 100 stock, and for some NASDAQ 100 stocks it trades as little as 25 times (over 10 years). With some NASDAQ 100 stocks, the strategy backtests as profitable over 10 years, over others it does not.

If I were analyzing a strategy’s performance over a single instrument, I’m comfortable about how I would do it. In this instance, I would simply say that 100 trades are not enough (in my opinion) for statistically significant testing of the edge, and I would go no further.

However, over all NASDAQ 100 stocks the strategy trades 9000 times (a statistically significant number) in 10-years, and netting all wins against all losses, the strategy was profitable over NASDAQ 100 stocks.

How do I proceed with analyzing the performance and profile of this strategy traded across all NASDAQ 100 stocks?

Do I just proceed the same way as I would for a single instrument (i.e. determine total cumulative return, total Profit Factor, total Sharpe Ratio, total % winners, total average trade, etc for all trades, regardless of the instrument)?

Is this the right way to do it?
 
Quote from abattia:

I am investigating a systematic strategy (stock swing trades).

However, over all NASDAQ 100 stocks the strategy trades 9000 times (a statistically significant number) in 10-years, and netting all wins against all losses, the strategy was profitable over NASDAQ 100 stocks.

How do I proceed with analyzing the performance and profile of this strategy traded across all NASDAQ 100 stocks?

Do I just proceed the same way as I would for a single instrument (i.e. determine total cumulative return, total Profit Factor, total Sharpe Ratio, total % winners, total average trade, etc for all trades, regardless of the instrument)?

Is this the right way to do it?
More...


How many variables go into determining the buy/sell signal(s)? For many variables, 9000 trades may not be enough.

IMHO averaging 900/trades a year is enough, the per-stock trades/year is less relevant. Some things to consider:

  • If not already part of the signal, consider trend of the Nasdaq 100, trends of the S&P 500 and Dow, and trend of the sector the stock is in.
  • If not already part of the system, try weighting the trades based on variance/covariance analysis.
  • Has the system been tested out-of-sample? In other words, backtest for 10 years looking back, test for 1 year blind.
  • Figure out return, profit factor, Sharpe ratio, etc. per year. If the system does not outperform the index for all 10 years taken individually, it might not be worth trading. What I'm saying is, the Nasdaq 100 becomes your benchmark to beat, consistently, every year.
 
Do I just proceed the same way as I would for a single instrument (i.e. determine total cumulative return, total Profit Factor, total Sharpe Ratio, total % winners, total average trade, etc for all trades, regardless of the instrument)?

Yes, and if you like what you see, start forward testing with real money.

Quote from abattia:

I am investigating a systematic strategy (stock swing trades).

Over 10 years, the most the strategy trades is 100 times for any given NASDAQ 100 stock, and for some NASDAQ 100 stocks it trades as little as 25 times (over 10 years). With some NASDAQ 100 stocks, the strategy backtests as profitable over 10 years, over others it does not.

If I were analyzing a strategy’s performance over a single instrument, I’m comfortable about how I would do it. In this instance, I would simply say that 100 trades are not enough (in my opinion) for statistically significant testing of the edge, and I would go no further.

However, over all NASDAQ 100 stocks the strategy trades 9000 times (a statistically significant number) in 10-years, and netting all wins against all losses, the strategy was profitable over NASDAQ 100 stocks.

How do I proceed with analyzing the performance and profile of this strategy traded across all NASDAQ 100 stocks?

Do I just proceed the same way as I would for a single instrument (i.e. determine total cumulative return, total Profit Factor, total Sharpe Ratio, total % winners, total average trade, etc for all trades, regardless of the instrument)?

Is this the right way to do it?
More...
 
Thanks for the excellent response Stoxtrader
Quote from Stoxtrader:
How many variables go into determining the buy/sell signal(s)? For many variables, 9000 trades may not be enough.
More...
  • There are 2 variables that could be optimized (compared to many other systems I have wroked with, it's rules are very simple ... they easily pass the back-of-an-envelope test). It's a third party's system that I read about, and am testing for myself ... I just left the parameters as recommended, haven't tried optimizing yet...
Quote from Stoxtrader:
Has the system been tested out-of-sample? In other words, backtest for 10 years looking back, test for 1 year blind..
More...
  • No, I have just backtested over ten years (haven't tried to optimize yet ...)
Quote from Stoxtrader:
Figure out return, profit factor, Sharpe ratio, etc. per year. If the system does not outperform the index for all 10 years taken individually, it might not be worth trading. What I'm saying is, the Nasdaq 100 becomes your benchmark to beat, consistently, every year.
More...
  • Great suggestions! Thanks!
 
I would think about testing it as a portfolio. You would like to know if your signals are clustered in time or evenly distributed, and also how the signal performance varies over time. Also if you want to be statistically anal, your tests are not independent when treat all the individual trades as a big sample and ignore whether they happened at the same time or not, and running your test on the full portfolio is one way to handle this.

I would also consider testing it by sub-portfolios - does it work in stocks of all market cap, trading volume, industry groups, etc?
 
Quote from black diamond:

I would think about testing it as a portfolio. You would like to know if your signals are clustered in time or evenly distributed, and also how the signal performance varies over time. Also if you want to be statistically anal, your tests are not independent when treat all the individual trades as a big sample and ignore whether they happened at the same time or not, and running your test on the full portfolio is one way to handle this.

I would also consider testing it by sub-portfolios - does it work in stocks of all market cap, trading volume, industry groups, etc?
More...

Great suggestions, thanks.
 
Also, please keep in mind that the index composition has changed over the years and comparing performance to its returns by trading current composition may not reflect realistic conditions.

I think the greatest challenge is the allocation. It makes no sense to allocate the same amount of money to stocks with different alpha and betas.

I have faced all these problems before and I never found a good solution neither for backtesting, nor for allocation. For the backtesting I chose just a subgroup of stocks that mostly replicated the index but were always in it for the testing period and for allocation I used ATR.
 
Quote from intradaybill:
Also, please keep in mind that the index composition has changed over the years and comparing performance to its returns by trading current composition may not reflect realistic conditions.

I think the greatest challenge is the allocation. It makes no sense to allocate the same amount of money to stocks with different alpha and betas.

I have faced all these problems before and I never found a good solution neither for backtesting, nor for allocation. For the backtesting I chose just a subgroup of stocks that mostly replicated the index but were always in it for the testing period and for allocation I used ATR.
More...

Thanks, intradayBill!
... for allocation I used ATR.
More...
How would this work? Is it something along the lines of "if instruments A and B trade on same day, and A has ATR = a, and b has ATR = b, then allocate a/a+b of capital to A, and b/a+b of capital to B?

[... and aren't you in continental US? In which case what are you doing up so early/late? Normally it's just us Europeans, and our Asian buddies, on ET at this time ...]
 
I ‘m wondering whether there’s a way to save myself unnecessary work... but my maths and statistics aren’t good enough to figure this out for myself!

I rely on Sharpe Ratio as a go/no-go metric for further detailed investigation (and eventually trading hopefully!) of any system.

If the range of Sharpe Ratios for the strategy applied to individual NASDAQ 100 stocks (with no more than 100 trades over 10 years in each case) is between X – Y, can I guestimate that the “TOTAL Sharpe Ratio” for the strategy applied across ALL NASDAQ 100 stocks together will ALSO fall in the X – Y range (in which case it won’t reach my threshold)?

Or could TOTAL Sharpe Ratio end up being above the X – Y range?

[... as I write the above I have the sneaking suspicion that the next book I read should be about Portfolio Theory ... :D ]
 
You must log in or register to reply here.
Back
Top