Survivorship bias

S

Stoxtrader

In backtesting I'm realizing that I may have survivorship bias... stocks more than a few months old that have been either delisted or bought out no longer have data on Yahoo Finance or MSN MoneyCentral.

Solution for backtesting? Are there sites that keep old data longer? (Going forward I can simply keep my own daily/weekly database going.)
 
If you are computing the performance of companies, index funds or mutual funds, or even indices themselves, then survivorhip bias plays a role and works against you but if you are developing trading systems a persistent survivorship bias is what you are looking for.
 
Quote from intradaybill:

If you are computing the performance of companies, index funds or mutual funds, or even indices themselves, then survivorhip bias plays a role and works against you but if you are developing trading systems a persistent survivorship bias is what you are looking for.
More...


Are you saying that you want certain trading strategies to not survive? Sure. My problem is that the strategies screened for companies, and the companies that didn't survive (data no longer available) were mostly large moves (gaps) up or down. Company ABC went from $10 to $0.01 and was delisted. Company XYZ was bought out by a private company for 25% premium above stock price.
 
I believe there are a number of commercial data vendors, but I don't know who to recommend to you.

You are correct to worry about the issue.
 
I said this before, finding old historical data is always expensive.

However...

Survivor-bias for any historical data (stock indexes, etc.) is over-rated. Very, very, very few testing methods require that level of precision.

Anytime you are back-testing in the past, you are using a static model that does not indicate the current or future realities of the market.

Test your strategy on the historicla data you have, if it works, forward test on the current pool of symbols and their price data.
 
Quote from Stoxtrader:

Are you saying that you want certain trading strategies to not survive? Sure. My problem is that the strategies screened for companies, and the companies that didn't survive (data no longer available) were mostly large moves (gaps) up or down. Company ABC went from $10 to $0.01 and was delisted. Company XYZ was bought out by a private company for 25% premium above stock price.
More...

I think you are using the term "survive" in a peculiar sense or betetr to say, in a different sense than I do. Survivorship bias has to do with focussing on the companies/systems that survived screening, not on those that failed it. You are probably talking about selection/sampling bias, which has to do with sample collection. But in your case, if your universe is large enough, the bias is very small or close to zero.
 
Ya I test strategies on all S&P 500 or the Russell 2000.. if you think about it, a good and highly profitable long biased strategy will filter out the shitty stocks that got delisted anyways... so I wouldn't worry too much.

Obviously though, if you have a short strategy, you want the delisted stocks to test against, because those would have made you a lot of money.
 
Any kind of a value (i.e. low p/e) strategy will be very sensitive to survivorship bias. Conversely, it probably doesn't matter much with the short-term momentum strategies like they talk about here. ETers are a bunch of gunslingers.

CSI (http://www.csidata.com/) claims they never delete anything from their database, making it the only service I know of available to individuals. I have used their commodity data but not their stock data.

The Compustat Point-in-Time (PIT) database is the gold standard for the elimination of both survivorship and lookahead biases. It's not generally available to individuals.

PS: Send me a PM if you know how to get access to the PIT database.
 
Quote from psytrade:

Obviously though, if you have a short strategy, you want the delisted stocks to test against, because those would have made you a lot of money.
More...

Woudn't a price limit take care of that? This limit on shorting is also dictated by the exchanges and brokerages.

Also, IMO backtesting screening results for shorts is dubious task because you never know if you were called or even if there was availability of shares to borrow. I think screening is only applicable to long-only strategies.
 
You must log in or register to reply here.
Back
Top