anatomy of a backtest report

ges · Dec 19, 2002

Quote from EliteThink:

My guess would be that your system would for the most part still work even with companies going broke. Especially if it was a situation with the entire market oversold, less so if it is company specific. Let me know if you need a couple of years of Enron ascii.

Thanks for the kind offer. You got me thinking and I checked back through some of my old data files and found ENE.

Ouch! It was a huge loser with this system, which now makes me suspect that survivorship is going to be a factor.

gs

EliteThink · Dec 19, 2002

Quote from ges:

Thanks for the kind offer. You got me thinking and I checked back through some of my old data files and found ENE.

Ouch! It was a huge loser with this system, which now makes me suspect that survivorship is going to be a factor.

gs

So much for intuition. What if you only played it when the entire market participated in the move? Perhaps this would filter out the bad trades.

man · Dec 20, 2002

since mid nineties there were about 300 changes within the S&P 500, inclduing mergers, renaming and delisting. within the S&P 1500 there were 1100.
long only systems looking for distressed stocks suffer most from this bias IMO.

peace

ges · Dec 20, 2002

Quote from EliteThink:

So much for intuition. What if you only played it when the entire market participated in the move? Perhaps this would filter out the bad trades.

I've tried a lot of filters for other purposes, but, of course, without the data for symbols that have disappeared, it's going to be hard to draw any conclusions.

This system bought ENE on a limit order 10% below the previous close. Unfortunately, the buy was on the day of the collapse, so the trade loss was 90%!! Can't afford many of those, even if the amount of capital per trade is carefully controlled.

The system never bought WCOM. I'll see if I can dig up more symbols from stocks that crashed and see if the system bought them as they collapsed.

The system does do better when the general market is oversold, but it really cuts down a lot on total trades and the system is already pretty selective.

gs

ges · Dec 20, 2002

Quote from man:

since mid nineties there were about 300 changes within the S&P 500, inclduing mergers, renaming and delisting. within the S&P 1500 there were 1100.
long only systems looking for distressed stocks suffer most from this bias IMO.

peace

I think you're right. Think of all the testing that is being done on incomplete databases. Almost all the data vendors out there supply only currently active symbols. Does anyone know of a data source that has price/vol info for all the symbols that have gone by the way?

gs