How to research and verify trading ideas

[black diamond]

Quote from talontrading:

The idea is "earnings drift". When a stock reports a large earnings surprise and gaps in one direction (see RIMM and STEC for recent examples) there is a strong tendency to keep moving in the direction of the gap over the next 1-3 months, sometimes longer.

Let's open the floor for discussion. What are the issues with testing this system? How would you set up a testing framework? Where do we start?

Hey Talon,

More good stuff. I never traded this but have backtested it. Here's what I did. This is from memory, if the discussion continues I'll look for the actual code and results. This is based on the academic literature, not my own ideas.

1) Start with a database of all earnings announcements for a long period of time.

2) Make 2 measures of earnings surprise: earnings minus last years' earnings for the same quarter divided by price and 4-day abnormal return at the announcement time.

3) Every month rank all stocks into deciles based on both surprise measures. Form porfolios long top decile (good surprise) stocks and short bottom decile stocks, and hold for 3 months.

4) Tried 3 main versions: 2 using each measure by itself and another using both together.

I think based on either surprise measure the return was about 1.5% per month. Using both together it was noticeably better, maybe 2.5% per month.

So here's a question: Any guesses why this still works? It's been public for a long time, I think the first papers on it were published in the 80's. I don't have any problem believing there is stuff like this out there that works for a while but it is surprising that it can work for so long.

Also I am with you on following the academic research. Everybody who thinks it is all ivory tower EMH stuff is clueless. There are people on wall street whose jobs are to review the academic literature and look for things to trade on.

 

[talontrading]

Trying to spark a little discussion here on this. My idea of starting this thread was to maybe guide and push the discussion in certain directions (and, who knows, maybe I'll screw up and it won't always be the best direction)... I didn't start it so that I could be the Ultimate Dispenser of Wisdom lol... so please feel free to chime in with your ideas. If this is totally off the path you have explored to date, no question is stupid... because this is more or less all I do 24/7 I'm probably just assuming people understand a lot of things here and that's not a good assumption... if you're confused someone else is too so speak up!

I guess the first thing is to step into a more scientific mindset. There are a lot of things in trading that are difficult to quantify (and some that may be unquantifiable), but for this kind of testing we need to have specific, quantifiable questions. That's the first task - clearly define the questions. If you don't ask the right questions, you will end up with a worthless or, even worse, misleading answer.

I deliberately posed the initial system idea in a conversational manner. This is completely unsuited for testing so we must refine it. If a trainee brought me this idea I would highlight the words I bolded below and say "define these... nail these down before we do anything."

So... thoughts?


When a stock reports a large earnings surprise and gaps in one direction there is a strong tendency to keep moving in the direction of the gap.

Quote from talontrading:



The idea is "earnings drift". When a stock reports a large earnings surprise and gaps in one direction (see RIMM and STEC for recent examples) there is a strong tendency to keep moving in the direction of the gap over the next 1-3 months, sometimes longer.

 

[CBuster]

Quote from talontrading:


When a stock reports a large earnings surprise and gaps in one direction there is a strong tendency to keep moving in the direction of the gap.
[/B]

Hi Talon

I think this is an interesting thread so I'll chip in..

Focussing on the first two bolded parts, the obvious questions are:

1) What counts as a "large" earnings surprise and what metric are we using to measure it (i.e. only EPS & Revs or others incl guidance etc)?
2) How do we define a price gap? Will any size gap / move do or is there a required minimum? Relative to the S&P or absolute performance?

I guess some follow-up questions that jump to mind are:

1) Do we care about what looks like a big surprise if the market doesn't react (ie insignificant gaps)?
2) If no, do we even need to bother to analyse the surprise? Will the mere fact there was a gap on earning day be enough for us to conclude that the market was surprised by the announcement?
3) If we DO want to analyse the earnings surprise, how the heck do we quantify the nature of changes in guidance?
4) Another spanner in the works analysing an earnings surprise, even at EPS level (easily quantifiable) is the effect of "whisper numbers". ie the official consensus expectations are for X but the market is actually already expecting Y which is much higher than X. Thus if the actual results are approx equal to Y, is this a surpise? Versus the offical numbers yes, but versus actual expectations no....

I guess we could hammer on in these directions for a long time but it will not get us many results. My inclination would be to start simple and simply look for price moves relative to the market of more that (eg) 5% on prior earnings days and analyse those stocks. Ignore the actual EPS, Revs etc to start with.

Not a complete answer I know but hopefully enough to get the discussion going...

 

[jim c]

Hi. Great thread! I have done some research on PEAD (post earnings announcement drift). Its an interesting idea and there seems to be some merit.
I have an interesting twist on it. Instead of using an earnings announcement I used a volume spike. The largest volume in X number of days. This could usually be linked to an "event" not necessarly earings. Such as a new product, guidance, company scandal, whatever.
So basically if the stock is up > 3% on the day and volume is highest in the last X days go long at the close. I also tested going long the next days open without much of a difference in profits. Do the opposite if the stock is down > 3% with the volume spike.
I tested lots of different parms and they all seemed to work about the same. Seemed rather robust. I tested a holding period of between 1 and 10 days. I got some good profit factors around the 5 to 7 day holding period.
One final twist , and this is what really turned the idea into a great method. If you get a buy signal the stock MUST be LESS than the 200 day ma. Reverse for shorts (stock must be greater than 200 day ma). Yes it seems counter intuitive. It is imo.
Lastly, I got these ideas from a couple of diff web sites and sorrta put them together. I did alot of backtesting and its pretty good imo.
I know I know, "where are the backtested results, otherwise its useless".
Thanks talontrading fro starting the thread. Nice work so far. Jim

 

[mindtrade]

Anyone else short PCLN right now? They were added to the S&P500 last thursday I believe.

They also had earnings today and will gap up tomorrow. The after hours trading was up around 13 points I believe.

It will be interesting to see which is stronger, the addition or PEAD theory Not sure if PCLN would qualify as a large earnings surprise though... Curious to learn what the qualifications for that would be.

 

[talontrading]

I think we have a philosophical divide here. We do everything possible to avoid data mining in any form whatsoever and would NEVER do a test like you're suggesting. In fact, we try to avoid multiple "cuts" through the data as much as possible, even if it's something like should we use 5% or 10% for a parameter.

As much as possible we try to start at concept and then frame it out before ever going to the data. The reason for doing this is frankly I know everyone in the industry data mines and I know everyone's models are not robust. Ours tend to be quite robust.

And yes, we do some pure systematic work here, but we also produce models and find tendencies to be used in a more discretionary manner.

So, please don't take this the wrong way, but you have provided a testing framework that many people would use, but we think it is actually the wrong approach.

Quote from whoisjohngalt:

There's really no point of trying to define what a "surprise" and "gaps" are, at this point... If you want to, all we need to do is set a few parameters and run a brute force to get what "conditions work" like:

Parameter A = {+, -, *, /, %}
Parameter B = {>, <, =, >=, <= }
Parameter C = {1... 100%}
Parameter D = {Boolean, Distribution, Relative, Absolute}
Paremeter E ....Z

In terms of the follow up Qs:

1. Just add another parameter into tests to check if Gap % and it's variations have any significance...

2. Seriously... testing this is more faster and easier. All the thinking is just imagining... you know... Just define the parametric factors and you can start data-mining...

3. + 4. It's all about the data... Actuals are actuals... but expectations are the tricky part. How tricky would the expected value be? I come back again to saying "Data"...

Seriously... developing the model (trading rules/signals) is quite simple for this if you have the prerequisites defined. Some people tend to feel like they need to "make sense" or "rationally accept" something before they run a test but... all that "imagining / thinking / rationalizing" can come later after having the results in your hand...

Talon... the development process you are trying to provide is really what "newbies" do... (Sorry... I'm a systematic trader... maybe you're inclined towards the discretionary side...)

Anyways... what are we testing here? Or is there is there any tests involved in this? (Don't get me wrong... all this is what I do far before the actual models or systems... it's data mining)

 

[talontrading]

this is interesting and something we have looked at too. my sense is that 3% probably isn't nearly big enough. Also, some stocks move 3% every day and some move .5% on average. A simple % change measure misses that so we use a standard deviation move in these cases. Even though market data is most certainly not normally distributed, using a standard deviation measure lets you say what is really an unusual and potential significant move for that stock.

The concept of course is that markets process news and find new prices, and they don't do that in a completely "efficient" way. They overreact at times and also undershoot sometimes.

I have to point out something very important thought because there is a grave danger here. Write system code to buy stocks that go down say 50% in 1 month and let me know what your results are when you apply it to a basket of stocks. I can guarantee that it wont have many trades but it will be a fantastic winner... this will look like a great system, but what's the problem?

Survivor bias... your sample probably doesnt have BSC, LEH, et al in it so you're eliminating a handful of 100% losers and completely skewing the system results. be careful because this can even apply if you're buying say 5% down.

Quote from jim c:

Hi. Great thread! I have done some research on PEAD (post earnings announcement drift). Its an interesting idea and there seems to be some merit.
I have an interesting twist on it. Instead of using an earnings announcement I used a volume spike. The largest volume in X number of days. This could usually be linked to an "event" not necessarly earings. Such as a new product, guidance, company scandal, whatever.
So basically if the stock is up > 3% on the day and volume is highest in the last X days go long at the close. I also tested going long the next days open without much of a difference in profits. Do the opposite if the stock is down > 3% with the volume spike.
I tested lots of different parms and they all seemed to work about the same. Seemed rather robust. I tested a holding period of between 1 and 10 days. I got some good profit factors around the 5 to 7 day holding period.
One final twist , and this is what really turned the idea into a great method. If you get a buy signal the stock MUST be LESS than the 200 day ma. Reverse for shorts (stock must be greater than 200 day ma). Yes it seems counter intuitive. It is imo.
Lastly, I got these ideas from a couple of diff web sites and sorrta put them together. I did alot of backtesting and its pretty good imo.
I know I know, "where are the backtested results, otherwise its useless".
Thanks talontrading fro starting the thread. Nice work so far. Jim

 

[bwolinsky]

Quote from mindtrade:

Anyone else short PCLN right now? They were added to the S&P500 last thursday I believe.

They also had earnings today and will gap up tomorrow. The after hours trading was up around 13 points I believe.

It will be interesting to see which is stronger, the addition or PEAD theory Not sure if PCLN would qualify as a large earnings surprise though... Curious to learn what the qualifications for that would be.

All right, we'll consider this it's first trade then.

Short PCLN at 166, currently at 173.73.

And we should be long the deletion, so which security did Priceline replace?

 

[talontrading]

How cute bwolinsky. I'm thinking you're kind of an idiot though. First of all, you must get your prices from some free source because PCLN is not currently at 173. I wish lol. Stocks trade after hours... you can find information on that on investopedia.com if you're confused. PCLN is more like 188.

However, there's more to this than that. Remember it's a hedged trade. (You said I proved I had no financial experience when I said I didn't measure my performance on a relative basis so you should understand that concept, no?) And also there's another part of the trade you're missing that we made some money on.

Every real trading system has some losers and this may be one of them... or not... hate to disappoint you you haven't "disproved" something that has worked for years with one trade.

we're currently showing a loss of around 8% vs a gain of a little less than 4% on the first part of the trade. Tell me... is that within normal expectation for this trading system or not? You have no clue do you?

At least you're trying... keep it up buddy... always good to hear from you for comic relief.

Quote from bwolinsky:

All right, we'll consider this it's first trade then.

Short PCLN at 166, currently at 173.

 

[bwolinsky]

Quote from talontrading:

How cute bwolinsky. I bet you think this is some great challenge? lol... at the current price the trade has a 5% loss. That's not a disaster, and if you had looked at any of the trades you would have some sense about whether a 5% loss is a big deal or not.

At least you're trying... keep it up buddy.

I edited before you made this comment. You'll be up on the long, so what security did PCLN replace? And, no, 5% is no big deal if it's a small position size. Obviously you're not at 100%, but I'd imagine something on the order of up to 10-15% of equity, so off about a half percent on the portfolio currently. Depending on what the deletion was, you'll probably be flat.