How to research and verify trading ideas

[talontrading]

Ok... PCLN... err... no. That's been a nasty loss for the system, but not outside of expectations for this system. It tends to be a bit wild -- we have returns over 150% for some of the trades and losses approaching 50% sometimes.

Look at PCLN through the earnings drift lens though.... it's a great illustration of that concept (so far lol).

I have put off answering this other question not because I'm lazy (well... not just because i'm lazy lol), but because it is such an important question. In fact, it's kind of the central question for this thread.

I guess I start here... I know, from my experience and prior research, that markets move in certain ways and these ways are really not complicated. Trading these patterns is made difficult by the amount of random noise that is also in the market, but the underlying patterns fall into a few categories.

one of those categories is pullbacks... markets tend to bounce around a lot and much more rarely drive in a straight line. another concept is mean reversion. when a market is stretched in one direction it tends to bounce back the other way... there is an element of predictability there.

so, for the original concept of this system it was working through a thought process like that... several years after i did the development and started using ideas like this, larry connors and other started publishing ideas like this... i wasn't pleased when that happened and honestly expected there could be negative impact on the expectation of these systems. that has fortunately not been the case.

as for the specific thing i posted, honestly i did that before i got out of bed. i wanted to publish a purely technical idea in this forum, i wanted it to work, and i also didnt want to post an exact system i was using. so i woke up and sitting in bed with my laptop coded those rules (which are approximations of rules i have used for a long time... so it wasnt intense development work) and ran a few tests to be sure they looked good... and then posted to the forum. the whole process took less than 15 minutes, but this is at the end of many years' work on development and actual trading.

does that answer the question? i imagine it is the first part--where the original concept came from--that is more interesting than knowing where i came up with these exact rules.

keep asking questions and i'll do my best to keep answering them.

Quote from thoreau777:

Dear Talon,

I had read this thread with the utmost care, and I have two brief questions. You mentioned you wish to help us with system development, and I really and truly feel you want to help, and I admire that. And I appreciate it. I wondered what made you suggest the parameters of the following:

Buy the close of this bar when:
1. it is above the 50th percentile of the past 200 days' closes.
2. it is below the 10th percentile of the last 15 days' closes.
3. It is the third consecutive close down.

Exit conditions:
1. Sell when today's close is higher than the 50th percentile of the past 5 day's closes.

I mean, why not 60th percentile of the past 200 day's closes
or "below the 8th percentile"... etc. and the second consecutive close down?

Is it just personal experience when you wish to construct a new system idea??

And second, on the original system of the S&P add/drop, wasn't it supposed to be LONG PCLN since it was ADDED to the S&P500, and short the SPY?? That would make your original system idea profitable up to this day.

Could you quickly answer these two questions?

 

[Redneck]

Quote from talontrading:

I guess I start here... I know, from my experience and prior research, that markets move in certain ways and these ways are really not complicated. Trading these patterns is made difficult by the amount of random noise that is also in the market, but the underlying patterns fall into a few categories.

one of those categories is pullbacks... markets tend to bounce around a lot and much more rarely drive in a straight line. another concept is mean reversion. when a market is stretched in one direction it tends to bounce back the other way... there is an element of predictability there.

TT,

Most excellent thread Sir



On another matter

I think we are all governed by fundamental laws (survival, physics, logic, etc.)

And although with concentrated effort we might break those laws for short periods of time – ultimately these laws will prevail


Do you think the market has fundamental laws – which at times become overwhelmed – then ultimately prevail?

Then of course as humans are prone to do – they repeat (maybe not exactly but close enough for government work) their past


Regards Sir

RN

 

[xburbx]

1. Did I read the Larry Connors aspect correctly? You were concerned that by him releasing his info it would be used by too many people and the edge would be gone?

2. I'm personally still a little confused as to why those percentiles and rules were used. Like the other guy mentioned, what made you say 50% instead of 60%?

3. Maybe to clarify the thought process. I was trying to think of a system that works in a sideways stock rather than a trending. My thought process might be way off, but I was looking for some specific candles in the range that lined up as a pattern. Something like 3 in a row that were stopped on either side of the range within X% of one and other. I hesitate to use indicators because i have heard they are death. I have also heard they are just fine if you know what you are doing with them. Any thoughts?

Quote from talontrading:

Ok... PCLN... err... no. That's been a nasty loss for the system, but not outside of expectations for this system. It tends to be a bit wild -- we have returns over 150% for some of the trades and losses approaching 50% sometimes.

Look at PCLN through the earnings drift lens though.... it's a great illustration of that concept (so far lol).

I have put off answering this other question not because I'm lazy (well... not just because i'm lazy lol), but because it is such an important question. In fact, it's kind of the central question for this thread.

I guess I start here... I know, from my experience and prior research, that markets move in certain ways and these ways are really not complicated. Trading these patterns is made difficult by the amount of random noise that is also in the market, but the underlying patterns fall into a few categories.

one of those categories is pullbacks... markets tend to bounce around a lot and much more rarely drive in a straight line. another concept is mean reversion. when a market is stretched in one direction it tends to bounce back the other way... there is an element of predictability there.

so, for the original concept of this system it was working through a thought process like that... several years after i did the development and started using ideas like this, larry connors and other started publishing ideas like this... i wasn't pleased when that happened and honestly expected there could be negative impact on the expectation of these systems. that has fortunately not been the case.

as for the specific thing i posted, honestly i did that before i got out of bed. i wanted to publish a purely technical idea in this forum, i wanted it to work, and i also didnt want to post an exact system i was using. so i woke up and sitting in bed with my laptop coded those rules (which are approximations of rules i have used for a long time... so it wasnt intense development work) and ran a few tests to be sure they looked good... and then posted to the forum. the whole process took less than 15 minutes, but this is at the end of many years' work on development and actual trading.

does that answer the question? i imagine it is the first part--where the original concept came from--that is more interesting than knowing where i came up with these exact rules.

keep asking questions and i'll do my best to keep answering them.

 

[jtrader33]

Thanks for continuing to address questions, Talon. I think what has made this thread so helpful and much better than anything I've previously seen on ET is the connection between broader general concepts and simple examples (as well as examples of what are improper interpretations of the broader concepts).

I certainly need to brush up on nonparametrics, but one thing I've found difficult when applying various statistical tests is the fact that trading results don't always fit neatly into the examples/explanations from the text. The leap from having a theoretical understanding and applying them judiciously in a trading environment concerns me to the point of running elaborate MC sims instead (not ideal). A very simple example from the current discussion would be to assume that I'm conducting a K-S test to compare the distribution of system returns vs. market returns (just an example, no idea if this is one you'd actually conduct) - i get stuck on things like 'what should my actual inputs for comparison should be?':
-individual trade returns for each symbol against daily returns for that symbol? (but my trades could have varying hold times..some 2 days, some 5 days, etc....also, my sample size will likely be much too small on a per symbol basis)
-daily system returns vs aggregate daily returns for all symbols in test portfolio? (seems messy)

Those probably aren't great examples but I hope they illustrate the point. The nice thing about the MC sims is I can have it enter trades randomly but replicate my systems # of trades and holding periods for each - run it a few thousands times and then see how my actual results compare...i at least feel like i'm doing apples to apples comparisons. However, I know this is not ideal and would like to get away from it.

I don't mean to ask to play statistics professor here, but I was hoping you might further clarify the tests you run, the motivation for running them, and what the specific inputs are?

Quote from talontrading:

No... you compare the shape of the distributions from the market's returns and the system's returns. There are a number of good books on non-parametric statistics... even someone with no background can be pretty comfortable with these techniques after a few month's hard study.

 

[talontrading]

1. Yes
2. I just guessed. I know a wide range of parameters would work so I just made these up.
3. Trading in ranges can certainly be profitable, but the risk you run is that when it breaks out it may really go... so you just have to be quick to take your losses. I wouldn't look too much for candle patterns (no edge to most of those btw) but rather would focus on how price acts at the edges of the range. I am guessing this may be hard to automate, but you should be able to do it from a discretionary standpoint.

Indicators... generally no point. You can make a case for using some indicators at some point... I used to teach starting with indicators because I thought it made it easier to learn but have even abandoned that now. People looking at stochastics, etc... is by and large a pretty misguided approach based on my experience. (Your mileage, however, may vary and there are slight edges to some indicators.)

Quote from xburbx:

1. Did I read the Larry Connors aspect correctly? You were concerned that by him releasing his info it would be used by too many people and the edge would be gone?

2. I'm personally still a little confused as to why those percentiles and rules were used. Like the other guy mentioned, what made you say 50% instead of 60%?

3. Maybe to clarify the thought process. I was trying to think of a system that works in a sideways stock rather than a trending. My thought process might be way off, but I was looking for some specific candles in the range that lined up as a pattern. Something like 3 in a row that were stopped on either side of the range within X% of one and other. I hesitate to use indicators because i have heard they are death. I have also heard they are just fine if you know what you are doing with them. Any thoughts?

 

[talontrading]

I'm thinking you may be too caught up in significance tests in general. Both testing structures you outlined re the KS test are reasonable and we do both.... but I am far more interested in the shape and character of the returns than an actual p value. This is why I said there's some art to this in addition to a lot of science.

Quote from bjs211:

Thanks for continuing to address questions, Talon. I think what has made this thread so helpful and much better than anything I've previously seen on ET is the connection between broader general concepts and simple examples (as well as examples of what are improper interpretations of the broader concepts).

I certainly need to brush up on nonparametrics, but one thing I've found difficult when applying various statistical tests is the fact that trading results don't always fit neatly into the examples/explanations from the text. The leap from having a theoretical understanding and applying them judiciously in a trading environment concerns me to the point of running elaborate MC sims instead (not ideal). A very simple example from the current discussion would be to assume that I'm conducting a K-S test to compare the distribution of system returns vs. market returns (just an example, no idea if this is one you'd actually conduct) - i get stuck on things like 'what should my actual inputs for comparison should be?':
-individual trade returns for each symbol against daily returns for that symbol? (but my trades could have varying hold times..some 2 days, some 5 days, etc....also, my sample size will likely be much too small on a per symbol basis)
-daily system returns vs aggregate daily returns for all symbols in test portfolio? (seems messy)

Those probably aren't great examples but I hope they illustrate the point. The nice thing about the MC sims is I can have it enter trades randomly but replicate my systems # of trades and holding periods for each - run it a few thousands times and then see how my actual results compare...i at least feel like i'm doing apples to apples comparisons. However, I know this is not ideal and would like to get away from it.

I don't mean to ask to play statistics professor here, but I was hoping you might further clarify the tests you run, the motivation for running them, and what the specific inputs are?

 

[xburbx]

Talon,

Would you recommend any specific pieces of material by Larry Connors or is it all good? I saw he has a lot of stuff.

 

[talontrading]

I wouldn't recommend anything from him unless you use it as a launching point for your own investigations. Start with the "How Markets Really Work" book... learn some statistics if you don't already have that under your belt... learn to program R or something else like that... and do the work yourself.

Larry has uncovered some fundamental truths about market behavior, but they are mixed with some stuff that isn't so valid so be careful. Just do the work... in a very real sense, the process is the goal.

Quote from xburbx:

Talon,

Would you recommend any specific pieces of material by Larry Connors or is it all good? I saw he has a lot of stuff.

 

[xburbx]

sounds good and thanks. just ordered it. i have no intent on taking any short cuts and not doing the work. i find that what you have offered in this thread is really great info for someone willing to put the work in. it seems to have given a guide as to where the brain should be heading as opposed to just a mindless q and a

 

[thoreau777]

Dear talon,

Thanks for responding about the PCLN trade. However, more than one post addressed this and was confused. Wasn't the trade system meant to ADD a new S&P addition and go LONG? For example, PCLN WAS added to the S&P, so shouldn't we be long? (And thus short the SPY?)

I think I misunderstood the rules of the system, and I would be humbled and much appreciative if you could straighten me on my course.

Godspeed,
Henry