Pattern Based Strategy Design

[goodgoing]

Quote from Wide Tailz:

Exits and position sizing are far more important than entry.

In trading, generalizations, are meaningless. Trading system performance is path dependent as DontMissTheBuss (where is this guy by the way?) has plainly and nicely put it once.

As a result, if you are following a strong trend, exit may be more important than entry but if you are doing options, entry is everything.

 

[Wide Tailz]

Quote from frostengine:

I have struggled for years with developing profitable strategies. Many times I have found what I thought was the "holy grail" only to crash back down after further testing/live trading.

I will be building a new strategy based entirely on patterns discovered through a custom neural network.

You may be giving in to your human bias to search for patterns where none exist. You may be suffering from a subconscious compulsion to be right as often as possible, thus spending the majority of your effort on what is least valuable in a trading system (the entry setup).

Why not pick a known good entry and use a robust optimization technique to design a few exits around it, then set your trade size based on how much drawdown you can live with, then forward test it and reoptimize periodically?

Does this sound like too much work? You may not be ready to operate like the casino. The gambler gets some intangible satisfaction in exchange for for the money they lose. Are you more interested in neural networks than hearing your cash register go off?

 

[Wide Tailz]

Quote from goodgoing:

In trading, generalizations, are meaningless. Trading system performance is path dependent as DontMissTheBuss (where is this guy by the way?) has plainly and nicely put it once.

As a result, if you are following a strong trend, exit may be more important than entry but if you are doing options, entry is everything.

I believe it is because options tend to blow up and have so little liquidity to begin with.

I'm open to new information.

 

[Wide Tailz]

Quote from Occam:

I diagree, if your objective is to outperform the market. If you're just churning your account for entertainment purposes or are diversifying as part of a retirement strategy, then maybe it makes sense, but this is a trading forum, so I'm going to assume otherwise.

“What’s nice about investing is you don’t have to swing at pitches. You can watch pitches come in one inch above or one inch below your navel and you don’t have to swing. No umpire is going to call you out. You can wait for the pitch you want.”

Wait for the pitch you want. The better the entry, the less significant the exit. As for sizing, just do something that makes sense -- don't blow up. It's really not that hard, if you've got positive-expected-value trades and don't lever too much (or don't lever at all, if you can reasonably avoid it).

Good logic but it can be improved greatly with empirical testing. I'm not in this game to outperform the market.

 

[schemer]

I would rethink your strategy. Essentially, you found a random indicator that seems to work. Why does it work? In what market conditions does it work? How do you know when it has stopped working?

It seems that you have some programming skills. Be more creative. Look at the market and find something that is unusual. That is where you might have a chance in finding an edge.

 

[frostengine]

I have unfortunately let this thread die down. Will be working on some new patterns soon. If you look at the pattern discussed/released in this thread thus far, it has performed well going forward. It has taken 7 trades, with 6 winners. Expectancy of +$62 per trade.


Also Note:
We have simplified the way the DLL loading process works. Now its much easier to use the patterns discussed in this thread. Go to www.thestrategictrader.com to download.

 

[jcl]

What algorithm are you using for finding patterns? A perceptron or a classification tree?

We've found so far that they both work much better for filtering trades in an existing strategy, than for generating trade signals.

 

[braincell]

Run a statistics test to avoid curve fit.

A statistics test is as follows:
-Take a larger number of inputs, for example 500. Now you're using only RSI 14, RSI 21, etc. Add inputs you think are significant, whatever it may be, like MACD h value for different periods, etc.
-For each of the inputs you need around 10 to 50 million runs. With 500 inputs you should do 5 to 25 billion runs, depending on total input count in your code.
-Rank systems by how they perform out of sample. Discard those that have less than 500 trades in-sample.
-All systems that are ranked should add their total success out of sample (ie NetProfit) to variables summing the significance of the input values. So a system that made NetProfit of 20k out of sample should add a value of 20k to all of the input values that were used in it (for example RSI 14, etc).
-Once you run all of the billions of systems, you will see a picture emerging where only some of the input values really seem to be re-appearing.
-If you have a poor search algo, you may need to do many more tests. If you are creating them purely randomly (no hill climbing of any kind etc) you will need 100s of billions.
-Once you select the top 50 input values from the statistics tests, creating systems should be much easier. Meaning, you should be getting a much higher success rate in OOS results compared to using randomly selected input values. If this is not the case, the statistics test suffered a curve fit and perhaps your search algo needs to be improved.

As you can see, running billions of backtests takes time or hardware or very fast search programs. However, it's the best (if not only) way to improve data mining of any kind, in terms of OOS results.

Also, you shouldn't trust any OOS results if the in-sample doesn't have at least 1000 trades in it (like someone said earlier) and the OOS has the unchanged trade frequency. Without this, you really can't say you've found anything "statistically significant".

 

[ssrrkk]

Quote from braincell:

Run a statistics test to avoid curve fit.

A statistics test is as follows:
-Take a larger number of inputs, for example 500. Now you're using only RSI 14, RSI 21, etc. Add inputs you think are significant, whatever it may be, like MACD h value for different periods, etc.
-For each of the inputs you need around 10 to 50 million runs. With 500 inputs you should do 5 to 25 billion runs, depending on total input count in your code.
-Rank systems by how they perform out of sample. Discard those that have less than 500 trades in-sample.
-All systems that are ranked should add their total success out of sample (ie NetProfit) to variables summing the significance of the input values. So a system that made NetProfit of 20k out of sample should add a value of 20k to all of the input values that were used in it (for example RSI 14, etc).
-Once you run all of the billions of systems, you will see a picture emerging where only some of the input values really seem to be re-appearing.
-If you have a poor search algo, you may need to do many more tests. If you are creating them purely randomly (no hill climbing of any kind etc) you will need 100s of billions.
-Once you select the top 50 input values from the statistics tests, creating systems should be much easier. Meaning, you should be getting a much higher success rate in OOS results compared to using randomly selected input values. If this is not the case, the statistics test suffered a curve fit and perhaps your search algo needs to be improved.

As you can see, running billions of backtests takes time or hardware or very fast search programs. However, it's the best (if not only) way to improve data mining of any kind, in terms of OOS results.

Also, you shouldn't trust any OOS results if the in-sample doesn't have at least 1000 trades in it (like someone said earlier) and the OOS has the unchanged trade frequency. Without this, you really can't say you've found anything "statistically significant".

Alternatively instead of searching for a pattern, you could simply posit an "ansatz solution" based on your hypotheses regarding buyers and sellers, informed or not, timings, market microstructure, constraints on orders due to rules, etc. etc. etc. Then check the stats on that idea. I believe you will have a much better chance of finding a significant pattern that way compared to brute-force automated searching. In other words, use your own biological neural net (aka brain) to find / propose the patterns based on unrefutable market structures and rules, then test it extensively with stats to make sure it's real and exploitable. But again, this is just my opinion. By the way, I think this is how black-body radiation was solved by Planck and the Ising models by Ising and later Onsager in 2D.

 

[braincell]

Quote from ssrrkk:

Then check the stats on that idea. I believe you will have a much better chance of finding a significant pattern that way compared to brute-force automated searching.

Not in my experience. Elements of the hypothesis can often be susceptible to curve fit just like a randomly created system can. The manual method allows a good researcher to test maybe one hypothesis per minute. The automatic method should be able to do 100s of thousands per minute. Often our convictions turn out to be a rephrasing of the random. I at least did my fair share of testing hypotheses, and having read some of previous postings by frosty, i'm sure he did too.

In my experience, finding truly statistically significant results takes resources and some funding because CPU power isn't cheap, nor are fast search algos (time is a cost if you develop them yourself).