Automated pattern recognition

[TSGannGalt]

Quote from stephencrowley:

Nitro, yes you are right about scale-free templates... I believe if you are going to go the wavelet route you need to formulate them in continuous-time rather than discrete. The problem is how to build the template library, and at what scale? The trick is to design some algorithm that can automatically detect 'new' situations at any, possible nested timescales and know when to add them to its library, update existing templates, etc. Maybe there is no 'when to update' but everything is always adapted continuosly as data flows thru the system.

There are models based on differential equations, where if you tune the parameters correctly the continuous-time system will automatically relax to a fixed-point attractor when a certain input pattern is present and it doesn't need to wait until the signal "arrives", it starts responding as soon as the pattern begins to emerge.

The issue is, there are patterns within patterns, and you are never completely within a single pattern, but any number of them simultaenously and at any number of timescales.

Btw, if you solve this problem generally, you will be a very very rich man because it can be applied to nearly any complex system and not just the markets.

Nice point.

If someone has figured it out, please PM me. I'll overlay Gann Analysis on top of it.

 

[fildi101]

This is an interesting thread for me.

I think it would be possible to do some sort of day based pattern matching.. (i.e. looking at the shape of a whole days trading, and finding similar shapes hitorically)

to do that you'd just need ot export the intraday chart each day in a standard size and format, and then image process it so the main shape is more clearly visible.... then downsize the image file to 100 or so pixels, and run a pattern matching analysis on that.

the idea would then be to see what happened the day after the historical day who's shape matched..

 

[Poncho]

Quote from dcraig:

If you want to experiment with neural nets,

http://www.jooneworld.com

is interesting, with a GUI for constructing nets, with input layers for Excel, SQL, Yahoo finance and others. Free with Java source code including distributed training if you ever get to the point of needing serious computational power.

Very nice link, going quickly through this, there is some nice info here, plus lots of information packed into the joone guide, it is 137 pages long.

 

[SmoothTraderFX]

For MetaTrader4 users.

I created a Harmonic Pattern Custom indicator.

Harmonic Pattern rocks.

USDJPY 1D

USDJPY 30M

5M

Regards,


fxsystemtrader@gmail.com

 

[itmediaco]

Take a look at Recognia:

"Using patent-pending pattern recognition algorithms and neural networking technology, Recognia then systematically analyzes the pricing data for these financial instruments overnight, producing charts for each security and automatically detecting current and historical chart patterns. The technology used by Recognia is proprietary, and has been developed and validated by experts in the field of technical analysis."

http://www.recognia.com/services/patternrecogn_desc.htm

 

[nononsense]

Quote from Poncho:

Hi Ivo,

I have been playing with with creating neural networks (NN) and was successful at creating some simple ones. I grabbed most of my stuff from Generation 5 http://www.generation5.org/content/2005/NeuralArchitecture1.asp although I have gotten several textbooks on the subject as well.

If you dig around and look at the numerous examples posted there, you should be able to come up with a general idea of how they work. I re-created them in C# from the C++ examples. The examples are readily expanded to do what you need I believe. A NN can be trained for pattern recognition. However, the ability of the NN to recognize a pattern is dependant on how good you describe it, the robustness of the rules you use, and the training.

Provided you can come up with these rules for the patterns and defined them, and trained the network, I believe they could be made to work very well. Then set the developed NN loose on some historical data, and see how well it performs. If it performs poorly, tweak the model and re-train. Then evaluate the historical data again, etc, etc.

As far as speed of platform, the speed aspect comes into play only when training the NN, not necessarily the evaluation of the historical data or the live operation of the NN. This is because the trained NN has constants associated with each node that were developed in the training phase (training can take between 10^3 and 10^6 cycles).

I did not develop mine any further regarding pattern recognition as I am using to evaluate the output of different trade logic, something that is, at least in my mind, a better fit. Presently I have set NN aside as I have been working on the IB TWS interface using C#, something that I continue to chew on, w/o success I might add.

I hope this is of some help.

Sound post.
Interestingly, we come right onto Python!
Of course, the url is only meant to be a set of test examples. As the Poncho says, a huge amount of references is available.

As to NN's use for making a bit more money in the markets, that's altogether another subject.

 

[nononsense]

Quote from nitro:


The only problem is that what you really want is a multi-timeframe version that can do it on all time frames at once. For this you need a set of wavelet "templates" or basis, and that requires quite a bit more sophistication on your part to implement.

nitro

You don't want a 'multi-timeframe' version. You want continuously adaptive scales in both directions. This is a well known research topic in data-mining. Wavelets have been tried a couple of times but seem to be of little use.
An inherent problem is the efficient cataloguing of the patterns searched for, this linked with a quick search/retrieval method.

 

[stephencrowley]

You might want to take a look at differential stochastic hopfield networks. I used to think that mulit-timeframe templates would be useful, but it is not straightforward to extend the idea of synchronized spikes to multiple time frames, the good news is that each template can have its own inherent timeframe so all sorts of templates operating on different timescales can be run in parallel.

Quote from nononsense:

You don't want a 'multi-timeframe' version. You want continuously adaptive scales in both directions. This is a well known research topic in data-mining. Wavelets have been tried a couple of times but seem to be of little use.
An inherent problem is the efficient cataloguing of the patterns searched for, this linked with a quick search/retrieval method.