local extrema detection

[intradaybill]

Quote from ssrrkk:

What is your favorite way to detect local minima / maxima on noisy timeseries? For example, if you looked at every single bar that is above it's two neighbors, that would create too many peaks. You could smooth the timeseries using an N-minute centered average or you could also use an N-minute window min/max method where you would place a centered window around a bar, and decide if it's a peak within that window or not. For these latter two methods though, you would not be able to detect the peak until N/2 minutes later.

I suppose in discrete math the problem is defined by N-th order centered differences, where you would compare with +1 and -1 minutes, or +2 and -2 minutes, etc. I guess you can define a suitable timeframe for your problem (e.g., 5 minutes, 15 minutes, 30 minutes) and check if a bar is above it's +/- Nth neighbors, (N-1)th neighbors, (N-2)th neighbors, and so on. I suppose it's not a peak if any single one of those neighbors are higher. I'm not sure about this though.

Another method is to use moving average crosses, and keeping track of the min / max in between the cross events. You could do this with a single moving average and use the crosses of the timeseries with the MA or you could use two moving averages, again keeping track of the min / max between crosses. Again these methods will suffer from delays which is probably inevitable, because a peak or trough is not defined until you wait N-minutes later to see that it was really a peak or trough.

Any other ideas?

Extreme value: derivative is zero.

Global extreme: an extreme value larger (smaller) than the other extreme values

Local extreme: an extreme value smaller (larger) then extreme values

The algorithm is relatively easy unless I am missing something here.

 

[ssrrkk]

Quote from intradaybill:

Extreme value: derivative is zero.

Global extreme: an extreme value larger (smaller) than the other extreme values

Local extreme: an extreme value smaller (larger) then extreme values

The algorithm is relatively easy unless I am missing something here.

Of course that is the case for a smooth or continuous function. For a real discrete minute bar time series, that derivative (first order difference) will cross from negative to positive and back very frequently may be even once in a few minutes (it will never or rarely be exactly zero so one needs to track zero crossings, not equality to zero). So the question boils down to how much smoothing I need. If I smooth appropriately then the zero derivative is the simplest conceptually. It's the one we are taught in first year calculus after all.

But there are ways to do it without explicit smoothing -- as I mentioned, you can use a windowed maxima minima detection using a timeframe of interest. Or you could use ma crosses and keep track of the peaks in between.

 

[MAESTRO]

A bit more sophisticated solution than centered MA cross over is using the FIR filters. I had always used them in the past for both noise filtering and sharp edge detection. If you don't mind programming one it will do the trick for you.

Cheers,
MAESTRO

 

[ssrrkk]

Quote from ssrrkk:

Of course that is the case for a smooth or continuous function. For a real discrete minute bar time series, that derivative (first order difference) will cross from negative to positive and back very frequently may be even once in a few minutes (it will never or rarely be exactly zero so one needs to track zero crossings, not equality to zero). So the question boils down to how much smoothing I need. If I smooth appropriately then the zero derivative is the simplest conceptually. It's the one we are taught in first year calculus after all.

But there are ways to do it without explicit smoothing -- as I mentioned, you can use a windowed maxima minima detection using a timeframe of interest. Or you could use ma crosses and keep track of the peaks in between.

By the way this is where discrete math appears deceptively easy until you actually try to implement it. I used to be quite adept at solving differential equations, even partial ones, but I have never really become used to so-called difference equations, i.e., the discrete version of differential equations. These are also known as recurrence relations. It is frequently difficult to come up with a closed form solution to a difference equation. I suppose the analogy is complete with the continuous version so I'm sure it's a matter of sitting down and studying the discrete versions enough.

 

[ssrrkk]

Quote from MAESTRO:

A bit more sophisticated solution than centered MA cross over is using the FIR filters. I had always used them in the past for both noise filtering and sharp edge detection. If you don't mind programming one it will do the trick for you.

Cheers,
MAESTRO

Yes, it's all coming back to me now, FIR filters, impulse responses, z-transforms, group delay, pole-zero plots, ROC... I guess in a sense I am already sort of doing that because a N-minute moving average filter is a simple constant coefficient Nth order FIR filter. I guess the power of the generalization comes in with non-constant coefficients to ensure stability and specific frequency characteristics...

 

[GoldStandard]

zig zag utc indicator might serve your purposes. You specify that you're looking for the highest or lowest bar with at least X (specify x in the settings) lower/higher bars next to it. It does a better job of identifying the extreme points of swings than regular zig zag indicator does. After re-reading your first post, I realize that it's pretty much the same as the algorithm you mention there.

Here's a link to download a ninja version: http://www.bigmiketrading.com/download/ninjatrader-7/indicators/510-download.html?view I haven't used this particular version but it should work the same way.

 

[MAESTRO]

Quote from ssrrkk:

Yes, it's all coming back to me now, FIR filters, impulse responses, z-transforms, group delay, pole-zero plots, ROC... I guess in a sense I am already sort of doing that because a N-minute moving average filter is a simple constant coefficient Nth order FIR filter. I guess the power of the generalization comes in with non-constant coefficients to ensure stability and specific frequency characteristics...

There you go. Actually, once you programed a FIR filter you will be pleasantly surprised by its usefulness for many types of signal processing. I am a big fan of them. Also, if you have MATLAB this might be useful.

http://billauer.co.il/peakdet.html

Cheers,
MAESTRO

 

[ssrrkk]

Quote from GoldStandard:

zig zag utc indicator might serve your purposes. Basicly you specify that you're looking for the highest or lowest bar with at least X (specify x in the settings) lower/higher bars next to it. It does a better job of identifying the extreme points of swings than regular zig zag indicator does.

Here's a link to download a ninja version: http://www.bigmiketrading.com/download/ninjatrader-7/indicators/510-download.html?view I haven't used this particular version but it should work the same way.

Thanks, yes, this is sort of the thing I was looking for. This is pretty much using a windowed min/max detection: place a window (e.g., 15 minutes) around your point of interest, then simply compare it with the min/max of that window. If the point is equal to that max, then you are at a peak, at least within the 15 minute timeframe...

 

[ssrrkk]

Quote from DontMissTheBus:

Don't mind him - he just likes to copy and paste stuff he googles... he's done that in numerous other threads where some math is involved.

On your question, have you tried doing a ma-type of difference between numerous time-scales (5D, 10D, 100D, etc) and then then do an average of the first difference?

This is an interesting idea if I understand it correctly. You would take multiple timescale MAs, then for every pair of consecutive points from the MA look at the difference. If the difference changes direction between two consecutive pairs then it's a peak or trough. Do that for the higher timescale MAs. Average the differences between consecutive pairs. Use the average difference to detect directional changes. I wonder what this will do. Anyway, thanks for the thoughts!

 

[vikana]

FWIW - I've found a 4-period WMA to be one of the best short-term smoothers. I'd say the lag is about 1 bar. I'd do that before I do any peak-detection.

I realize the choice of '4' isn't especially scientific, but it seems to work well. WMAs have good spectra.