Minimizing noise in Trend identification?

ironchef · Nov 9, 2023

murray t turtle said:
%%
GREAT blue screen points=============== or sometime red loss\ ++ good green points. LOL=over all good.
dont like MACD, too far from price .
But that could acTually help\ if its hated by so many LOL.
murraY / T\Turtle,.Peace+ Like Dave ramsey says '' ''Financial Peace''

Why don't you ask @MACD, I am sure he likes it.

What do you mean it is too far from price? Why you think it is too far from price, Mr Turtle?

Thanks.

ironchef · Nov 9, 2023

steve42 said:
That almost sounded like some ancient zen text or something.

See the following paper for a application of the concept of noise robust estimation to the problem of decoupling short and long-term volatility something relevant to the problem at hand completely unlike the b******* crap the OP was selling

Decoupling the short- and long-term behavior of stochastic volatility - arXiv https://arxiv.org/pdf/1610.00332

My math is quite rusty so I will have to go back to my college math text book for some refreshing course.

However, in a nutshell, they are trying to develop a filter that can filter out the short term noise and tease out the "long term memory". In a way, the simplest filter is averaging and that is why 50ma/200ma often works.

I think day trading is slightly different. I want to filter out short term noise but tease out short term intentional moves, i.e., trends. I think their filter will lag, just like ma, and not be very useful?

Any feedback you can give me is greatly appreciated.

steve42 · Nov 10, 2023

ironchef said:
My math is quite rusty so I will have to go back to my college math text book for some refreshing course.

However, in a nutshell, they are trying to develop a filter that can filter out the short term noise and tease out the "long term memory". In a way, the simplest filter is averaging and that is why 50ma/200ma often works.

I think day trading is slightly different. I want to filter out short term noise but tease out short term intentional moves, i.e., trends. I think their filter will lag, just like ma, and not be very useful?

Any feedback you can give me is greatly appreciated.

Stochastic volatility models don't really lag per se but their influence kernel does function as a weighted sum over the past so in that sense the distance between the present and the past impulse (in terms of time In whatever base units we're using) is now affecting the present is called the lag in the random process literature. So the goal is then to calibrate the perimeters of the process in such a way that you can be reasonably assured that it is accurately representing what's going on and then forecast produced by such models because because they would then take into account all the information up to the present moment and the most optimal fashion and in that sense if the perimeters were estimated robustly we would also have error bars on the outfits of our results proportional to the errors on the estimates of our parameters. This is sort of a pseudo non-parametric approach to prediction.

Spelling be damned for typing or whatever this voice dictation is

The gist being that it is taking into account superposition of multiple time scales and not just ratio of two moving averages of different periods which really has no theoretical motivation or justification at all

ironchef · Nov 10, 2023

steve42 said:
Stochastic volatility models don't really lag per se but their influence kernel does function as a weighted sum over the past so in that sense the distance between the present and the past impulse (in terms of time In whatever base units we're using) is now affecting the present is called the lag in the random process literature. So the goal is then to calibrate the perimeters of the process in such a way that you can be reasonably assured that it is accurately representing what's going on and then forecast produced by such models because because they would then take into account all the information up to the present moment and the most optimal fashion and in that sense if the perimeters were estimated robustly we would also have error bars on the outfits of our results proportional to the errors on the estimates of our parameters. This is sort of a pseudo non-parametric approach to prediction.

Spelling be damned for typing or whatever this voice dictation is

The gist being that it is taking into account superposition of multiple time scales and not just ratio of two moving averages of different periods which really has no theoretical motivation or justification at all

Thanks for taking the time to explain.

Question: Is Kalman Filter doing something similar?

Also, I need help: If I day trade and the win rate is 50.5/49.5 R:R 1:1. How big a sample size do I need to confirm that the win rate is statistically significant and I have true positive expectancy?

Thank you

panzerman · Nov 10, 2023

ironchef said:
Question: Is Kalman Filter doing something similar?

Kalman Filter Tutorial

Some say there is no noise in finance, and that it all signal. However, it is just a model.

ironchef · Nov 11, 2023

panzerman said:
Kalman Filter Tutorial

Some say there is no noise in finance, and that it all signal. However, it is just a model.

I actually know how to do Kalman Filter that was why I asked if the paper is similar, if so I can try out the concept using Kalman Filter.

Also same question to you: What is the correct sample size? Someone suggested N = ((Za/2 + Zb)/d)^2. Is that a good approximation, I only need an order of magnitude estimate. It is an important question for my day trading method. If it is, I don't have a system, the positive results cannot rule out the mull hypothesis, i.e., no edge.

Thank you sir

ironchef · Nov 11, 2023

ironchef said:
I actually know how to do Kalman Filter that was why I asked if the paper is similar, if so I can try out the concept.

Also same question to you: What is the correct sample size? Someone suggested N = ((Za/2 + Zb)/d)^2. Is that a good approximation, I only need an order of magnitude estimate. It is an important question for my day trading method. If it is, I don't have a system, the positive results cannot rule out the mull hypothesis, i.e., no edge.

Thank you sir

If sample size is ~ (1/d)^2, I need ~50,000 continuous trades to establish if the method truly has an edge. :banghead:

That would be like @padutrader spending 30 years trading Brooks' style.

Specterx · Nov 11, 2023

faust said:
remarkable how simple directional trading is when fog of time is lifted.

You know that markets can consolidate sideways, right?

Specterx · Nov 11, 2023

ironchef said:
My math is quite rusty so I will have to go back to my college math text book for some refreshing course.

However, in a nutshell, they are trying to develop a filter that can filter out the short term noise and tease out the "long term memory". In a way, the simplest filter is averaging and that is why 50ma/200ma often works.

I think day trading is slightly different. I want to filter out short term noise but tease out short term intentional moves, i.e., trends. I think their filter will lag, just like ma, and not be very useful?

Any feedback you can give me is greatly appreciated.

The premise that there’s an underlying Trend Signal which can be revealed by stripping away the noise is flawed, IMO. Any kind of filter or transformation on OHLC time series data ends up being a tradeoff between accuracy, precision, and timeliness.

Markets are (again IMO) much better approached or viewed using ensemble methods. You can do this effectively with longer term trading using manual point-and-click methods, but intraday probably needs some automation.

SunTrader · Nov 11, 2023

ironchef said:
If sample size is ~ (1/d)^2, I need ~50,000 continuous trades to establish if the method truly has an edge. That would be like @padutrader spending 30 years trading Brooks' style.

A poster from way back in 2003 acrary mentioned a minimum of 100 trades and putting that into 1/(sqrt(# trades)) = a standard error of 10% which would be acceptable.