Mathematically Predicting the Future?

[kalzayani]

yes
convient theories

is what they are

Random walk....CAPM...etc
non work
I agree...
it's how it's used or sometimes overlooking
to creat a better model
Modelers sometimes make it complicated
Good modelers do not want to...gets harder that way...
but I think its in the sheer observation
U think if someone had great model...they will post it?...Noway..(ie. Gann)
Trust me my friend you are on the right track to use math
Check what they say about Oil...these days

getting the emotional rise world-wide

recall the Nasda 100 when was 5200 area
Same thing

DJI Index heading to 11,300
in july
politicans and cnbc people wont do their Jobs if they knew were the MKT is going

they Just Tag the excuses for the MKT doing its thing

So short the ^DJI till july
then Wait for cnbc to excuse it in july

 

[kalzayani]

Hey Guys
Doctors recommended Ciggretes in the 1930 as a digesting thing + relaxing method

1960 they recommended qoilodes
over the counter drug to relax
elvies dies

at least math is the one field that uses profs
and not empirical

80 processers in a normal house...computer clocks...etc
maybe Albert had something to with it?
I do not think the FDA likes it

 

[GloriaBrown]

Mathematically Predicting the Future?

Has it Can it be done? As I watch, look and read everything available about trading I have to wonder where did all this junk come from. More than ever I think many people talking about trading using this and that method. Meaning the methods are not purely mechanical, I think we have and are going backwards.

Man in not progressing forward, it seems we are just becoming a Byzantine trading society. We just jewel up a moving average and get some guy on a video saying had you done this and had you done that you would have - could have made this on that trade. Where's the beef? Hey just forget all that stuff tell me where to buy it and where to sell it etc.

Mathematically is it possible, can man define the markets and surround it with a mathematical fence? Pull the market around on a leash like you were walking a dog? Sophisticated software with Ph.D.'s and global knowledge and light speed communication all at our fingertips. Why is it that the perfect trading method does not exist or does it?

Inquiring Minds wanna know, your thoughts. mb

Any update after 13 years?

 

[volpri]

Years ago Futures Magazine did a study on "predicting the market with artificial intelligence".

Their final conclusion... "there is a small degree of predictability for the next bar, only".

NEXT BAR …Reason for scalping LOL. SEE most of the time the market is probing back and forth. Price will go where more transactions take place.

 

[engineering]

IMO...
Fundamentally the markets are driven by psychology, external events and business success/failure. However, there are a lot of people who want to treat them as a random process.
I would suggest that this is dangerous.

Basic statistical calculations are unlikely to tell you that a 1 billion dollar hole is going to appear in a balance sheet out of the blue.

I think that mathematical modeling is potentially useful for spotting odd characteristics worthy of further attention. It might also be useful for spotting the effects of other market participants using similar modeling.

Looking backwards, you can generate a model the fits basically anything, but looking forward is a different story.
If someone believes they have a model that predicts a small segment of the market for a short period of time I would consider that a success, but not count on it lasting long.

How many people are running around saying, "My financial model predicted covid and its effect on the market” ?

On a side note:. Combinations of probability distributions tend towards gaussian. (Central limit theorem). When someone is talking about modeling a big complex system, but somehow not have it be gaussian, that should suggest some extra attention.

 

[Overnight]

This is just a note to myself.

@Overnight 12 year bump.

Carry on.

 

[MarkBrown]

written by mb - edited using gpt



Comprehensive Analysis of the Savitzky-Golay (SG) Filter for Enhancing Stability and Reducing High-Frequency Noise in Trading

1. What is the Savitzky-Golay (SG) Filter?

The Savitzky-Golay (SG) filter is a powerful tool designed to smooth noisy data by fitting a low-degree polynomial to a moving window of data points using least squares. Introduced in 1964 by Savitzky and Golay, this filter is particularly effective at reducing high-frequency noise (often referred to as white noise) while preserving the essential structure and trends within the data.

Key Features of the SG Filter:

• Polynomial Fitting: The SG filter uses a polynomial to fit a series of data points within a moving window. The degree of the polynomial and the window size determine how effectively the filter smooths the data and preserves trends.

• Noise Reduction: The primary function of the SG filter is to reduce high-frequency noise, which can obscure the true underlying trends in financial data.

• Signal Preservation: The SG filter excels at maintaining the shape and features of the data, such as peaks and troughs, ensuring that the essential characteristics of the trend remain intact.

2. Enhancing Stability in Trading Through the SG Filter

The SG filter's ability to smooth out high-frequency noise and stabilize data makes it an invaluable tool in identifying true market trends, particularly in environments plagued by the volatility of high-frequency trading. By filtering out the noise, the SG filter provides traders with a more stable and reliable view of market conditions, allowing for more informed decision-making.

A. Reducing High-Frequency Trading Noise

• Stabilizing Price Data: High-frequency trading (HFT) often introduces significant volatility and noise into market data, making it difficult to discern true trends. The SG filter addresses this by smoothing the price data, filtering out the erratic movements caused by HFT activities.

• Clarity in Trend Analysis: By removing these short-term fluctuations, the SG filter provides a clearer picture of the underlying market trends. This clarity is crucial for traders who need to make decisions based on the long-term direction of the market rather than reacting to every minor price change.

B. Enhancing Trend Identification

• Preserving Essential Trends: The SG filter’s polynomial fitting technique allows it to smooth the data without distorting key features such as peaks, troughs, and trendlines. This ensures that traders can reliably identify and follow trends over time.
• 
  
• Improving Signal Stability: In addition to reducing noise, the SG filter enhances the stability of trading signals. By providing a more consistent view of the market, the SG filter helps prevent false signals that could lead to poor trading decisions.

3. Application of the SG Filter in Real-Time Trading

When integrated with real-time price data, the SG filter not only smooths out noise but also provides a stable foundation for making trading decisions. This combination is particularly useful in environments where the noise from high-frequency trading can otherwise obscure the true market direction.

A. Stabilizing Real-Time Trading Signals

• Consistency in Decision-Making: The SG filter, when applied to real-time data, offers a stable, smoothed view of market movements, allowing traders to make decisions based on consistent, reliable information. This reduces the risk of reacting to noise-induced fluctuations and helps focus on the actual market trend.\

• Filtering False Signals: High-frequency noise often generates false signals that can lead to premature or incorrect trading decisions. The SG filter minimizes these false signals by smoothing the data, making it easier to identify genuine market movements.

B. Enhancing Confidence in Trend-Following Strategies

• Reliable Trend Confirmation: The SG filter’s ability to maintain the integrity of trends over time provides traders with a stable reference point for confirming market direction. This stability is critical when deciding whether to enter, hold, or exit a position.

• Support for Long-Term Strategies: For traders who focus on long-term trends, the SG filter provides the necessary stability to ignore short-term noise and remain focused on the bigger picture. This is particularly useful in volatile markets where frequent, small fluctuations could otherwise lead to overtrading.

4. Benefits of the SG Filter in Trading Stability

By integrating the SG filter into a trading system, traders can achieve greater stability and reduce the impact of high-frequency noise on their decision-making process:

• Enhanced Signal Stability: The SG filter smooths out erratic price movements, providing a stable foundation for identifying trends and making trading decisions. This stability reduces the likelihood of reacting to false signals and improves overall trading performance.

• Reduced Impact of High-Frequency Noise: High-frequency trading often introduces noise that can obscure true market trends. The SG filter effectively reduces this noise, making it easier to focus on the actual direction of the market.

• Improved Decision Confidence: With a more stable and clear view of market trends, traders can make decisions with greater confidence, knowing that their actions are based on reliable, noise-free data.

5. Comparison to Other Smoothing Methods in Terms of Stability

To fully appreciate the SG filter’s role in enhancing stability, it is useful to compare it with other smoothing techniques commonly used in trading:

A. Hull Moving Average (HMA)

• Responsiveness vs. Stability: The HMA is designed to reduce lag and respond quickly to price changes, which can sometimes result in a less stable signal. The SG filter, while slightly less responsive, offers greater stability by more effectively smoothing out high-frequency noise.

• Use in Trend Following: For traders who prioritize stability over immediate responsiveness, the SG filter is often more suitable than the HMA, especially in markets prone to frequent, small fluctuations.

B. Adaptive Moving Average (AMA)

• Dynamic Adaptation: The AMA adjusts its smoothing based on market volatility, which can be beneficial in dynamic environments but may also introduce instability in the signal. The SG filter provides a more stable, consistent smoothing effect, making it a better choice when the goal is to reduce the impact of noise.
• 
  
• Use in Stable Trading Strategies: The SG filter is ideal for strategies that require a consistent, stable signal to track long-term trends, whereas the AMA is more suited to environments where rapid adaptability is needed.

C. Tillson T3 Moving Average (T3)

• Smoothness with Lag Considerations: The T3 provides a smooth curve with minimal lag but may not handle high-frequency noise as effectively as the SG filter. The SG filter offers superior stability by maintaining the integrity of the trend while reducing noise.

• Use in High-Frequency Noise Environments: The SG filter is particularly advantageous in environments with significant high-frequency noise, offering a more stable signal than the T3.

D. Exponential Moving Average (EMA)

• Noise Sensitivity: The EMA is highly responsive but can be sensitive to noise, making it less stable in volatile markets. The SG filter, by comparison, smooths out these fluctuations, providing a more stable and reliable signal.

• Use in Noise-Heavy Markets: In markets where high-frequency noise is prevalent, the SG filter outperforms the EMA in terms of stability, making it a better choice for traders looking to avoid reacting to short-term price spikes.

E. Jurik Moving Average (JMA)

• Advanced Smoothing: The JMA is known for its advanced smoothing capabilities, which allow it to reduce lag while maintaining responsiveness. However, it is a complex and proprietary indicator that requires specific parameter tuning to achieve optimal results.

• Comparison with SG Filter: While the JMA offers both smoothness and responsiveness, the SG filter excels in maintaining trend stability by effectively reducing high-frequency noise. The JMA is more adaptive and responsive, making it useful in fast-moving markets, but the SG filter provides a more stable baseline that can help traders avoid overreacting to noise.

• Use in Trading Systems: The JMA is particularly effective in systems where quick adjustments are needed, while the SG filter is better for traders focused on reducing noise and maintaining a stable, clear view of the market’s underlying trends.

Conclusion

The Savitzky-Golay filter is a powerful tool for enhancing stability in trading by effectively reducing high-frequency noise and clarifying underlying market trends. When combined with real-time price data pattern's, the SG filter provides a consistent, reliable signal that helps traders focus on true market movements rather than reacting to the noise generated by high-frequency trading.

This stability is crucial for trend-following strategies and long-term decision-making, making the SG filter an invaluable component in a trader’s toolkit. By providing a clearer, noise-free view of the market, the SG filter enables traders to make more informed and confident decisions, ultimately improving their trading performance in volatile environments.

 

[MarkBrown]

written by mb - edited using gpt



Comprehensive Analysis of the Savitzky-Golay (SG) Filter Enhanced with the Momentum Summation (MS) Filter for Trading Stability

Understanding the Role of Kernel Functions in Savitzky-Golay (SG) and Modified Sinc (MS) Filters

The Savitzky-Golay (SG) filter is a widely-used method for smoothing data, particularly valuable in reducing high-frequency noise while preserving the underlying signal structure in financial trading. The effectiveness of the SG filter is rooted in its kernel functions, which are based on orthonormal polynomials generated through processes like the modified Gram-Schmidt orthogonalization. These polynomials create a set of n+1 orthonormal polynomials that fulfill certain orthogonality conditions, which are crucial for the accurate smoothing of data.

Kernel Functions in SG Filters
The SG filter’s kernel functions are constructed by fitting a polynomial to a moving window of data points. The key advantage of these kernels is their ability to smooth data without distorting the underlying trend, which is essential for maintaining the integrity of the data, especially in noisy environments.

Key Aspects:

• Polynomial Basis: The kernels are based on polynomials that are orthogonal under a weighted inner product. This allows the SG filter to minimize the error in polynomial fitting over the data window.
• Pre-calculated Coefficients: A unique and powerful feature of the SG filter is its reliance on pre-calculated coefficients. These coefficients, determined by the polynomial order and window size, remain constant regardless of future data. This pre-calculation enables the SG filter to perform smoothing operations quickly and efficiently, making it highly stable and reliable for real-time applications. By not relying on future data points, the SG filter remains robust against future market fluctuations.
• High-Frequency Noise Reduction: The SG filter excels in eliminating high-frequency noise (white noise) while preserving the essential features of the data, such as peaks and troughs. This capability is particularly valuable in noisy environments, where maintaining the accuracy of the underlying trend is critical.

Enhancements with the Momentum Summation (MS) Filter

The Momentum Summation (MS) filter represents a significant enhancement to the Savitzky-Golay filter, particularly in improving trend identification and stability. The MS filter works by accumulating data points over a specific window, which emphasizes momentum and highlights significant changes in market conditions. Unlike traditional moving averages, the MS filter focuses on summation, making it a robust complement to the noise-reducing capabilities of the SG filter.

1. What the Momentum Summation (MS) Filter Does

• Summation of Momentum: The MS filter calculates a cumulative measure of momentum over a defined window of data points. This approach emphasizes the impact of significant changes in momentum, making it easier to detect emerging trends in the market.
• Amplification of Trends: By focusing on the summation of momentum, the MS filter enhances the visibility of significant market movements. This provides traders with an early indication of trend strength or weakness, offering a more dynamic view of market activity.

2. How the MS Filter Improves the Savitzky-Golay (SG) Filter

• Enhanced Signal Stability: The SG filter already provides a stable, noise-reduced foundation by smoothing out erratic price movements. The MS filter further builds on this stability by enhancing the detection of momentum changes, which ensures that traders can more reliably identify and confirm trends.
• Improved Trend Confirmation: The MS filter’s ability to highlight momentum shifts helps confirm the trends identified by the SG filter. When the smoothed data from the SG filter aligns with the momentum indicated by the MS filter, traders gain a more reliable confirmation of market direction.
• Reduction of False Signals: The combination of the SG filter’s powerful noise reduction and the MS filter’s momentum detection significantly reduces the likelihood of false signals. This synergy ensures that trading decisions are based on clearer, more reliable data, minimizing the risk of being misled by noise-driven fluctuations.

Conclusion

The Savitzky-Golay (SG) filter is a powerful tool for reducing high-frequency noise and preserving essential market trends, thanks to its use of pre-calculated coefficients. These coefficients enable the SG filter to perform smoothing operations quickly and efficiently, providing stable and reliable performance in real-time applications.

When combined with the Momentum Summation (MS) filter, the SG filter’s capabilities are further enhanced. The MS filter amplifies significant momentum changes, offering an additional layer of trend confirmation that complements the noise reduction provided by the SG filter. Together, these filters offer a balanced approach to trading, where noise is minimized, and true market movements are highlighted.

This combination allows traders to make more informed and confident decisions, ultimately improving trading performance in volatile environments. The integration of the SG and MS filters provides a powerful, stable, and responsive toolset for navigating today’s fast-paced markets.

 

[MarkBrown]

written by mb - edited using gpt



Combining Velocity (Momentum) with the SG Filter

1. Velocity as a Derivative of Momentum:
   • In physics and finance, velocity can be understood as the rate of change of momentum, which in trading terms often relates to the speed at which price is moving. The MS filter already provides a cumulative measure of momentum by summing up the changes over a defined window. By taking the derivative of this momentum, we could derive a velocity signal, which would provide insights into how quickly the momentum is changing.
2. Acceleration as a Second Derivative:
   • Similarly, acceleration would be the second derivative of momentum, indicating how quickly the velocity is changing. This could offer deeper insights into the dynamics of price movement, especially in terms of detecting shifts in market sentiment or the strength of trends.

Potential Benefits of Using Velocity and Acceleration with the SG Filter

1. Improved Responsiveness to Price Changes:
   • Velocity derived from the MS filter could enhance the SG filter’s ability to respond to rapid changes in price action while still benefiting from the smoothing effects that reduce high-frequency trading noise. This might allow traders to follow trends more closely without being misled by short-term volatility.
2. Enhanced Trend Detection:
   • The combination of the SG filter’s smoothing capabilities with velocity and acceleration indicators derived from the MS filter could offer a more nuanced view of market trends. For example, the SG filter could smooth out the noise, while the velocity and acceleration indicators could provide real-time signals about the strength and direction of those trends.
3. Balancing Stability and Reactivity:
   • One of the key challenges in trading is balancing stability (to avoid reacting to noise) with reactivity (to capture significant market moves). By integrating velocity readings derived from the MS filter, traders could achieve a better balance. The SG filter would ensure that the data remains stable and smooth, while the velocity and acceleration indicators would allow for quicker reactions to genuine market shifts.
4. Potential for Fewer False Signals:
   • Since the velocity and acceleration indicators would be derived from a smoothed momentum signal (thanks to the MS filter), they would likely produce fewer false signals than similar indicators derived directly from raw price data. This could lead to more accurate trade entries and exits, improving overall trading performance.

Challenges and Considerations

1. Complexity:
   • Implementing a system that combines SG filtering with velocity and acceleration readings derived from the MS filter would add complexity to the trading algorithm. This could make it more challenging to tune and optimize, particularly in different market conditions.
2. Parameter Sensitivity:
   • The effectiveness of this combined approach would depend on carefully tuning the parameters of both the SG filter and the MS filter. Traders would need to experiment with different settings to find the optimal balance between smoothing and responsiveness.
3. Latency:
   • While the SG filter provides smoothing and the MS filter offers momentum insights, the introduction of velocity and acceleration might introduce some latency in decision-making. Traders would need to assess whether this latency is acceptable given their trading style and objectives.

Conclusion

Integrating velocity and acceleration readings derived from the MS filter into the Savitzky-Golay (SG) filter framework has the potential to create a powerful tool that balances noise reduction with responsiveness to price action. This approach could help traders better follow trends while avoiding the pitfalls of high-frequency noise. However, the success of such a system would depend on careful tuning and an understanding of the trade-offs involved, particularly in terms of complexity and potential latency.

Step 1: Existing Setup with SG and MS Filters
Assuming you already have code that integrates the SG filter with momentum via the MS filter, you’re smoothing the price data and deriving a momentum signal. This momentum signal likely reflects the strength and direction of the current trend.

Step 2: Calculating Acceleration from Momentum
Acceleration, in this context, would be the rate of change of the momentum (which is analogous to the second derivative of price).

Step 3: Integrating Acceleration into the Trading Strategy
With acceleration calculated and smoothed, you can now integrate this into your trading decision-making process. The idea would be to use acceleration to confirm trend strength or potential reversals. For example:

• Positive Acceleration: This might indicate that a trend is gaining strength.
• Negative Acceleration: This could suggest that the trend is weakening or might be about to reverse.

Step 4: Testing and Optimization

• Test the Implementation: Initially, you would want to test this setup on historical data to ensure that the acceleration indicator behaves as expected. Check if it provides meaningful insights into trend strength or reversals.
• Optimization: Based on your observations, you may need to adjust the smoothing parameters or the window length used in calculating acceleration to optimize the balance between responsiveness and noise reduction.

Conclusion

Adding acceleration to your existing SG and MS filter-based system introduces additional complexity, but it could significantly enhance your ability to track and predict trend changes more accurately. This additional layer of analysis would help you distinguish between strong and weak trends, offering more robust decision-making capabilities. Implementing and testing this approach in code will give you a clearer understanding of its benefits and how it can be fine-tuned for your specific trading needs.

SG MS Velocity Acceleration

Mark Brown

 

[TVIS]

First of all great thread name.
Mathematically predicting the future or Structuring the Market with Maths;
I think both mean the same.
Probably yes, we can
It gives buy/sell, takes days and weeks to hit TP; sometimes can hit TP intraday.
It has drawdown levels and max DD level. Possible behavior before TP.
What are you expectations from maths to structure the market except above-mentioned.
These were taken according to above-mentioned.