Neural Networks Revisited

[userque]

Yes, but I guess the system uses all historical price data in order to "tune in" to price action. That means if those historical prices become obsolete because of some sort of change in the instrument / time series / market, the system could become "falsely tuned" or lagging to some degree? What happened 10 years ago might never be applicable now or in the future, but what is happening the last 3 months could be key.

It doesn't work like that. It matches recent action to the best match in the past, as determined by internal backtesting. It matches to a specific time-frame in history. That time frame resulted in the best internal backtesting results.

Back to "Machine Learning". I admittedly don't have much experience with state of the art ML, but the concept itself seems to have a bad name. Machines don't really "learn", at least not in the sense humans do, yet. They can take data input, process that and spit it to output, something akin to what humans also seem to. But the processing itself is usually not very "deep" and lack broader understanding of context and awareness (though, this is true for humans as well!). What can be done, is using pattern recognition, statistics, weighting, etc. to search for solutions to questions / problems. It might seem to be "learning", but really, is just a slightly more advanced form of data transformation. One key ingredient to human learning, is to be able to "unlearn", to discard falsifiable knowledge. Probably there are other key ingredients which can provide more intelligent capabilities to machines, or at least tools that can be used to reach more flexible results when facing pure CHAOS .

Well, it doesn't matter to me what they call it; but, my system also can unlearn.

 

[Jerry030]

It doesn't work like that. It matches recent action to the best match in the past, as determined by internal backtesting. It matches to a specific time-frame in history. That time frame resulted in the best internal backtesting results.



Well, it doesn't matter to me what they call it; but, my system also can unlearn.

Yes, but I guess the system uses all historical price data in order to "tune in" to price action. That means if those historical prices become obsolete because of some sort of change in the instrument / time series / market, the system could become "falsely tuned" or lagging to some degree? What happened 10 years ago might never be applicable now or in the future, but what is happening the last 3 months could be key.

Back to "Machine Learning". I admittedly don't have much experience with state of the art ML, but the concept itself seems to have a bad name. Machines don't really "learn", at least not in the sense humans do, yet. They can take data input, process that and spit it to output, something akin to what humans also seem to. But the processing itself is usually not very "deep" and lack broader understanding of context and awareness (though, this is true for humans as well!). What can be done, is using pattern recognition, statistics, weighting, etc. to search for solutions to questions / problems. It might seem to be "learning", but really, is just a slightly more advanced form of data transformation. One key ingredient to human learning, is to be able to "unlearn", to discard falsifiable knowledge. Probably there are other key ingredients which can provide more intelligent capabilities to machines, or at least tools that can be used to reach more flexible results when facing pure CHAOS .

I is important to remember that a human can only effectively make a judgement by considering about 7 things max. Give a person 100 factors and they sit there and sputter all day. I am not sure about a DL but give a SVM 3000 things and it will come back to you after 30 minutes of training with the statistical import of 57 relationships you never dreamed even existed. Hence the term Data Mining.

 

[userque]

I is important to remember that a human can only effectively make a judgement by considering about 7 things max. Give a person 100 factors and they sit there and sputter all day. I am not sure about a DL but give a SVM 3000 things and it will come back to you after 30 minutes of training with the statistical import of 57 relationships you never dreamed even existed. Hence the term Data Mining.

NN's (and I suspect SVM's) rely on correlations to reduce inputs etc. I did an experiment months ago that showed that a test indicator with a .02 correlation, as calculated by the excel correlation function, could be used to forecast whether the next close would be up or down with 100% accuracy.

These non-traditional 'correlations' confound NN's (and I suspect SVM's). My system spots them.

I intentionally made the indicator chaotic.

Can you see how to use the indicator? It's easy! No computer/maths needed! I never got around to plugging it into a NN or the like--but my guess is that they wouldn't be able to 'figure it out.'

Due to the low correlation, NN's and the like would likely prune this input as useless.
....
I found the original image (a little fuzzy):

 

[userque]

I made the original image for someone that didn't believe in a correlation between moon phases and the markets. (My system doesn't use moon phases currently).

I explained to him that my system (then, moon phases were an input) sometimes chose to use them--only phase three, for whatever reason.

He responded that his calculations found no correlation with the phases and price action. I rebutted with the image and challenged him to figure out how to use the indicator to achieve 100% up/down forecast results. He didn't respond further.

 

[Simples]

It doesn't work like that. It matches recent action to the best match in the past, as determined by internal backtesting. It matches to a specific time-frame in history. That time frame resulted in the best internal backtesting results.

Well, it doesn't matter to me what they call it; but, my system also can unlearn.

I know famous investor Larry Williams has people maintaining such systems. Never actually got around thinking much about it. I guess my poor brain just doesn't see too many similarities in price action accurately enough, though I suspect a fuzzy searching algorithm with backtest could find opportunities: their probabilities, strengths/risks, kinds of pseudo-statistical significance. No doubt, there are sell-offs and rallies that behave similarly, but not identical.

I am wondering if you are searching through multiple timeframes (intra-daily-weekly-monthly, etc.) to find patterns to use for your trading timeframe, or just the same TF as you're using to trade?

If you employ chaotic indicators/formulas for your system, wouldn't these possibly diverge quickly in the future, since you don't know the real "variables" and "functions" of the market? Traders call it "context", but it could be anything, not just price or volume. So you could mine all kinds of data-relationships and even more chaotic possibilities, but due to your efforts, it could be fool's gold (not saying your system is btw, just curious)?

Have you tried including other instruments in your analysis for one instrument, and would you say this could be a valid / invalid approach?

I'm not adverse to curve fitting, although there should surely be some foundational reason for the fitting curve? Otherwise, you could generate random data and use that as curve fitting. You could also take some semi-unrelated time series, as in your example, see what could work, and try to use that going forward. Although finding significant patterns would probably take some computational power and data.

 

[userque]

I know famous investor Larry Williams has people maintaining such systems. Never actually got around thinking much about it. I guess my poor brain just doesn't see too many similarities in price action accurately enough, though I suspect a fuzzy searching algorithm with backtest could find opportunities: their probabilities, strengths/risks, kinds of pseudo-statistical significance. No doubt, there are sell-offs and rallies that behave similarly, but not identical.

I am wondering if you are searching through multiple timeframes (intra-daily-weekly-monthly, etc.) to find patterns to use for your trading timeframe, or just the same TF as you're using to trade?

For the chart I posted, just daily. I am building one that combines weekly and daily.

If you employ chaotic indicators/formulas for your system,

I don't. If you are referring to the experiment I posted. That chaotic indicator represented a worse case scenario example. I currently only use dates and price action.

If you are referring to when I said the system matches chaos, that's on the output side of things, not the input...

wouldn't these possibly diverge quickly in the future, since you don't know the real "variables" and "functions" of the market? Traders call it "context", but it could be anything, not just price or volume. So you could mine all kinds of data-relationships and even more chaotic possibilities, but due to your efforts, it could be fool's gold (not saying your system is btw, just curious)?

...and if you are referring to price action as being chaotic; well, yes. It has chaos as well as order. The order is not obvious. I call it hidden. Another type I call cryptic. For my system to 'decrypt' encrypted order would require more computer time/power.

Have you tried including other instruments in your analysis for one instrument, and would you say this could be a valid / invalid approach?

If you are asking have I mixed data from two instruments and ran it through the system? No. I wouldn't call it valid/invalid. I would call it confusing. I suspect the system would still be profitable, depending upon how the two were merged. But it probably wouldn't be as profitable as using the system as it was designed to be used.

I'm not adverse to curve fitting, although there should surely be some foundational reason for the fitting curve? Otherwise, you could generate random data and use that as curve fitting.

It tries to curve fit to the price action.

You could also take some semi-unrelated time series, as in your example, see what could work, and try to use that going forward. Although finding significant patterns would probably take some computational power and data.

That's how a NN etc. would do it. A kNN would try to find a matching pattern.

A question for you. You've seen the chart. Give an estimate: How long (additionally) would this particular system have to 'work' before you would be convinced it's not possibly 'fools gold?'

 

[maninmoon]

I did an experiment months ago that showed that a test indicator with a .02 correlation, as calculated by the excel correlation function, could be used to forecast whether the next close would be up or down with 100% accuracy.

I am sure I am misunderstanding you when you say 100% accuracy. Can you clarify what you meant please.

 

[userque]

I am sure I am misunderstanding you when you say 100% accuracy. Can you clarify what you meant please.

Sure. I needed to clarify something else too.

I'll rephrase the question. Find how the indicator can be used to forecast whether the following day will close up or down. Start with the day before the most recent date, and work backwards.

You should then begin to see how it works, once you do, your forecasts will be 100% accurate. You will reach a point where you will find you need more data. But the method will be obvious at that point.

 

[lucysparabola]

I made the original image for someone that didn't believe in a correlation between moon phases and the markets. (My system doesn't use moon phases currently).

I explained to him that my system (then, moon phases were an input) sometimes chose to use them--only phase three, for whatever reason.

He responded that his calculations found no correlation with the phases and price action. I rebutted with the image and challenged him to figure out how to use the indicator to achieve 100% up/down forecast results. He didn't respond further.

you should look into binary options via NADEX if you're anywhere North of 70% accurate with up/down predicting. I think my stuff can get into the 90s, 100% is silly. (and completely awesome)

I have one glorious setup that hits nearly 100%, but it's difficult to trade as the move happens so fast that I can't get the fill I need on many of them. combine that with getting maybe 2 signals per 24hr session and you have a sad me. I'm working on streamlining the trade process for these though, independently of the ML project.

 

[userque]

you should look into binary options via NADEX if you're anywhere North of 70% accurate with up/down predicting. I think my stuff can get into the 90s, 100% is silly. (and completely awesome)

I have one glorious setup that hits nearly 100%, but it's difficult to trade as the move happens so fast that I can't get the fill I need on many of them. combine that with getting maybe 2 signals per 24hr session and you have a sad me. I'm working on streamlining the trade process for these though, independently of the ML project.

Please tell me you've posted this before. I'm sure I read this before, if not, it's a weird deja vu.

That image is not "real." The indicator is made up to show how even when an indicator doesn't "correlate," it can still work. I purposely made it work perfectly.

That's good to hear about your results. I eventually may do something with binary options.