Neural Network

[nitro]

An outstanding post. I have a gret deal of commentary I could add, but I can't.

nitro

Quote from slacker:

Interesting thread...

I wasted 2 years on Neural Nets. I purchased a lot of software including packages from BioComp, Ward and Braincell.

The best package that I used was SNNS. It was best because it had the best documentation, user interface and examples, runs on Linux and Windows, and was free. The commercial packages have better UIs for newbes but at the end of the day you will probably be using scripts anyway so the UI does not help.

I trained the system daily using many different markets and was looking for an intermarket edge.

Some thoughts from that time:

Coordinating data is a bitch. One data-fart can cost a lot of money and be very hard to find. Often I would not know that a data-fart exists until after I would lose a lot of money and go looking for the cause. There are tools that can help but not much. Unless you have to time to write real data validation routines that can do more than identify when a time series has shifted 1 or more bars or the data is way outside the range of the last 20 bars.... Maybe validating data against multiple sources would work. I was able to get the same or better results using less and less data as I learned more about the problem. Throwing tons of data at a NN and expecting it to sort everything out does not work (IMHO)...

More on data. What is the best way to handle 'market shocks' with a NN? I don't know. Market shocks in historical data messes up your results. If the shock was a scheduled report maybe you can plan to handle that day differently. You could clean your data to remove market shock spikes, but what will your system do when it faces that unscheduled market shock tomorrow???

One way many improve the NN results is to 'train every day'. Fine. The system signals a LONG, the next day the market goes south, the system says BUY MORE, market goes faster south BUY LOTS MORE, you retrain the nets that night and the system says, 'SOLD 3 DAYS ago, SELL MORE!' WTF!!!!

What about 'walk forward testing'. Some vendors claim 'walk forward testing' is not 'curve fitting' because even newbees know curve fitting can be dangerous. Curve fitting can give the user false confidence that their curve fitting results during the sample data period did not destroy their account and then find that even routine moderate volatility days can completely eliminate profit goals reached during testing. The vendors claiming walk forward testing is not curve fitting are being dishonest to the point of committing fraud in my opinion.

The curve fitting and walk forward testing assume that tomorrow there will be a buyer for every seller and a seller for every buyer. Not always the case, 9/11, Oct 1987, and LTCM are good examples.

The NN is a trend following system and needs wider stops than most trend systems because of the noise in the forecast. Even a good system in backtesting will have 5 losing trades in a row. If you are trying to forecast a price 2 days in the future or 6 days in the future you will probably need a 1.2 or a 1.5 * AvgTrueRange of 2 or 6 days, this plus 5 losing trades in a row starts to get expensive. Managing risk is very difficult to do as the nets are always changing. Every time you re-train the system; the system you will use tomorrow is new. Think about that statement, tomorrow's NN is NEW. Your results from the past 3 months have nothing to do with tomorrow's NET retrained last night.

With a state machine or a GeneticAlgo you can track down the specific trigger for an entry or an exit. With a multilayer NN you can look at the layer weights and inputs but it is hard (impossible for anything except the most simple net) to find the trigger that gets into into/out of a trade. Just how much do you trust your net?

One guy on the web, around 2000, started a site and published his real time results for 20 markets. He did great for 6 months, people were subscribing, sending money. Looked great. Then the market turned. All of his NNs were on the wrong side of the market and stayed wrong for about 3 weeks. It was ugly. He held fast to his NN until he reached 50% drawdown and then closed the site. Each day he would publish a 'holding fast WE believe our technology' letter to the web site. As long as the behavior of the market says either volatile or within a range the nets are fine. However, when the markets turns from a trend to a range period or from range to volatile, look out baby!!!! How can you be confident when you have had 4 losing trades in a row and not wonder if you are moving to a market fundamentally different than the one used to train data??? You cannot. IMHO

I started to scale back my NN project and just use NN to replace indicators such as RSI. In effect making the NN a better filter. Not predicting the future as much as reducing noise. This is a good application of NNs. However, there are better filters that will do the same without the overhead such as T3Average and the Jurik Research products. (You can find alternatives to Jurik's stuff on Tradestations forum. The Tradestation stuff is free. However, if you want to play with NNs you probably have a fat software budget so spend away. Jurik's indicators are good and cheap for what they do.)

Can a system be profitable and not re-trained 'often'? The fact that all NN vendors retrain and now have tools to choose from 1 out of many alternative systems is proof that they do not work. "They can demo, but they cannot trade." Big difference.

The good news is that computers are cheaper now and much faster. Databases are much better. And best of all, if you spend 2 years developing a NN you can learn enough about the market to reach a point where you can say, "Hey, I can do this good with a price action or KISS system and enjoy my evenings and weekends!" Only then will you start to focus on money management, position size, and price action and maybe make some money. Maybe.

Lastly, the biggest advocates of NNs have something to sell, or are in denial (halfway into the woods). Don't believe any 'historical testing' of NN as it is fake. I would only believe system results that were gained trading real dollars. It doesn't have to be large size, 100 shares of anything would be fine. Trading micro-pips at oanda is fine but it has to be something real. You won't find any vendor providing it or names of anyone who you can talk to not vested with the software provider. Somebody prove me wrong and post the result here please or where they can be found. I still have all of my work on the cold linux boxes over in the corner. Give me a reason to attach them to a monitor!!!

Sorry for the Hershey length post!!!!

Good luck!

 

[PitchBlack]

Slacker,

A posting of the highest quality that can be found on ET. Congratulations.
What you say is confirming my point of view.


Momoneythansens,

The title of this thread is: What is your experience with Trading using Neural Networks?

My answers are pointed to that question. Maybe NN can achieve many tasks and probably it can analyse better 623 indicators simultaneously. But I would like to ask you: give me a name of a NN that does real-time trades and give me the returns that are generated. I’m almost sure I can beat that system. And that is the essence of the thread: a human brain normally will beat a NN in trading.
My personal opinion is that a human brain is superior for pattern recognition/signal generation in trading systems. You can make the remark: maybe Spike is too stupid to develop a successful NN. Maybe true, that’s why I want returns of successful NN’s. If I can beat them it would mean that it has nothing to do with my stupidity, but that indeed the human brain is superior.

 

[momoneythansens]

Quote from spike500:

Momoneythansens,

The title of this thread is: What is your experience with Trading using Neural Networks?

The title of the thread is irrelevant when you introduce a fallacious argument to back up your point of view. I was merely pointing out the fallacy. This does not preclude the point of view you hold from being being correct.

Furthermore:

- I cannot make neural nets work for trading.
- Therefore no-one can make neural nets work for trading.

This is another logical fallacy.

My answers are pointed to that question.

I think you know that's not true. Your answers were predicated on neural nets needing equal/better intelligence to a human brain in order to be useful for trading systems.

My implied suggestion was that it might be better to leverage the sheer number crunching abilities of a neural net rather than focusing on making a neural net as "intelligent" or "smart" as a human brain.

The comparison between net and human brain in the context of intelligence is meaningless as I have already discussed.

The intent is not to make a sentient being from the neural net! Indeed, if one were to do so, he/she/it would probably decide not to trade.

But I would like to ask you: give me a name of a NN that does real-time trades and give me the returns that are generated. I’m almost sure I can beat that system.

LOL. Well, I don't know of many people that give their neural nets a name. Unless you are referring to the software that is used to build the nets, in which case the question is also meaningless.

If we are going to be pedantic about being on topic (suggest we shouldn't be ) the the title of this thread is: What is your experience with Trading using Neural Networks?

...which makes whether you can beat any system or not (neural net or otherwise) a non sequitur doesn't it?

And that is the essence of the thread: a human brain normally will beat a NN in trading.

That is your interpretation. If you want to make it into a contest between human brain and neural net then that is a different matter.

My personal opinion is that a human brain is superior for pattern recognition/signal generation in trading systems.

Once again, there is a limit to the number of things your eyes can monitor and also a limit to how fast one can react. Make of that what you will.

Given several terrabytes of information, does it seem feasible that a neural net could be more efficient at mining that data for knowledge discovery, relationships, patterns, causality etc. compared to a human sitting down and manually sifting through the data?

There are strengths that can be leveraged which outperform human brains. It has little to do with intelligence.

You can make the remark: maybe Spike is too stupid to develop a successful NN.

I would never make that remark as a) I don't know you and b) that is as flawed an argument as any of those put forward by you. Ad hominen.

Maybe true, that’s why I want returns of successful NN’s. If I can beat them it would mean that it has nothing to do with my stupidity, but that indeed the human brain is superior.

This is bordering on argumentum ad ignoratium. It takes only one black swan to disprove your theory.

Again, I'm not clear why there is a need to compare neural nets with human brains on the basis of stupidity or intelligence. This is a clear cut straw man fallacy.

I'm getting the impression that you aren't a fan of neural nets? Am I close? LOL.

They are a tool. They are not here to compete with human brains but to compliment. This is my humble opinion.

MoMoney.

 

[Pathfinder]

Quote from spike500:

My answers are pointed to that question. Maybe NN can achieve many tasks and probably it can analyse better 623 indicators simultaneously. But I would like to ask you: give me a name of a NN that does real-time trades and give me the returns that are generated. I’m almost sure I can beat that system. And that is the essence of the thread: a human brain normally will beat a NN in trading.

Unfortunately, human being can operate only with 7 variables in conjunctive-disjuctive forms with negations even less. That means if NN can operates with more than 7 variables, then it would definitely bit you if you got same amount of information. That means that NN that is developed only for generation TS should be more efficient than human being.

My personal opinion is that a human brain is superior for pattern recognition/signal generation in trading systems. You can make the remark: maybe Spike is too stupid to develop a successful NN. Maybe true, that’s why I want returns of successful NN’s. If I can beat them it would mean that it has nothing to do with my stupidity, but that indeed the human brain is superior.

May be you are right but amount of studies and software developed to assist human being in trade demonstrate that here is not so easy as you think.

 

[PitchBlack]

Quote from momoneythansens:

Furthermore:

- I cannot make neural nets work for trading.
- Therefore no one can make neural nets work for trading.

This is another logical fallacy.

You see logical fallacies that don’t exist. I clearly said and will quote a previous posting to be sure I say exactly the same I said before:
My personal opinion is that a human brain is superior for pattern recognition/signal generation in trading systems. You can make the remark: maybe Spike is too stupid to develop a successful NN. Maybe true, that’s why I want returns of successful NN’s. If I can beat them it would mean that it has nothing to do with my stupidity, but that indeed the human brain is superior.

So I clearly say: or I’m better than a NN or the NN is better but I’m too stupid to develop a good system. To me this is just the opposite of what you say: what I cannot do no one can do.
I never pretend that things are impossible because I cannot do them. I’m just smart enough to know that there are things that go beyond my intelligence.



[QUOTE
LOL. Well, I don't know of many people that give their neural nets a name. Unless you are referring to the software that is used to build the nets, in which case the question is also meaningless.
[/QUOTE]

People who start playing on words and definitions in a discussion in general do this because of their weak position in the discussion. My mother language isn’t English, I speak two languages fluently, and I can express myself more or less in English and in German. If we want to have a real good laugh we could continue in one of the two languages I master well, you probably wouldn’t even get further than saying yes or no.

I don’t like discussions that are too theoretical and /or too hypothetical.
If two athletes both say they are the best, the only way to find out is to compete and show in reality who is right and who is wrong. In this thread I said that my experience with NN was negative and that my own brain beat it. That’s on topic and a clear statement. I’m curious to see a day trading NN in action. And if we compare the results of that NN with my day trading results it should be beyond discussion that the winner is the winner.

When I read the postings of Bali_Survivor and Slacker, all the memories of my “NN period“ come back in my mind as a movie. I had the same problems and obstacles, and finally the same conclusion.

I quite this discussion now because I don’t see any positive element that can be useful for others the way the discussion goes now.

 

[momoneythansens]

Quote from spike500:

I quite this discussion now because I don’t see any positive element that can be useful for others the way the discussion goes now.

Agree. You appear either to not want to or not be able to form a convincing argument to support your point of view Good luck!

 

[PitchBlack]

Quote from momoneythansens:

Agree. You appear either to not want to or not be able to form a convincing argument to support your point of view Good luck!

The only convincing argument is a comparison of the RETURN of the NN and mine. Because the ultimate goal is making lots of money.

 

[momoneythansens]

LMAO. I thought you quit this discussion! Let's not get into a pointless slanging match and destroy the thread.

At least you are being honest in your last post rather than formulating a specious argument surrounding the number of neurons and synapses required to compete with a human brain.

You don't need to respond but I would ask you to at least consider the black swan problem as it pertains to finding a neural net that can produce a better return than yours.

I don't want to dwell on the straw man fallacy, but how about we don't compare human performance versus neural net performance? Instead compare:

human performance vs human + neural net performance?

i.e. using neural nets as an enhancer or compliment.

I believe this is in keeping with the spirit of the original post. If you disagree, so be it.

Again, you don't need to respond but if you are looking for acknowledgement that a human brain has superior intelligence to a neural net then you have my acknolwedgement on that matter. You won't find any arguments here! Hence the straw man.

It's interesting to note that I have not necessarily disagreed with your point of view, only the tools you have used to back up your point of view.

MoMoney.

Quote from spike500:

The only convincing argument is a comparison of the RETURN of the NN and mine. Because the ultimate goal is making lots of money.

 

[nononsense]

Considering Neural Networks, it first must be pointed out that NN is in itself highly diverse. Not simply a matter of the number of nodes and layers. For a deeper understanding, look at the book by C.M. Bishop and the book by B.D. Ripley.

In developing a trading strategy, one has many mathematical disciplines and tools at his disposal. I use some of these according to my needs, but I can assure you that I never ran into a case where I simply had to throw realtime ticks into a hopper of a single mathematical tool for juicy buy/sell signals to turn up as results.

From my work with NN, I always discarded it as too cumbersome and ill adapted to what I set out to do, given that more elegant approaches could be found giving equal or better results.

Selling a "NN trading strategy package" as an all inclusive approach is in my considered opinion an outright fraud pulled on simpleton believers.

 

[urbansound]

Neural Networks can produce some very interesting and very accurate forecasts, but there are still many problems associated with trying to use them reliably for active trading.

For one, there is no industry standards on the design/application of NN's for a novice to go by, though they all tend to use multiple streams of various input data to ultimately project the next bar of data. By iterating through recursive cycles that build on the "Next Bar" from the prior forecast bars, the network can build a projection of multiple bars out into the future.

One of the biggest problems however, is deciding which input data and at what resolution, will be truly relevant toward building a reliable forecast.

There's also considerable demand to having a true understanding of fairly complex math and statistical probabilities and how you want those probabilities to contribute to an overall parent population forecast, so while it may be easy to find, launch and play with NN's, getting truly usable results takes a huge investment of time and effort to research trial and error concepts.

NN's are very subject to the adage of "Garbage in, Garbage out". As market temperaments go through cycles of different characteristics, which are subject to many outside influences, the long term value of an NN is fairly poor, without considering this. Accordingly the question comes to play as to how much historical data will remain relevant for how long, versus too much historical data causing stratification or saturation killing the sensitivity or reliability of the network output.

Last, NN's seek to be a profit monger's tool which stands against the thought of being a skilled trader and truly working the market from fundamental trading approaches, which if you understand market analysis well enough to build a reliable NN, you wouldn't really need the NN to find reliable profit.

The market functions based upon random events of market makers and herds of losers/winners. Essentially if all traders went to NN methods, the market itself would tend to stratify and fail to provide adequate variation required to fuel NN's; a similar complication in using more orthodox statistical analysis theory, as well.

In the end, it's likely more rewarding to learn how to trade and put one's energy into learning the true essence of market and risk management and build expertise upon individual styles.

Of course, this is a single and unofficial opinion of a programmer with only twenty years in statistical analysis. At only three years learning in trading I'm biased toward the fun of learning to become a trader, not a math genius.


Mike