Quote from Rocko Bonaparte:
I don't know if I'm veering too much off topic here but neural networks geek me out and I love to retell this story. I'll probably only stop when somebody finally makes fun of me for constantly bringing it up.
The neural network software was neat4j. I think that software worked fine. It was my program that generated the training set from some indicators I had that was the problem. Most particularly, I located the problem to be somewhere between the keyboard and the chair. 
The whole data set was cooked so the network would have given it a thumbs up no matter what. The problem was the mean I used I had applied to the entire time series, not just up to the point that the network would have been aware in time. So say I have six data points [1, 1, 1, 2, 3, 4]. Average the first three and get 1. Average them all and you get 2. I meant to apply, say, a 3 sample rolling mean so that on sample 3 (day 3) I'd only have the first few samples. But I screwed it up.
So the interesting part in all that was the network managed to infer some things from that cooked data. When I fixed the problem, the network suddenly couldn't figure out anything from what I gave it, and rather preferred to saturate to one extreme or another.
I have much of the infrastructure in place to run some new tests but I wanted to spend some time figure out what the heck I'd even want to feed it. Just grapping a sack of indicators and shoving it down its throat was not good enough.
I read this stuff about the network being able to infer hidden variables, and that throwing indicators at it is kind of obscuring it. Consider it instead as a form of compression. I could give it 20 points to deal with but the training time starts to get huge. Would it not be better then to represent those 20 points in less inputs?
That was a real question BTW. The topology could just become just as gross as the original system but I couldn't say until I've tried it.
I'm thinking of that one episode of Dirty Jobs
"Try, try it."
"He go, eh?"
Your story makes a point about how an ATS can be set up for use and also maintain its efficiency.
I be;lieve a lot of people have had to sit down and figure out how to go about creating a system that operates like an experienced trader would.
Another way to put it might be the consideration of how to set up an ideal approach. I once sonsidered looking into using a tool that advertizers use to see how people consider television adds. Used ones are available for about 20k.
Underlying the concept of advertizing is the idea that people will look at the ads in various ways but the better ways are made available by testing observers and what observers do.
I concluded that most people fail because they never "see" the markets. A lot of posts verify that in ET.
In trading, one principle is that different things are important at different times.
To get to this being available requires that two differnt concepts be put in play:
!. Trading from the general to the specific, and
2. Building a system from a foundation and putting building blocks upon the foundation.
1. is analysis and 2. is synthesis. Their combination is how effectiveness and efficiency comes into being.
An example of this is the consideration of people who know how to do something and people who do not know how to do that thing. the ones who know how have the advantage that their memory automatically prings up what is needed when it is needed; these memories also automatically eclude what is unimportant at a given time. People who do not know how to do the thing lack a memory resouse that chimes in in an appropropriate way to let them move forward to the solution.
In this thread we see almost nothing getting figured out. Probably it because of the fact that few have little notion about going about the business of problem solving.
At the end of this tunnel there is light and a resolution. For anyone who has made the trip, they can look back at others as they proceed.
It is OJT mostly where the job is defined.
Curves and curve fitting comes from letting the tail wag the dog as we see. People learn various skills and they have a propensity to apply those skills to any problem that comes their way. Too bad.
Problem solving skills are what are needed to get to an excellent trading approach (manual or ATS). The glaring mismatch between the common trading ATS's and the markets they are apply to comes largely from the fact that the potential problem solvers know little about markets or trading. Read the current thread on Acrary's determination about money making (an included poll shows how the choices are fucked up, for example).
Look for a moment at a market and its action in terms of its variables. Apparently to take the market's offer a trader needs to have his timing down cold. Taking each segment of profits offered is the objective.
As this is seen and realized it is possible to conclude that the market moves from right to left and then left to right relative to what defines its movement.
There is a statement: "Let the trend be your friend". The rhyming apparently is important for keeping this in mind.
The anchor for trading successfully is the correct side of the market as the money extraction tool (price change) is considered.
The beauty of supply and demand is who is in charge; we know it is the minority. And that determines the correct side of the market.
The mathematics that makes up the ATS's that use this principle is simple just because it is processed by computers.
Russians give their children a toy that has wooden dolls inside of wooden dolls. How did this toy get created and become so successful and entertaining in the minds of those who associate with it?
The whole market involves all those who participate. Each approach is identifiable and they are comingled just as are the wooden dolls. Few people would consider taking the market apart from the collective data; these few do do that for their reasons. Others examine inefficiencies.
Acrary suggests making money to an extent that 200% accumulates a year. Daily, this is a small amount to compound and it is certainly available as detrmined by the market's offer.
The problem is timing the aggregate of the activity. The solution is taking the dolls apart and as you do, dicovering that they all look alike and are in relation to one another.
The two major observable pieces are volume pace and price volatility. This was handed to talon on a silver plater recently. When crossed to expose the tendencies, the distributions are heart warming. The problem solver's conclusion is that markets are orderly.
Supply and demand is explained by one principle; the degree of agreement of the participants. Obviously volume pace and price volatility explain the character of AGREEMENT or it ABSENCE.
Each fractal exhibits the ebb and flow of the character of the participants. fractal to fractal, all is interlocking in a specific ratio (3:1).
As agreement ensues, the market moves from right to left; when agreement wanes the market moves from left to right.
This is true for either market sentiment. when sentiment is changing, then there is an overlap of the two degrees (differing bases) of agreement; gradually one fades into the other until a relative minimum is reached.
Putting all of the above into an ATS for any market all of RTH's given sufficient liquidity, works out as a solution that involves what I introduced in the beginning. Long ago I encapsulated it in a chart of nine tables. At any time there needs to be a proper resolution for what is going on. In jerry030 terminology about 70 items are invloved. Rdolution comes by automating what to pick at any time. 6 or 7 degrees are needed to have the focus handled at any time.
The market does either of two things at any time: continue or change. these are orthogonal as is well known. ATS's cannot deal in opposites in this regard. But the ATS does deal in opposites "inside" the dolls using the 6 or 7 "picked" degrees of resolution. As daytraderbill explained Boolean is how opposites are defined.
In trading volume leads price and A/D leads volume. A/D means accumulation/distribution (See supply and demand). Their cyclic frequencies are 1:2:4.
