How to Build an Automated Trading System

Status
Not open for further replies.
Quote from greaterreturn:

...This seems to be more a call for perhaps a "decision tree" design.

I'm looking at a few decision support system patterns to figure out what will be the quickest to start using and be easy to extend and improve over time.

We'll see. Any other ideas?

Wayne
More...

I will share some of my thoughts...

Jack speaks often about the binary nature of the algorithm. He also often speaks of how matrices are incorporated. I cannot comment on the extent, dimension, and final incorporation of these concepts but my thinking here is that he is giving hints as to how data points and module outputs outputs might be structured and analyzed to produce as decision.
 
Quote from greaterreturn:

Okay, the Windows Workflow is way over complicated and generated different classes of every state. That will proliferate so many classes that it's unbearable.

I better design I found is the State Machine Compiler. Open source and free.

http://smc.sourceforge.net/

It builds a state machine for one class so it can have the states, receive events, call actions, and then optionally change state.

Now my thought process is to break down the different parts of trading JH methods into individual components.

For example, the bases of everything is using tapes or small bar by bar channels.

I will build one state machine that can handle drawing and taping one bar at a time.

But it won't make any trading decisions.

Instead, I will model work either Jack recommends. For example, this taping component will be called a "housekeeper". Jack calls it that. It's always easier to make changes later if you build software using domain specific terminology.

Later additional housekeepers can be added like MACD monitor, DOM Monitor, etc. Intermediate channels. That's all housekeeping to have the data so the software can "see" the market.

Next come M.A.D.A (monitor, analysis, decision, and action). Jack calls the MAD a sweep for short.

It means the human quickly reviews all the channels, indicators, and information. Makes a quick analysis mentally of what that means. Makes a decision of hold/continue (enter or exit) and then takes action.

In the case of hold, the action is to log the decision. In the case of a reverse or trading decision, you first trade and then log.

So we'll have a monitor component which will proactively and periodically look at the housekeeping components to get their latest stats, at least the high level, pertinent ones for making an analysis.

It will have an internal state machine because what it monitors when varies. As Jack describes, at times your monitor the on coarse grained controls but then switch to monitoring fine grained information like the DOM.

Next comes analysis. That component will take the information complete information from the last sweep by the monitor and analyze it each time to generate a single conclusion or "event to pass to the decision component.

Decision component will take the event information along with its own internal state and decide what action to take and whether to change states.

Each of these have their own state machine and handle changes in events concurrently.

At the high level, these will combine into a "trader robot" that will monitor, analyze, decide, and take action.

The advantage of state machines is that they are designed to deal with the back that events may arise that are unforeseen. So the state machines have the ability to "get confused", handle confusion with certain actions, recognize events and finally reset to a known state to continue.

That's all very critical in trading to avoid the software "crashing" or failing in trading conditions that confuse it.

This system would simply sideline until it can figure out what's going on.

The it can be "debriefed" that is, the logs of any confusion and charts can be analyzed (sorry this requires a human being!!!) to add new states or events to account for it's confusion so it will know what to do the following time.

The beauty of this is that we can build in a few states and conditions. See them work and then iteratively add more and possible discover others.

I think this describes in a nutshell what Jack refers to as a multidimensional matrix of market operating points that the market moves between.

If not, we'll refine it. Any suggestions?

Sincerely,
Wayne
More...

I like the approach you have described.

I have taken the approach that every incoming tick is run through the "modules". MADA is done at the speed of the incoming data. Decision and Action though (for now) is only made at the close of each bar and thus constrained by the chosen fractal.

As witnessed, given my current simplistic and crude model (taken from the point of view of both the trader (me), the system, and the implementation of the algorithm), this tends to show up as making decisions "late" and/or "incorrect" and/or "un-optimized" which would suggest the need for an IF1 safety event as a decision and action overrider. But (IMHO) IF1 is not desired as it masks underlying fundemental flaws in the model and also hints at the lack of maturity and "degrees of freedom" present in the current model and it's developer :D The most recently discussed is the channels module and also the immature implementation of modelling internals and performing ADA with such modules.

So at a high level, the current design could be described as a funnel. where the fluid is the incoming tick data, passing thru the modules, and being filtered via the developers understanding of algorithm, filters, and context, to finally produce a decision.

The code is refined as the developer is refined as the trader is refined :)

The trader is refined through drills! The coach defines the drills, the student performs the drills. The student develops skills, knowledge and experience, and describes these to the developer, and the developer codes them to produce an ATS...

This is my approach :)
 
I read the thread this am.

To help out I'll post quite a few pics that will be helpful.

I'll also cut and paste a few of your remarks to focus them a little from my perspective.

Your initial posts are very informative and the two programming tools you use to convert business activity to ATS operations are right on.

Over the years in ET, several people have come up with some real nuggets. They have otherwise eluded the literature and the academic magnifying glass. I'll tab them up because they are independently scriptable and apply to a good assortment of human, and therefore convertable, trading styles and strategies.

As a caveat, this will not overlap IP and TS stuff which I know you understand has a penalty associated with it (for me).

ehorn points out the value of using PE. And I believe the link in the chain of transfer that most demands attention is the schematic to task defining link. Since you test (in the objective sense) the the QA issue is settled by iterative refinement.

I think it is turning out in ET that the word descriptions are providing a basis for your making schematics.

An overview.

The computer is fast and so all degrees of freedom can be generated and available all the time (70 approximately). This is "knowledge" to me and its attainment is not complex. At this point it is available.

A paradigm shift occurs primarily when we go from analyzing value to analyzing participants. The shift is so far into the scene that statistical significance can be used to determine the price to pay for the effort.

One very important thing to notice, and you did, was that different things are important at different times. If an approach is examined and this factor is not evident then the system's crudeness level could cause concerns about things like money management and rish protection, all unnecessary band aids, it turns out.

Shifting from the human to the ATS is a matter of rep rates. In real time people, possibly, are limited. I am on automatic, personally, so it is not a consideration.

The multiple of the ATR goes up pretty fast as the you go from 100 ms to 10 ms and to smaller fractals. The asymptote is around 7X I think in pragmatic participating terms. At this point participant analysis dominates.

Your schema for always having completed fractal data sets using landmarking is a boon. But we will shift to the right of that when the participant analysis becomes dominant.

Prof logic uses some noise minimization strategies. that is fine but our money velocity, nominally, at 50% efficiency is four times his. Neither approach has a draw down component to consider for logic reasons.

what I call operating point will be renamed to the "state" name. As you saw in the industry (infomation), people did "states" according to the cart before the horse and yet another blow up logically speaking occurred.

Why I have guarded against dealing with anything accept certainty, lays in the reasoning that lead states to always blow up. A logician cannot entertain uncertainty as part of his practise and especially in functions that are not continuous functions (the case for trading to make money with capital in markets).

Why bars are used in trading is based on the analysis done before the paradigm shift to participant analysis. Bar analysis is the "ball park" analysis. At 3x ATR the ball park is Big League as most of the financial industry understands making money. I guess it more like Olympic to those people. I have seen 30 year floor persons say they have never in their lifetimes seen prints, calls, etc while using a simple set up manually since on those three floors of the MERC they were not feed and supportwise set up well enough in their trading rooms. Fortunately there was one losing trade of 1 tic per contract to reverse sides of the market.

When the two analysis operations (values and participants) are going on in parallel, the shifting of decision sets is done as the next possible "states" are eliminated in the zone bounded by the very near future on the right and just before the Present on the left. Before the state changes, there is only one state that is available because all others have been eliminated.

There is no oscillation as a function of having a potential multivariant "natural frequency". All the bearings of the Nike Ajax and Nike I has a mechanical dither oscillator built into the drive motors to eliminate starting friction in the 108 dc amplifiers of the analog computers. The drift of the dc amplifiers was zero set 48 times a second.

In monitoring and analysis of trading, whether value or participant, oscillation is removed by going to binary vector analysis. 70 degrees of it. State is a set that is a multivariant array where gating is used to select the subset in advance of the opportunity. There is no "reaction" concept involved nor is there a subset oscillation facet.

Smoothing is never used in the pane (zone above mentioned).

The seed of the seven bar cases, by the use of the null hypothesis, leads to MADA, MODE and Sentiment. MADA is the science of finite mathematics. MODE is the value differentiator. sentiment is the participant control function which is derived counterintuitively.
 
Quote from jack hershey:

I read the thread this am.

To help out I'll post quite a few pics that will be helpful.

I'll also cut and paste a few of your remarks to focus them a little from my perspective.
More...

Jack,

Thanks very much for your ongoing contributions. I am very excited by the potential of this thread.

Wayne,

Seems we wear similar shoes in the business world :) Thank you for initiating the discussion and sharing your insights as well.
 
Quote from jack hershey:

The computer is fast and so all degrees of freedom can be generated and available all the time (70 approximately).
More...

Jack, With the paradigm shift of the equinox, one can easily allow for 72.

Shifting from the human to the ATS is a matter of rep rates. In real time people, possibly, are limited. I am on automatic, personally, so it is not a consideration.
More...


Of course not Jack, you can ring the register in your sleep ... you are the man Jack!


what I call operating point will be renamed to the "state" name. As you saw in the industry (infomation), people did "states" according to the cart before the horse and yet another blow up logically speaking occurred.
More...


Of course, always make everything a moving target, change terms, etc ...

This is classic ... You should write science fiction. Got to run ... a Gaussian is forming that could be an FTT.
 
Wayne - I too have done a lot of business process re-engineering & Best Practice Implementation work professionally. I previously worked for a firm that consulted to companies looking to reduce headcount in administrative & support functions (i.e. accounting, supply chain management, etc.). We had methods to reduce Accounts Payable headcount by 95%.

While I think that using some of high-end business process analysis methods to identify discretionary trading methods to automate is a great idea, it does not seem that (from what I have seen) Jack's ideas are concrete enough to be automated. In reading this thread: http://www.elitetrader.com/vb/showt...=136304&perpage=40&highlight=who is jack her\[\^\\s\][^\s]*&pagenumber=1 Jack's ideas seem to be vague.

I think a better approach for discussing how to take one's discretionary method & automate all of some of it would be to get a more defined method that more people can relate to & understand. The method chosen is unimportant as long as it can be described fairly easily. I think this would bring the thread back to 'how to convert a discretionary trading system to an automated one' as the thread title implies.

Regards,
Eric
 
Quote from Trader922:

Wayne - I too have done a lot of business process re-engineering & Best Practice Implementation work professionally. I previously worked for a firm that consulted to companies looking to reduce headcount in administrative & support functions (i.e. accounting, supply chain management, etc.). We had methods to reduce Accounts Payable headcount by 95%.

While I think that using some of high-end business process analysis methods to identify discretionary trading methods to automate is a great idea, it does not seem that (from what I have seen) Jack's ideas are concrete enough to be automated. In reading this thread: http://www.elitetrader.com/vb/showt...=136304&perpage=40&highlight=who is jack her\[\^\\s\][^\s]*&pagenumber=1 Jack's ideas seem to be vague.

I think a better approach for discussing how to take one's discretionary method & automate all of some of it would be to get a more defined method that more people can relate to & understand. The method chosen is unimportant as long as it can be described fairly easily. I think this would bring the thread back to 'how to convert a discretionary trading system to an automated one' as the thread title implies.

Regards,
Eric
More...

Eric, that is kind of you to make such a practical suggestion. Also, your assessment of it being vague is valid.

I will certainly consider changing which methods and what we're approaching here. After all, my focus is to be profitable as quickly as possible and extract all the market offers and secondly to share process of how to get there with hopefully some assistance and collaboration.

Please allow me to proceed a little further with this endeavor to attempt to take the extremely complex and break it down into manageable pieces.

My suggestion for you and others who find Jacks work vague or confusing, avoid reading his posts and focus on reading mine (assuming you understand them). It's because I'm going repeat each step back to Jack to make sure I understand and let him correct me until I can describe it so that I understand and Jack agrees I got it right.

That's the process of B.P.A. in the simplest terms. You keep repeating back to the subject under study what you understand they said, until they agree you got it right.

Frankly, I don't know of anyone else who has a realistic premise (however vague it appears) to extract everything the market offers. If it turns out to truly be possible, then it's certainly a worthy goal.

Now, I simply want to explain my personal assessment of his vagueness.

I know other very skilled traders who teach people in forums but NEVER truly attempt teach how they trade themselves. Why? It's because they're UNABLE to express it in simple terms others can understand. Most of them simply say, "Watch price action for X number of hours and you'll get it."

What they do to avoid annoying people his only tell them what they're capable of handling at the moment. Or they simply answer the question asked.
We've all done that at times who are experts in various fields.

On B.P.R. and B.P.A, you can see obviously that I greatly over simplified the problem and left how a great deal of details.

Jack is a contrast to that mindset. He is kind enough and bold enough to frequently tell us that we're asking the WRONG question. That certainly feels frustrating at first and very humbling.

Plus, Jack seems to be the first I come across who is truly trying to explain essoterically how he trades himself.

Frankly, what Jack will probably recognize is that his 50 years of experience was focused on efficient and profitable trading with much less emphasis on training or experience with effectively teaching.

Have you heard that saying, "Those that can't do, teach"?

That truism isn't always the case. However, it's very common that those who are the MOST skilled at any undertaking are often the LEAST qualified to teach others, isn't that right?

The reverse is sometimes true as well that the most qualified teachers may be among the least qualified at doing something.

Well, to get to the point, Jack is not a teacher of trading. Just look at all the frustration generated on ET by potential students.

However, he is a superb trader. And right now, for this exercise I seek the skilled traders who can be vague or confusing as long as they will continue to patiently correct and refine till we get right.

Sincerely,
Wayne
 
I made an exception on the last post.

Allow me to clarify. This thread is never to discuss the merits of any discretionary strategy.

It's only a discussion of how to break down a complex strategy to manageable chunks in order to automate.

If you have questions or ideas about another strategy and how to automated it or thinking to do so, fine.

But any messages that critique or compare different strategies in this thread will get deleted.

Please do that in other threads as there are already plenty like that on ET.

One person was kind enough to PM me about his concerns rather than polute this thread. Thank you.

In short, in the OP and this post, it should be crystal clear as to the intention of the thread. In that way the moderator will delete any message that jumps outside these well defined boundaries.

Sincerely,
Wayne
 
Quote from ehorn:

I like the approach you have described.

I have taken the approach that every incoming tick is run through the "modules". MADA is done at the speed of the incoming data. Decision and Action though (for now) is only made at the close of each bar and thus constrained by the chosen fractal.

As witnessed, given my current simplistic and crude model (taken from the point of view of both the trader (me), the system, and the implementation of the algorithm), this tends to show up as making decisions "late" and/or "incorrect" and/or "un-optimized" which would suggest the need for an IF1 safety event as a decision and action overrider. But (IMHO) IF1 is not desired as it masks underlying fundemental flaws in the model and also hints at the lack of maturity and "degrees of freedom" present in the current model and it's developer :D The most recently discussed is the channels module and also the immature implementation of modelling internals and performing ADA with such modules.

So at a high level, the current design could be described as a funnel. where the fluid is the incoming tick data, passing thru the modules, and being filtered via the developers understanding of algorithm, filters, and context, to finally produce a decision.

The code is refined as the developer is refined as the trader is refined :)

The trader is refined through drills! The coach defines the drills, the student performs the drills. The student develops skills, knowledge and experience, and describes these to the developer, and the developer codes them to produce an ATS...

This is my approach :)
More...

First, what is IF1?

Second, you're approach is VERY similar to the one I'm adopting after researching a lot of design patterns for software decision support.

I wonder why you only make decisions at the end of the bar? I'm still fine tuning the M.A.D. of MADA (Monitor, Analyze, Decide, Action).

But I plan to make decisions instantly as the conditions and states change, tick by tick. Maybe you have a good reason I didn't consider.

In my next post, I'll spell out my progress, my approach, and issues.

Wayne
 
Status
Not open for further replies.
Back
Top