Zen and the art of ATS design...

Quote from Valdis:

It's all done in QuantDeveloper, including multi-strategy multi-portfolio level backtesting, reporting and optimization. Just read more docs :)
More...

Thanks for that. I will read more docs. Unfortunately, QuantDeveloper is not open source so I can't examine the implementation and make contributions to it. I didn't see anything about portfolio level backtesting when I scanned the docs you referred me to earlier but I may have missed it. At least I don't recall anything that would help in hammering out the implementation details for this project.
 
3. Access to OrderManager and PositionManager

The most logical thing to do would be for Strategy to provide references to these two objects to SignalGenerator when it is being constructed.

Then from within any method in the SignalGenerator implementation they can be accessed as member variables e.g.

positionManager.doThis();
orderManager.doThat();

Simple enough. Anybody see flaws with this approach, let me know.
 
4. Optimization

There are a couple of approaches I can think of to take for optimization.

One is perhaps more OO than the other. I'll start with the approach that is probably the easiest for an end-user but has potential cross-language issues.

Java annotations. With Java annotations you can markup source code with extra user-defined annotations. For optimization purposes, say you have a member variable named foo, you could add an annotation thus:

Code: 
@Optimizable int foo;

The annotation definition is trivial and something like this:

Code: 
import java.lang.annotation.*;

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
public @ interface Optimizable {}

Then at runtime, using reflection etc. any variables that are defined as optimizable can be retrieved and also set before running the strategy.

Simple enough. I have seen some software allow you to optimize on non-numeric variables such as arbitrary Strings. I can't remember which software, it was probably Amibroker or Neoticker. At the time I couldn't come up with a reason for needing this. Does anyone have any use cases for needing to optimize on non-numeric variables?

The other question on optimizing variables is whether it is best to define the range or set of values for optimizing a variable at optimization time by the end-user, or have the variable itself define the range or set of values to optimize on?

When defining a range you would specify a start, end and step values. When defining a set you would simply list the values.

Two different annotations could be used for this e.g.

Code: 
@RangeOptimizable (
  start=1,
  end=20,
  step=2
) int foo;

@ValueSetOptimizable("1,3,5,7,9,11") int bar;

Any Java developers spot any flaws, suggestions, improvements etc. to this approach, please add your input here.

Using annotations to defined optimization parameters is quite powerful and easy for the end-user system developer. However, something feels wrong about setting member variables via reflection even when they don't have public setter methods to do so. It's perhaps just me. Thoughts?

I'll discuss the other approach to dealing with optimization parameters next...
 
4. Optimization (Part 2)

The problem with the approach described is that it doesn't necessarily work in different languages. More specifically, I'm not sure it will work at all in scripting engines that I want to support (more on that later)

So, the alternative approach is to define a specific interface e.g. IOptimizable that would need to be implemented by the subclasser of SignalGeneratorSupport.

The interface would have methods/operations as follows:

getOptimizationParams()
setOptimizationParam() and/or
setOptimizationParams()

The first method would return a structure containing the parameter name and type in addition to optionally the range or set of values. This structure would probably be modelled in RangeOptimizationParam and ValueSetOptimizationParam classes etc.

The second method would allow at runtime the value(s) of the parameters to be set.

Both of these methods would have default implementations that wouldn't need to be overriden. The end-user would simply add optimization parameters in a "setup" or "init" method of the SignalGenerator e.g.

Code: 
setup() {
  addOptimizationParam(new RangeOptimizationParam("foo",1,20,2));
}

The addOptimizationParam method would add optimization parameters objects to a member variable which is then returned by getOptimizationParams()

It seems feasible that this would work across languages and scripting environments. The only question that remains is how the optimization parameters are accessed in the strategy code.

The programmer would either be forced to access the current values of optimization parameters via a local structure using syntax such as the following:

Code: 
if optimizationParams.getParam("foo").value() >= bar
...

OR, when parameters are set by the optimizer via the setOptimizationParam() method, it uses reflection to look up actual member variables of the class that correspond to the name given and then sets the value accordingly. This would allow the programmer to use code in the normal way:

Code: 
if foo >= bar
...

The problem with this is that it again, kind of breaks encapsulation. I'm also not sure how reflection responds when being used in a script.

As usual, thoughts, comments, ideas please!
 
Before I continue delving into the specifics of Strategy implementation I'd like to see if anyone has some input on the topic of real-time.

Given that Java Real-Time System (JRTS) is relatively new and doesn't support Java 5, and only runs on Solaris 10 and SPARC I can safely rule it out for the time being. The next version should remedy those restrictions but it could be a while till it arrives.

So, is using regular Java SE or EE going to be a problem for high-frequency trading systems? The same question applies to other languages which are not real-time and run on non real-time operating systems.

If you are a Java dev, do you have any techniques or recommendations for minimizing potential garbage collection interruptions at inopportune times e.g. JVM settings. Programming methods etc.

Does anyone have experience in this area that they would like to share? What are your favorite real-time operating systems? What kind of hardware do you use?
 
Whilst I'm waiting for the inevitable flood of responses to the real-time topic I'll continue with where I left off on the SignalGenerator.

I realize it might be a little difficult to see the whole picture at the moment but hopefully things will become clearer once I draft the next UML for this part of the system.

In order to do that, I need to refine the list of order management methods that I want to make available to the system developer . So, I'm jumping back to that before I cover scripting:

1. Order Management methods

I've spent a little while thinking about this one and have gone around in circles a bit. Looking at Wealthlab for example, there is a bewildering array of order management or "trading system" functions: http://www.wealth-lab.com/cgi-bin/WealthLab.DLL/getpage?page=FnTradingSystem.htm

Neoticker has a similar range of functions.

Cursory analysis suggests that some of the proliferation is perhaps due to the lack of callback functions that Mojo Trader will have. The most important one is the onPositionOpened() handler. This method I envisage as being used to trigger stop loss or profit target orders.

If anyone has experience with other trading software e.g. Tradestation, Amibroker etc. and are familiar with their order management methods, please add your input here!

I'll start off with the basics and some design options.

Terminology

1) Method names could reflect: long or short
2) Method names could reflect: buy or sell
3) Method names could reflect: buyToOpen, buyToClose, sellToOpen, sellToClose.
4) Method names could reflect buy, sell, buyToCover, sellShort

So effectively, there are two issues there: whether to use Long/Short or Buy/Sell nomenclature and whether to differentiate between operations if there is already a position.

I have seen some software give their order management methods the following semantics:

Buy - will close existing short position and open a new long position.
Sell - will close existing long position and open a new short position.

In effect, the order will be double the size if there was an existing position in the opposite direction.

Order Types

It seems to be the norm to provide separate methods for the different order types e.g.:

buyMarket (or just buy)
buyLimit
buyStop
etc.

Ditto for Sell methods.

Annotations

It seems sensible and is widely practiced to have annotations or notes metadata for an order for logging purposes. How best to implement this? From an end user point of view, simply having two different methods seems the most obvious choice e.g.

buyMarket(arg1,arg2);
or
buyMarket(arg1,arg2,"Yabadabadoo!");

i.e. provide an overloaded method for each order management method there is. This doubles the number of methods which bothers me a little. I wonder if Java's varargs might be of use here? Thanks to dcraig for bringing that to my attention.

Position Size

It makes a lot of sense and again is not uncommon to separate out position sizing or money management e.g. http://www.geniustrader.org/

The question is how best to do this or rather at this point, how best to present the options to the end-user developer via the order managment methods.

The conclusion I have come to is to again offer two versions of the same method. One where the position size is explicitly specified and the other where it is not and thus deferred to the position size strategy to figure out e.g.

buyMarket(arg1,100,"I have explicitly set the position size of this order!);

or

buyMarket(arg1,"The position size strategy is going to figure out the size of this order for me!");

I'll leave the details of position sizing or money management to a later discussion as there are a few options for dealing with that.

What I think is important is that a reference to the order created is returned from all of these order management methods so that the SignalGenerator can keep a hold of them and interrogate their properties later on. So, however the position sizing is done as long as this requirement is met there shouldn't be a problem.

Specifying the Instrument for the Order

This is yet another issue that could result in the proliferation of methods.

Multi-instrument strategies are crucial for my purposes and I'm not convinced this issue is supported properly by some software though I see the latest RightEdge beta now seems to have support for this.

It seems as if Multi-instrument support was a bit of an after-thought for NeoTicker as all of the order management methods are replicated with "Ex" versions which allow the specification of the instrument.

The question is how best to deal with specifying which instrument to trade. There are various options:

1) If you want to trade the Instrument that is applicable to the current event e.g. onBar() then either the Instrument is specified in the handler e.g. onBar(Instrument i,Bar b) and there fore when it comes to order management you do:

buyMarket(i,100,"100% up room to go!");

2) Alternatively, the instrument applicable to the current event is set by the Strategy so all you have to do is:

buyMarket(100,"No fear!");

This mandates an extra method on the SignalGenerator interface: setCurrentInstrument(Instrument i) which gets called just before the event handler. I'm trying to think if there are possible synchronization issues with this approach.

3) However, if you want to trade a separate instrument which is distinct to those for which you are receiving market data then you can explicitly create the appropriate instrument (perhaps in a initialization method and saved to a member variable) and then reference that Instrument in the order method e.g.

buyMarket(myOtherInstrument,100,"Arb city!");

Conclusion

Your thoughts an opinions much appreciated! Ideally, I'd like to keep the order management methods as clean and simple as possible but it's difficult to see how the number can be kept down whilst at the same time keeping the burden on the programmer to a minimum in terms of remembering which arguments to specify.

To summarize the optionality of arguments we have:

1) Annotation/Note
2) Position Size
3) Instrument

And then different methods for each order type and direction combination.

Remember these order management methods are not only creating an Order object but they are also submitting them to the OrderManager or perhaps being passed through some filters before it reaches the OrderManager.

If you're still awake after reading this lengthy post then let me know any ideas you have.
 
Quote from TraderMojo:

Before I continue delving into the specifics of Strategy implementation I'd like to see if anyone has some input on the topic of real-time.

Given that Java Real-Time System (JRTS) is relatively new and doesn't support Java 5, and only runs on Solaris 10 and SPARC I can safely rule it out for the time being. The next version should remedy those restrictions but it could be a while till it arrives.

So, is using regular Java SE or EE going to be a problem for high-frequency trading systems? The same question applies to other languages which are not real-time and run on non real-time operating systems.

If you are a Java dev, do you have any techniques or recommendations for minimizing potential garbage collection interruptions at inopportune times e.g. JVM settings. Programming methods etc.

Does anyone have experience in this area that they would like to share? What are your favorite real-time operating systems? What kind of hardware do you use?
More...

I don't have any great words of wisdom to offer and trying to read various articles on GC always seems to lead to a state of confusion. However, you can get a gut feeling for GC performance, by running any real time "ish" Java application with jconsole and just looking at the GC stats. eg run TWS with jconsole.

With the client JVM, it seems that there are two different GC algorithms - mark/sweep/compact and I forget the other one. Anyway mark/sweep/compact is the expensive one. It seems it gets called only when the heap approaches it's maximum size or if System.gc() is called. One might arrange to call System.gc () once in a while when nothing is cooking to minimize heap size and preempt the jvm calling it at an inconvenient time.

On the topic of jconsole, it might be the way to go for remote monitoring. You could "instrument" your app with the associated API.
 
Quote from TraderMojo:

Before I continue delving into the specifics of Strategy implementation I'd like to see if anyone has some input on the topic of real-time.

Given that Java Real-Time System (JRTS) is relatively new and doesn't support Java 5, and only runs on Solaris 10 and SPARC I can safely rule it out for the time being. The next version should remedy those restrictions but it could be a while till it arrives.

So, is using regular Java SE or EE going to be a problem for high-frequency trading systems? The same question applies to other languages which are not real-time and run on non real-time operating systems.

If you are a Java dev, do you have any techniques or recommendations for minimizing potential garbage collection interruptions at inopportune times e.g. JVM settings. Programming methods etc.

Does anyone have experience in this area that they would like to share? What are your favorite real-time operating systems? What kind of hardware do you use?
More...

Not a java guru myself, but maybe you can take a look at www.javolution.org and see if this can help you with some of the issues address by RTSJ.
 
Quote from dcraig:

On the topic of jconsole, it might be the way to go for remote monitoring. You could "instrument" your app with the associated API.
More...

Yes, I was considering MBean wrappers for the Strategy Container, Provider Registry etc. It's non-intrusive and gives an instant GUI (Jconsole + other management applications) and also web interface via an appropriate HTTP adaptor.

Given that the entirety of JBoss is built from MBeans, it seems feasible enough to extend the interfaces to provide more than just management functionality. Indeed, it's on my list to investigate deploying it as a service to run on the JBoss microkernel and get clustering support etc.

Althought it's fairy trivial to implement MBean wrappers, Spring makes it even easier! Just a few extra lines of XML and you have instant JMXification.

Using MBean notifications for the event bus in a similar vein to JBoss is also a possibility.

I was even looking at putting the MBean inside of an OSGi bundle and composing the application of OSGi bundles. Talk about a riddle wrapped inside a mystery inside an enigma!

Given that there are so many friggin' technologies to choose from (most of which are new to me), I'm trying to ignore it all for the moment lest I get information overload and focusing on the abstractions instead.

Thanks for the input! I was beginning to think I was talking to myself. Any other ideas you have on this matter much appreciated.
 
Quote from xiaohu:

Not a java guru myself, but maybe you can take a look at www.javolution.org and see if this can help you with some of the issues address by RTSJ.
More...

Oh boy. Thanks for that....but I wish you hadn't. Now I'm going to spend yet more countless hours on that research tangent. Damn those hyperlinks.
 
You must log in or register to reply here.
Back
Top