LMAX inside spread liquidity

[volpunter]

I have not used the Java API but I would find it weird if it would not allow the submission and reporting of custom tags, such as your own OrderID. FIX allows for that as part of the standard protocol definition and it works. Can you maybe talk again with the tech guys at LMAX? I am willing to bet that you can submit a custom tag as I have not seen any API so far that would not allow for such.

Here is a high level summary of development so far, as I am migrating
a complex multi-threaded, multi-order scalping platform from Dukascopy
to LMAX. The goal here is to preserve the core logic but, in order to do
that, we had significant challenges in emulating the Dukascopy API and
Order management concepts on top of LMAX orders.

Here we are using the Java API, which presents lower level concepts and
is missing many of the higher level facilities we relied upon in the Dukascopy
API, such as the Order IEngine, and the simple ability, for example, to
change the Price of a Live order. With Dukascopy, it's simple to change
the price of a live order, but this high level concept is absent in the LMAX
Java API and thus we will have multiple Cancel/Place cycles underlying
the filling of a single high level Order.

1) Mapping a high level "Dukascopy style" order to underlying LMAX orders.
I've had to emulate the Dukascopy IEngine interface, which handles all of
order state, as well as IOrder states and operations such as IOrder.setPrice.
Within this emulation, we may use dozens of underlying LMAX orders to
achieve a single high level order fill objective, given that we will need to
change price during execution, which requires us to Cancel/Place a new
LMAX Limit order.

2) Emulating Dukascopy's IEngine was a challenge, as we had to "host"
Dukascopy IStrategy code, create a SingleThreadedExecutor, and subclass
"public class MyThreadPoolExecutor extends ThreadPoolExecutor ..."
to give us access to "afterExecute" so that we could "batch" together
multiple IOrder emulated operations, such as change price, lot size, add
StopLoss, add StopProfit, etc. And then at the end of the Thread task, on the
same Thread, the afterExecute Java thread pool would "batch invoke" the code
which performs all of the required operations on the underlying LMAX Order(s),
including Cancel/Place sequences, when price changes... Also, because multiple
LMAX orders are needed to support our high level objective, we are not able
to use LMAX Order StopLoss and StopProfit operations, as they represent
only a portion of our high level IOrder's objective...

3) Dukascopy allows us to differentiate facilities by a user-supplied "Order Label"
so we can differentiate activity by parsing the Order Label, thus ignoring orders
which are not related to the current client's orders. LMAX does not appear
to allow us to assign a meaningful Order ID in this way, so we are likely to
have to implement 1) a database which preserves Dukascopy style order
Labels mappings to LMAX Orders, or 2) much easier, just use multiple
LMAX accounts, each for a specific trading BOT or client trading platform.

4) Differentiating between actions of our Client, and actions of the LMAX
platform or another client. By assigning "instructionIDs" in our client
which are always AlphaNumeric, then when we receive instructions IDs
which are purely Numeric, we know the activity did not originate from
this client this is not a comprehensive solution, and it looks like we'll
have to implement a database.

However, without going into further detail, their Java API (and, by extension,
I suppose, their .NET API which is very similar in its semantics) is quite
powerful in permitting the multi-threaded management of multiple orders,
on both sides of the market simultaneously, with very good response times.

But it makes me appreciate the high level facilities offered by the Dukascopy
API which we have to emulate here at LMAX.... We're still a couple of days
away from a minimally working platform and, by substantially emulating
the basic Dukascopy API, we will then be able to adapt many of our existing
BOTs also, but many other high level constructions such as Indicators
are areas which I am hoping not to have to emulate at all, and we can just
grab that information from another streaming source such as Dukascopy.

And all of this work is to get "wholesale, or better than retail" pricing liquidity,
which we haven't been able to validate yet. Lots of work, hoping for results...

HyperScalper

 

[volpunter]

its really not all that complicated. You assign original orders with a custom tag, you get back a submit confirm with the custom tag and assigned LMAX order ID, you store both in a hash table (or Dictionary if you are coding in .Net) and have access whenever you need. With that you can at any time cancel/modify existing orders.

Some complicated sh*t ..hahahha

 

[FutureScalper]

I have targeted LMAX through their FIX gateway. What in the Java API doc is inaccurate? Care to explain?

Well, "instructionId" , "originalInstructionId" are not java "long" data
types. They have been changed to a String's, and neither Javadoc
nor demo examples have been updated to reflect that fact.

Simply regenerating the Javadoc would be something they
could easily do... Other than that, with a great deal of ingenuity as
examples are very sparse, it works well. Reconnection is an issue that
has been challenging for a long-running professional application, so
opposed to their "demo quality" examples.

They don't expect pro-level firms to be using their Java API, and stated
as much, saying it was not "perfect" and that we should use FIX. Sure,
I'll consider that down the road but they should really do a better
job of showing real complex applications using their Java API. All in all,
we've nearly figured it out and it was probably easier than going
FIX right away.

 

[FutureScalper]

its really not all that complicated. You assign original orders with a custom tag, you get back a submit confirm with the custom tag and assigned LMAX order ID, you store both in a hash table (or Dictionary if you are coding in .Net) and have access whenever you need. With that you can at any time cancel/modify existing orders.

Java API here, which is quite similar to their .NET, both being relatively "thin" layers over the XML.
I'm aware of, and we always use our own String custom instructionIds in the Java API which helps
a bit. But our specific requirement is to be able to change the price of a high level emulation
of a Dukascopy-style order. And AFAIK an LMAX order has no price change in the Java API, so
we have to do Cancel / Resubmit. I could possibly be wrong, but I don't see how to provide our own
"order id" to LMAX as it seems to generate its own. I don't think I can get away from having
to keep a database mapping our high level order id strings, to LMAX order id's but that's a
relatively minor issue to resolve, and relates to restart and recognition of prior existing order
mappings, etc. Our goal was to determine the liquidity inside the spread, and migrate an
existing application which uses Dukascopy semantics. Given that, we have to emulate the
Dukascopy API IEngine and related concepts. LMAX API does not have a lot of these "convenience
features" so it's a bit challenging but coming along pretty well. Ironically, we still don't yet
know what the real liquidity is going to be, and are a few days away still...... always takes
longer than expected, I've found, in a real robust migration. HyperScalper.

 

[FutureScalper]

I have not used the Java API but I would find it weird if it would not allow the submission and reporting of custom tags, such as your own OrderID. FIX allows for that as part of the standard protocol definition and it works. Can you maybe talk again with the tech guys at LMAX? I am willing to bet that you can submit a custom tag as I have not seen any API so far that would not allow for such.

To clarify, their "Order ID" is LMAX assigned and is returned to the client.
An "Instruction ID" is a freely formatted String which is used to correlate confirmations of
individual instructions. A special case of that is the "Original Instruction ID"
which is the one which initially created the Order. These Instruction ID
strings can have any proprietary String format. However, if you have to
change Order price.... then (as far as I can fathom) the order must be
Cancelled and a new Order (with a new LMAX assigned Order ID) is created,
again your "Original Instruction ID" is associated with this new Order, and
so on. Our system which will change prices of a "high level Order concept"
will thus have to submit potentially dozens of underlying LMAX orders
in order to FILL the desired quantity. It would have been nice to be able
to change the LMAX limit order price while the LMAX-assigned Order ID would
remain a constant, but that does not appear to be possible.

 

[volpunter]

I agree with you that always a better job can be done in the "documentation department", especially as a lot of this can nowadays be entirely automated.

But I think you get to the core point of this with mentioning FIX. If you want a production stealth quality API then you should go the FIX route rather than Java. If you use even an entry level proprietary FIX engine or quickFIX, a number of base classes can actually really speed up the process to target a new broker via FIX. In the end the only task you are left with is message parsing and mapping. You will see that if most your brokers offer FIX you will be much better off in terms of operational stability as well as speed of development to go the FIX rather than Java API route (or .Net route for that matter).

Well, "instructionId" , "originalInstructionId" are not java "long" data
types. They have been changed to a String's, and neither Javadoc
nor demo examples have been updated to reflect that fact.

Simply regenerating the Javadoc would be something they
could easily do... Other than that, with a great deal of ingenuity as
examples are very sparse, it works well. Reconnection is an issue that
has been challenging for a long-running professional application, so
opposed to their "demo quality" examples.

They don't expect pro-level firms to be using their Java API, and stated
as much, saying it was not "perfect" and that we should use FIX. Sure,
I'll consider that down the road but they should really do a better
job of showing real complex applications using their Java API. All in all,
we've nearly figured it out and it was probably easier than going
FIX right away.

 

[volpunter]

What exactly do you intend to modify? A quick look reveals that the Java API allows for a modification of stops and limits even of existing orders without having to cancel/re-submit. (see AmendStopsRequest [refers to both limits and stops unless I remember incorrectly]).

Yes LMAX generates its own orderIDs which is why you tag your original order with your own orderID and then when you get a confirm you get both, your tagged orderID and the LMAX generated orderId back which you store in a hash table or other type of dictionary.

Java API here, which is quite similar to their .NET, both being relatively "thin" layers over the XML.
I'm aware of, and we always use our own String custom instructionIds in the Java API which helps
a bit. But our specific requirement is to be able to change the price of a high level emulation
of a Dukascopy-style order. And AFAIK an LMAX order has no price change in the Java API, so
we have to do Cancel / Resubmit. I could possibly be wrong, but I don't see how to provide our own
"order id" to LMAX as it seems to generate its own. I don't think I can get away from having
to keep a database mapping our high level order id strings, to LMAX order id's but that's a
relatively minor issue to resolve, and relates to restart and recognition of prior existing order
mappings, etc. Our goal was to determine the liquidity inside the spread, and migrate an
existing application which uses Dukascopy semantics. Given that, we have to emulate the
Dukascopy API IEngine and related concepts. LMAX API does not have a lot of these "convenience
features" so it's a bit challenging but coming along pretty well. Ironically, we still don't yet
know what the real liquidity is going to be, and are a few days away still...... always takes
longer than expected, I've found, in a real robust migration. HyperScalper.

 

[volpunter]

I think this is where you are incorrect. You can modify limit and stop prices of orders without having to cancel and re-submit. Hence, the InstructionID you assigned will remain unchanged and refers to the same order you originally created.

To clarify, their "Order ID" is LMAX assigned and is returned to the client.
An "Instruction ID" is a freely formatted String which is used to correlate confirmations of
individual instructions. A special case of that is the "Original Instruction ID"
which is the one which initially created the Order. These Instruction ID
strings can have any proprietary String format. However, if you have to
change Order price.... then (as far as I can fathom) the order must be
Cancelled and a new Order (with a new LMAX assigned Order ID) is created,
again your "Original Instruction ID" is associated with this new Order, and
so on. Our system which will change prices of a "high level Order concept"
will thus have to submit potentially dozens of underlying LMAX orders
in order to FILL the desired quantity. It would have been nice to be able
to change the LMAX limit order price while the LMAX-assigned Order ID would
remain a constant, but that does not appear to be possible.

 

[FutureScalper]

I think this is where you are incorrect. You can modify limit and stop prices of orders without having to cancel and re-submit. Hence, the InstructionID you assigned will remain unchanged and refers to the same order you originally created.

Hey Volpunter, I'd dearly love if you were correct, but I can't see how. Could you specifically say how an existing open order can have its limit price amended through the Java API? "Stop Losses" and "Stop Profits" can be added/removed from an existing order, but I'm not smart enough to see how the Limit price of an existing open order can be modified. They include a "TradeInsideTheSpread" crude example, and that would be an ideal application for a limit price change, but the code Cancels and then submits a new Limit order every time it needs to change price. This is not a big deal for me, the Cancel/Submit sequence, but if I could avoid it, again, using their Java API, please tell me how. Here is their AmendStopsRequest, but of course this next signature is incorrect. All Instruction ID's are now Strings.

AmendStopsRequest(long instrumentId, long originalInstructionId, long instructionId,FixedPointNumber stopLossOffset, FixedPointNumber stopProfitOffset)

The real signature is:

AmendStopsRequest(long instrumentId, String originalInstructionId, String instructionId, FixedPointNumber stopLossOffset, FixedPointNumber stopProfitOffset)

This is achieved through Session.amendStops an unlimited number of times on an existing order identified using its Original Instruction ID.

voidamendStops(AmendStopsRequest amendStopLossProfitRequest, OrderCallback orderResponseCallback)

You may have more insight than I do, so can you say how to change the Limit price of an existing order with the Java API? Once again, I don't mind the Cancel/Submit requirement for price change if it's unavoidable.

HyperScalper

 

[volpunter]

please give me a day, I am happy to dig up some of the docs I may still have lying around.

Hey Volpunter, I'd dearly love if you were correct, but I can't see how. Could you specifically say how an existing open order can have its limit price amended through the Java API? "Stop Losses" and "Stop Profits" can be added/removed from an existing order, but I'm not smart enough to see how the Limit price of an existing open order can be modified. They include a "TradeInsideTheSpread" crude example, and that would be an ideal application for a limit price change, but the code Cancels and then submits a new Limit order every time it needs to change price. This is not a big deal for me, the Cancel/Submit sequence, but if I could avoid it, again, using their Java API, please tell me how. Here is their AmendStopsRequest, but of course this next signature is incorrect. All Instruction ID's are now Strings.

AmendStopsRequest(long instrumentId, long originalInstructionId, long instructionId,FixedPointNumber stopLossOffset, FixedPointNumber stopProfitOffset)

The real signature is:

AmendStopsRequest(long instrumentId, String originalInstructionId, String instructionId, FixedPointNumber stopLossOffset, FixedPointNumber stopProfitOffset)

This is achieved through Session.amendStops an unlimited number of times on an existing order identified using its Original Instruction ID.

voidamendStops(AmendStopsRequest amendStopLossProfitRequest, OrderCallback orderResponseCallback)

You may have more insight than I do, so can you say how to change the Limit price of an existing order with the Java API? Once again, I don't mind the Cancel/Submit requirement for price change if it's unavoidable.

HyperScalper