Quote from ScoobyStoo:
A few things occur to me...
1. If you rely on a stateful API the problem of maintaining state is just pushed one rung further up the ladder to the API technology provider. If they lose their connection to the exchange then they are going to slip out of synch just as you would if you were maintaining internal state.
First, when talking about this stuff, it would be helpful if you differentiated between order state, position state, and marketdata (subscriptions etc) state.
Look, you have a class like this.
Code:
public class MarketData
{
ThirdPartyAPI thirdPartyAPI;
....
SubscriptionEntity SubscribeSymbol(Security s)
{
return thirdPartyAPI.SubscribeSymbol(...);
}
}
as opposed to
Code:
public class MarketData
{
ThirdPartyAPI thirdPartyAPI;
Dictionary < Security, SubscriptionEntity > subscriptionEntities;
....
SubscriptionEntity SubscribeSymbol(Security s)
{
SubscriptionEntity se;
if(false == subscriptionEntities.TryGetValue(s, out se))
return thirdPartyAPI.SubscribeSymbol(...);
}
}
The point I am trying to convey is that you are keeping state in two places in the second case where the first case simply delegates state to the API (if it keeps it). This is poor design with no added benefit (this is a simplified case. Maybe you want to have multiple users. Then you have to keep a count of references to the subscription, so that you only unsubscribe from the thirdPary if there truly are no subscriptions. It is just a simplified example.)
In the case of OrderState, if the API is doing it for you already, same comment. It doesn't mean that you shouldn't duplicate some code in certain cases, it just means trying to delegate the state as much as possible to the owner of that state, to have lazy evaluation instead of eager evaluation as much as possible. The more the system needs to be fast, the more duplication that needs to occur. This is nothing more than saying you should cache state locally (although I have seen cases in functional programs where there is no state be faster than C++ threaded implementations that keep state. There is no one size fits all). But notice how perspective changes when you make explicit what you are doing. Now everything you learn about the perils of a cache immediately come to mind.
2. In the event of a connection failure between you and the API technology provider you have absolutely no idea what's going on. All those API requests to replay order state are going to fail. The advantage of maintaining local state is that at least you have a snapshot of that last known configuration. Alright, the config may change significantly whilst the connection is down but without it you are totally in the dark.
Your not understanding what I am saying. You are assuming that the API itself has to sync with the remote server. If the connection between the API and the provider goes down doesn't mean the API doesn't have the current state. Obviously someone is keeping state. either locally or remotely. I am saying if the API already keeps state locally, why would you duplicate that and offer another way that state can get out of sync? If the API has to query a remote server for that information, then of course you have to keep local state. I already said that. What I am saying is avoid duplication of local state.
3. I just don't see how you can build a system that doesn't maintain internal state if you are using a stateless messaging API (e.g. FIX). At the lowest level the exchanges maintain the state of orders in their engines and then spit out data messages regarding any state changes. In order for an app to have any idea of the current state of play, at some point these messages have to processed and the information contained within them persisted in some manner. Either the API does this and exposes the data via some sort of replay functionality or you do it. Either way, by definition this is maintaining state. Otherwise, how can you have an application which cannot query the state of the orders it has generated? State must be maintained somewhere in a format the app can query as required.
Maybe I'm missing something obvious here, if so put me on the right tracks...
Right. In the case where there is no local state maintained like FIX, which I mention above in another post, is another case where you have to do the local housekeeping yourself, you need to maintain local state yourself.
Finally I am also saying that in exceptional cases like crashes, the last thing I want to do is rely on local state if at all possible. What I do is on login to the broker, I request by some mechanism the opening position coming into the day (that often comes in the morning in some manner that you have to parse and store in SQLServer or whatever), and then request all intra-day orders up to now. I then rebuild my position and act only on that. Some APIs will send you orders you missed so you only have to update those on your local state. They have a software order-ACK. But even in this case I might still prefer to relive the days orders and fills. It takes 1 minute instead of 1 second. The point is I ultimately trust what my broker says my position and my order state is in exceptional cases. If given the choice, I always rebuild state from a broker on major exceptions.
