I've already posted here some rant about data providers from the black box developer point of view. Today I have some free time, so this time I will complain about brokers.
The black box trading differ from human trading by one very important aspect: error handling. It means, that in case of ANY problem program should either know how to solve it, or if it can't, then get to human and ask him for fix
The approach I took with my black box was this: I don't want to babysit it. Basically, I want this program to be automated as much as possible - to the point of automated money farming (or printing) machine. I mean, I should be able to leave it running for months without me even checking "how are you doing?". I know that this will never happen, but it is a goal.
So, the best way here would be to not react to errors, but prevent them from happening.
I've already coded for all sort of errors that I can think of - connection failure, harware failure, etc. Now we are at a point of prevening broker errors. And here comes unpleasant discovery: there are NO brokers properly suited for black box trading. Well, at least in my price range.
First of all, let's start with an API.
The sort-of-standard right now is FIX gateway. From reliability point of view, this is probably the best API at this moment. The reason is - each message has a number, and both sides are very careful to process them in order; and if any number is missing, they will re-request for it as long as needed. If it never received, and other party does not provide an excuse, then no further input is taken, preventing from errors. This was good part. Now to the bad part. This protocol is not mature enough. For example, you can request list of all open/filled orders for today (this is done via requesting list of executions), but - surprise - this list doesn't have "end" mark. It means, that you never know if you received list of ALL orders, or you should wait for more. There are ways to overcame this issue (namely - you have to keep ALL messages for the day, but if you lost them - you doomed), and I coded for that, but let me tell you guys - it's pain in the #ss.
Let me be somewhat technical here. Some brokers provide their own API, so you can link their lib to your program and use their DLL. Here comes "small" issue - so far ALL APIs that I've seen are designed to be used in Windows (that's OK for me), and require you to have Windows message loop running in your program (that's their way of synchronizing/sending messages). Basically, it means that it's pain in the #ss to use their API in a windowless program (like service, for example). I understand why it is done this way - 99% of API users create programs with user interface, so they will have that loop anyway, and who cares about the rest - black box developers?! It has nothing to do with the error handling, just some rant - I have to spend precious time to solve these issues, that could have been totally avoided if programmers in broker's companies would have been a little more intelligent.
Anyway, let's move along. Now we have some API to use, and we want to apply our logic. Here is the scenario how human works:
1. (Open a position). Send an order (order A) to buy X shares of Y stock at market (or limit) price.
2. Get filling confirmation.
3. (Protect position). Send a stop order (order B) to sell X shares of Y stock if it fails below Z price.
4. (Close position). When decision is made to close a position, send an update request - to update order B to market. Once it gets updated, it gets executed and position closes.
The pitfall is obvious: if something goes wrong between steps 2 and 3 (for example, workstation goes down, Internet conection goes down etc) and stock at the same time plunges 50% down - account gets hit of 50%, because protection order (order B) is not in place. Possible solution for human is to pick up a phone and try to place order by phone.
Now, how it should work for blackbox (or carebear human):
1. (Open a position). Send conditional order (order A) to buy X shares of Y stock at market price; once filled, server creates a corresponding stop order (order B) to sell X shares of Y stock if it fails below Z price.
Quantity for the second (conditional) order is always equal to the quantity of the first order, so it is not specified in the request.
2. Get filling confirmation for the first order.
3. Get creating confirmation for the second order.
4. (Close position). When decision is made to close a position, send an update request - to update order B to market. Once it gets updated, it gets executed and position closes.
As you can see, because creation of the order B takes place at the server (not a client!), position is always protected (unless server itself gets down, but chances are way less). Even if client goes down right after opening a position and never gets any conrifmation, position is still protected, as protection order was created on a server once opening order got filled.
Scenario sounds reasonable, right? Wrong!
At first, I found this company: MBtrading.com. They provide FIX gateway (good!), they provide complex orders for the step 1 (some limitations apply, though: you can place order A limit + order B stop, but you cannot place order A market + order B stop). But - they cannot provide step 4. Literally: you cannot update order B from stop market to market. You can only update stop market to stop limit (very useful, right?).
Then, I found Ameritrade. They provide their own API (some issues exist, but nothing major), they provide complex orders for the step 1 (no limitations, by the way). But again - they cannot provide step 4! To be exact - part of complex order cannot be modified at all. It can be only cancelled.
After that, I reverted to the human logic, i.e. no complex orders to start trade, and take a risk between steps 2 and 3. Well, MBTrading still cannot provide step 4. Ameritrade can, but does it very strange way - they cancel order (on a server side) and issue new one (also on a server side) instead of just updating the order. This created some minor technical issue (I've already fixed it), but at least it works.
Now, once I don't need complex orders, I started to look for other brokers - may be they got something better - may be better price, or API, or stability. To my surprise:
- Genesis: cannot modify order at all - only cancel.
- TradeWallStreet (also know as ECNDirect and other names): can modify only number of shares in order, not it's type.
I did not check others, but have a feeling, that situation is not better there either.
By the way, speaking about error handling. How do you like error message from Ameritrade API: "Your order MIGHT have been sent". Nice, right? While it's not an issue for human, how my program should react to this?!
So, my conclusion is this: if you have a blackbox for trading - you have no broker suited for you.
All-in-all, I'm still with Ameritrade. They are not ideal, but still no one else is there for me who would be better
. And I don't like this situation - there should be at least SOME competition!
The black box trading differ from human trading by one very important aspect: error handling. It means, that in case of ANY problem program should either know how to solve it, or if it can't, then get to human and ask him for fix
The approach I took with my black box was this: I don't want to babysit it. Basically, I want this program to be automated as much as possible - to the point of automated money farming (or printing
) machine. I mean, I should be able to leave it running for months without me even checking "how are you doing?". I know that this will never happen, but it is a goal.So, the best way here would be to not react to errors, but prevent them from happening.
I've already coded for all sort of errors that I can think of - connection failure, harware failure, etc. Now we are at a point of prevening broker errors. And here comes unpleasant discovery: there are NO brokers properly suited for black box trading. Well, at least in my price range.
First of all, let's start with an API.
The sort-of-standard right now is FIX gateway. From reliability point of view, this is probably the best API at this moment. The reason is - each message has a number, and both sides are very careful to process them in order; and if any number is missing, they will re-request for it as long as needed. If it never received, and other party does not provide an excuse, then no further input is taken, preventing from errors. This was good part. Now to the bad part. This protocol is not mature enough. For example, you can request list of all open/filled orders for today (this is done via requesting list of executions), but - surprise - this list doesn't have "end" mark. It means, that you never know if you received list of ALL orders, or you should wait for more. There are ways to overcame this issue (namely - you have to keep ALL messages for the day, but if you lost them - you doomed), and I coded for that, but let me tell you guys - it's pain in the #ss.
Let me be somewhat technical here. Some brokers provide their own API, so you can link their lib to your program and use their DLL. Here comes "small" issue - so far ALL APIs that I've seen are designed to be used in Windows (that's OK for me), and require you to have Windows message loop running in your program (that's their way of synchronizing/sending messages). Basically, it means that it's pain in the #ss to use their API in a windowless program (like service, for example). I understand why it is done this way - 99% of API users create programs with user interface, so they will have that loop anyway, and who cares about the rest - black box developers?! It has nothing to do with the error handling, just some rant - I have to spend precious time to solve these issues, that could have been totally avoided if programmers in broker's companies would have been a little more intelligent.
Anyway, let's move along. Now we have some API to use, and we want to apply our logic. Here is the scenario how human works:
1. (Open a position). Send an order (order A) to buy X shares of Y stock at market (or limit) price.
2. Get filling confirmation.
3. (Protect position). Send a stop order (order B) to sell X shares of Y stock if it fails below Z price.
4. (Close position). When decision is made to close a position, send an update request - to update order B to market. Once it gets updated, it gets executed and position closes.
The pitfall is obvious: if something goes wrong between steps 2 and 3 (for example, workstation goes down, Internet conection goes down etc) and stock at the same time plunges 50% down - account gets hit of 50%, because protection order (order B) is not in place. Possible solution for human is to pick up a phone and try to place order by phone.
Now, how it should work for blackbox (or carebear human):
1. (Open a position). Send conditional order (order A) to buy X shares of Y stock at market price; once filled, server creates a corresponding stop order (order B) to sell X shares of Y stock if it fails below Z price.
Quantity for the second (conditional) order is always equal to the quantity of the first order, so it is not specified in the request.
2. Get filling confirmation for the first order.
3. Get creating confirmation for the second order.
4. (Close position). When decision is made to close a position, send an update request - to update order B to market. Once it gets updated, it gets executed and position closes.
As you can see, because creation of the order B takes place at the server (not a client!), position is always protected (unless server itself gets down, but chances are way less). Even if client goes down right after opening a position and never gets any conrifmation, position is still protected, as protection order was created on a server once opening order got filled.
Scenario sounds reasonable, right? Wrong!
At first, I found this company: MBtrading.com. They provide FIX gateway (good!), they provide complex orders for the step 1 (some limitations apply, though: you can place order A limit + order B stop, but you cannot place order A market + order B stop). But - they cannot provide step 4. Literally: you cannot update order B from stop market to market. You can only update stop market to stop limit (very useful, right?).
Then, I found Ameritrade. They provide their own API (some issues exist, but nothing major), they provide complex orders for the step 1 (no limitations, by the way). But again - they cannot provide step 4! To be exact - part of complex order cannot be modified at all. It can be only cancelled.
After that, I reverted to the human logic, i.e. no complex orders to start trade, and take a risk between steps 2 and 3. Well, MBTrading still cannot provide step 4. Ameritrade can, but does it very strange way - they cancel order (on a server side) and issue new one (also on a server side) instead of just updating the order. This created some minor technical issue (I've already fixed it), but at least it works.
Now, once I don't need complex orders, I started to look for other brokers - may be they got something better - may be better price, or API, or stability. To my surprise:
- Genesis: cannot modify order at all - only cancel.
- TradeWallStreet (also know as ECNDirect and other names): can modify only number of shares in order, not it's type.
I did not check others, but have a feeling, that situation is not better there either.
By the way, speaking about error handling. How do you like error message from Ameritrade API: "Your order MIGHT have been sent". Nice, right? While it's not an issue for human, how my program should react to this?!
So, my conclusion is this: if you have a blackbox for trading - you have no broker suited for you.
All-in-all, I'm still with Ameritrade. They are not ideal, but still no one else is there for me who would be better
. And I don't like this situation - there should be at least SOME competition!