Writing Robust Trading Code
- Thread starter gmst
- Start date
Above kind of guys will typically experience following problems:
1) Temporary connection loss due to internet outages
2) Temporary Feed loss (and missing packets for extended duration say 5 seconds or more), even when internet is fine because their feed provider has a temporary outage.
3) Sometimes their application will crash and they will have to re-start the application during the trading day. (say you trade through Excel or MC or your custom written C# program and it crashes during the trading day).
1) Temporary connection loss due to internet outages
2) Temporary Feed loss (and missing packets for extended duration say 5 seconds or more), even when internet is fine because their feed provider has a temporary outage.
3) Sometimes their application will crash and they will have to re-start the application during the trading day. (say you trade through Excel or MC or your custom written C# program and it crashes during the trading day).
While backtesting, above issues are not present. Your backtesting data is typically complete, so there are no missing bars/ticks due to internet outages or outages at your feed provider. Say, your application crashes while a backtest is in progress, you can simply re-start it and re-run your backtests. You are not affected since you are not executing anything in real-time.
Due to above differences in backtesting environment and live execution environment, I have come to conclusion that actual execution code for such retail traders has to be different than the backtesting code. The execution code must be robust and must handle above issues. It doesn't matter if you are fully automated or are just using your system as a gray-box.
So, my question is:
1) Do you guys write much more robust execution code than your backtesting code?
2) How large is the difference between such robust code for execution Vs simpler code for backtesting in terms of increase in complexity, increase in number of variables/arrays that you have to use and round-about logic that needs to be used to make the live execution code robust?
Due to above differences in backtesting environment and live execution environment, I have come to conclusion that actual execution code for such retail traders has to be different than the backtesting code. The execution code must be robust and must handle above issues. It doesn't matter if you are fully automated or are just using your system as a gray-box.
So, my question is:
1) Do you guys write much more robust execution code than your backtesting code?
2) How large is the difference between such robust code for execution Vs simpler code for backtesting in terms of increase in complexity, increase in number of variables/arrays that you have to use and round-about logic that needs to be used to make the live execution code robust?
Examples of how complexity might increase would be -- at every step, you would have to check:
1) if an internet outage has happened, and if it has happened, what do you do? Well in most cases, you will do nothing and wait. Say your data connection gets re-established after 10 minutes, you have got a gap in your data. How do you make changes in your code so that your strategy will continue to behave as expected?
2) your code must be fail-safe in the event of your application-crash and upon re-starting in the middle of a trading day - with live data coming into your platform and your trading code acting on it - you should be able to re-assign all the variables/arrays with the values present in variables/arrays before the crash happened. This means that after every critical operation, you must (print/write to file) the array/variables values somewhere and then in the next step in the code - read those values from that location. Such an approach will make your code robust against your application crash.
However, the code complexity increases a lot - in my limited experience. So, is incorporating above points worthwhile in your live code? What has been the experience of gray/black box semi/fully automated traders?
1) if an internet outage has happened, and if it has happened, what do you do? Well in most cases, you will do nothing and wait. Say your data connection gets re-established after 10 minutes, you have got a gap in your data. How do you make changes in your code so that your strategy will continue to behave as expected?
2) your code must be fail-safe in the event of your application-crash and upon re-starting in the middle of a trading day - with live data coming into your platform and your trading code acting on it - you should be able to re-assign all the variables/arrays with the values present in variables/arrays before the crash happened. This means that after every critical operation, you must (print/write to file) the array/variables values somewhere and then in the next step in the code - read those values from that location. Such an approach will make your code robust against your application crash.
However, the code complexity increases a lot - in my limited experience. So, is incorporating above points worthwhile in your live code? What has been the experience of gray/black box semi/fully automated traders?
hm.... well the "average" feeds you mention are pretty good.
not sure how many problems with this, i would imagine that most of your problems come from your first #1 and #2.
this is why people running discretionary programs often trade in an office and/or run their programs from the cloud (eg aws), even if they're not colocated next to an exchange... cause the infrastructure is beefier to handle these problems on regular basis.
still nothing is perfect. even in a colo environment there is no guarantee you won't have a feed hiccup, so most of the same things apply. you need backup connections and notifications, and ability to restart like you allude to, just as a failsafe if nothing else.
not sure how many problems with this, i would imagine that most of your problems come from your first #1 and #2.
this is why people running discretionary programs often trade in an office and/or run their programs from the cloud (eg aws), even if they're not colocated next to an exchange... cause the infrastructure is beefier to handle these problems on regular basis.
still nothing is perfect. even in a colo environment there is no guarantee you won't have a feed hiccup, so most of the same things apply. you need backup connections and notifications, and ability to restart like you allude to, just as a failsafe if nothing else.
backup UPS beeps pretty loud if power outage. you have at least a few minutes to get out of the trade
if internet down too, there's dongle backup wireless
you can setup code with a status of long, short flat for example so if by chance the feed went out during barupdate, then it gets up to status of being long for example
other broker account in case have to get in and offset a position unable to get to through other broker
if internet down too, there's dongle backup wireless
you can setup code with a status of long, short flat for example so if by chance the feed went out during barupdate, then it gets up to status of being long for example
other broker account in case have to get in and offset a position unable to get to through other broker
I think fault tolerance is defined by the brokers' rules. I use FIX protocol to place orders and each FIX message has a sequence number. Every time I receive a new message from a broker I check if it is not out of sequence. If it is out of sequence I send out Resend Request message to resend me missing messages. The broker checks my messages in the exact way. I use sockets to connect to the broker's FIX server. If I do not send order messages within 30 sec. then I send Ping message to see if connection is still alive. If my socket raises an error anytime I send or receiving a message then I know something is wrong: it's either broker's server is down, or connection is down. Then I reboot my socket connection and try to re-connect every 2 min. When I connect to broker's FIX server, first I send Logon Request and receive Logon Confirmation, from this first messages' sequence numbers my broker and I find out if there are any lost messages and initiate Resend Request. If you place orders using a broker's proprietary API (which I never used) I imagine it should have similar mechanism to retrieve lost messages o.w. you might be stuck forever waiting for an order confirmation. Fault tolerance does add complexity but nothing terrible.
If you lose connection to a data feed server, one technique I saw is to fill missing ohlc bars with an average. On the other hand, I trade using tick data, if, say I loose last 2 min of ticks, I do not know how to handle that - I would like to find out too.
If you lose connection to a data feed server, one technique I saw is to fill missing ohlc bars with an average. On the other hand, I trade using tick data, if, say I loose last 2 min of ticks, I do not know how to handle that - I would like to find out too.
IMO it's important to run exactly the same signal code and order generation code in "live" as in backtest, to avoid inadvertently trading something different to the edge identified. But, yes, unfortunately the code for "live" must often have additional complexity and dimensions to deal with issues like those you have stated. And so sometimes the principle of keeping the code the same cannot be adhered to 100%.
I attempt to include the "live" specific stuff within the same code I backtest with, but wrap it within a conditional statement that will only execute when "live". In this way I can use the same code in "live" and backtests (but obviously there is unfortunately a slight difference in which code actually gets used in each case).
So, with NT for example ...
if (!Historical)
{
// only gets executed in "Live"
}
I'm not familiar with all platforms, but don't most have a backup feed option.. For people wanting to create their own platform, it seems to make sense to have a backup feed/broker.
I believe it is essential (from robustness point of view) that your "live execution" code be identical to your backtest code - ideally, the code-path for live execution should be 100% identical to the code-path in backtest - of course, all of the error-handling that deals with issues you identified won't be exercised in "performance" backtest (but some of it might be testable in MarketReplay, for example automated rollover, strategy stop/restart).
I have grown an underlying infrastructure that minimize the strategy-specific handling of those "live" error situations, it has become pretty big (and complex), but most of that complexity is encapsulated, distinct from the trading strategies themselves, and reused from strategy to strategy (using C# object-oriented features to that effect).
I prototype a lot of ideas outside of that infrastructure, but as soon as something shows enough potential, I migrate that prototype to my infrastructure, and from that point onwards backtest & live execution code are one & the same.