Activate/Deactivate System?

E

EricP

When reseaching a trading system, at some point you must make a decision about whether your new pet system is 'worthy' of activation. Similarly, once activated most trading systems will, at some point, go through one or more periods of poor performance and you must determine whether this is just an inevitable minor drawdown, or whether this system is now 'broken' and must be deactivated to preserve your capital.

Over the years, I have developed some methods that have worked very well for me, addressing these issues. In the spirit of education, and in memory of acrary's useful posts on this board, I will share my methods for managing system activation/deactivation.

As with acrary, I will no reply to every post if I have nothing to say. In addition, I trade fulltime every day (although automated) and may or may not have time to respond to questions. Please avoid petty bickering on this thread, as it just ruins the thread for everyone.

I'll post some of my initial thoughts on this topic as the day goes on, and after I've posted 3-5 posts would welcome questions if there are any.

-Eric
 
First, before I forget, let me address one of the biggest mistakes people make when developing a system: optimization.

Typical traders will take a program like TradeStation or WealthLab or the equivalent, and create a spectacular trading system with clever and creative ideas, which has a near-linear P&L curve, and with very few minor drawdowns... They are excited and can't wait to start making the 'easy money' with live trading.

Will they actually make money with their new methods? Actually, we don't have a clue, as they've never tested whether their new method will make money in the future on not. All they have actually done is prove that their new system does any excellent job of being optimized to fit the past data, which in and of itself, is worthless.

How do we improve upon this? Test your system in the future! Hmmm... the future, you ask, how would we do that?

Let's assume that you are doing your backtesting on 5 years of data, and the system generates 150 trades per year, for a total of 750 trades. In order to tell how well the system performs in the future, you must withhold all of your data from the optimization tests. Instead, for example, only optimize on the first two years of data (300 trades, pretty decent number). Then, use your optimized system to now test on the 'future' => i.e. test year 3 of data with the system optimized on years 1 and 2.

Next, reoptimize your system on the data from years 2 and 3, and give it a future test with the data from year 4. Similarly, optimize your system with the data from year 3 and 4, giving it a 'real-time' test against the data from year 5. In this way, you can see how your system will perform in the 'future' based upon whatever optimization methods you are using.

This sort of method should give you increased confidence when you go to live activation that it should work well in the 'future'.

-Eric
 
activating/Deactivating any system is a system on its own. Example:
Switching between a trending system (MA, LinReg, MACD, etc) and counter trent system (RSI, Bol. bends, Chalkin etc.) is a Nobel Prize challenge.
 
For me, the question of system activation really depends on answering the question of "Is this system profitable?"

Obviously, once activated, money management can be used to determine the best trade size in order to minimize drawdown on your overall account.

So, how do we know if our system is 'profitable'? Obviously, the average P&L of the trades must be above zero. But, that alone should not give us confidence. For example, the following sequence of trades has an average above zero:

+$2000
+$350
($300)
+$1400
($3000)
+$750

Avg = +$200

=> But I certainly would have little confidence that the system was net profitable. The next trade could be ($2200) and then we would be seeing a net losing system.

We would get better confidence that we have a profitable system if the trades were more consistent (i.e. lower standard deviation). For example, the following six trades also average +$200, but provide greater confidence that the system is likely profitable over a long term:

+$500
($300)
+$400
($50)
+$950
($100)

Avg = +$200

While not ideal, the smaller spread of the P&L results increases our confidence that this system MAY be profitable over the long haul.

How else might we generate increased confidence of a systems long term performance? One key is certainly the number of trades that are in your sample being assessed. Assuming you have got 100 trades that show an average profit of $200 per trade, versus on 6 trades showing an average profit of $200 per trade => you will have much greater confidence in your system based upon the 100 trade analysis.

Fortunately, statistics offers us some tools that we can use to scientifically answer these questions a little more accurately.

What is the likelihood that my system is profitable? What is my confidence level?

(continued)

-Eric
 
Quote from abogdan:

activating/Deactivating any system is a system on its own. Example:
Switching between a trending system (MA, LinReg, MACD, etc) and counter trent system (RSI, Bol. bends, Chalkin etc.) is a Nobel Prize challenge.
More...

I completely agree, abogdan. That is why (for me), it is critical to have a structured approach in place that is completely free of any emotion or judgment. Instead, my systems automatically turn themselves on and off as appropriate based upon my system activation/deactivation 'system'. A system within itself, as you say. Very true.

Welcome to the thread, by the way. I have long valued your contributions on ET, and felt that thoughts from you and acrary have been the most helpful for me as any on ET. I am hopeful that you will find something useful in my posts as well.

-Eric
 
Is my system profitable? At what level of confidence?

Assuming we are using 'future' data (whether this is backtested future data or actual forward tested data doesn't matter), then we can use statistical analysis to our data to determine the likelihood that our system is a profitable system over the long run as follows:

x = abs(Avg-Profit) * (Number of Trades)^0.5 / (Std Dev of Profits)

Once "x" is calculated, you can look up the confidence level on the chart below to see the likelihood that this system will be profitable over the long haul:

x ------------------ Confidence Level

0 ------------------ 50%
0.075 ------------ 53%
0.126 ------------ 55%
0.25 -------------- 60%
0.36 -------------- 64%
0.52 -------------- 70%
0.67 -------------- 75%
0.84 -------------- 80%
1.04 -------------- 85%
1.28 -------------- 90%
1.4 ---------------- 92%
1.645 ------------- 95%
2.05 --------------- 98%
3.0 ----------------- 99%
4.0 ----------------- 99.5%
5.0 ----------------- 99.7%

Note that if the average profit is negative, then this will give you the confidence level that this is a losing system, over the long run.

One quick example to illustrate this equation. Assume that we have 50 trades, yielding an average profit of $40 per trade, with a standard deviation of $150 per trade. Using the equation and table:

x = abs(Avg-Profit)*(Number of Trades)^0.5 / (Std Dev of Profits)
x = abs(40)*(50)^0.5/150
x = 1.88, from table, Confidence Level ~97%

-Eric
 
Let me point out that these calculations were designed for series of data that have a Gaussian distribution (i.e. 'bell curve' distribution).

Is this true for our typical stock trade P&L's? Heck no, not even close!! 'Fat tails' from the occasional large stop loss, for example, will make our P&L distribution non-Gaussian.

Therefore, recognize that these equations and methods, while providing a good structured way to analyze your data, will not yield perfect results. The actual system may NOT be 97% likely to be profitable in the future, maybe it's only 93%... So what, big deal. The point is that a value of 97% is better than getting a value of 84%, and worse than getting a value of 99.6%. Greater 'confidence level' results should increase your confidence that you have a good system, while not necessarily being a statistically perfect answer for the true probability of future results. But, we are traders here, not statisticians. I don't mind if my predictions are off a few points, so long as I can make consistent money.

-Eric
 
As abogdan pointed out, determining when to activate and deactivate a trading system really becomes a system within itself.

My choice has been to develop a very simple method that will shutdown a trading system when I 'lose confidence' in it (statistically), and activate a trading system when it proves itself to be worthy (statistically). I exclusively use the equation and table shown in the above posts for making these decisions.

Anyone can arbitrarily decide their activation/deactivation levels, as well as the number of trades to be used for calculating their confidence levels for a system. For me, I have arbitrarily decided upon using the last 120 trades and I deactivate a system if my confidence level drops below 90%, and I activate a system if my confidence level rises above 93%. Note that I will use as few as 50 trades for these calculation if that is all the data that I have.

Somewhat arbitrary? Yes. Statistically sort of invalid (non-Gaussian)? Sure. But, it provides a solid structure for making the activation/deactivation decisions in a consistent and reasonable manner. And, most importantly, it works for me.

I hope that this has been helpful for a few folks, and this is all that comes to mind on the topic at this time. Questions or comments are welcome, and I'll post additional thoughts if they come to me.

-Eric
 
You must log in or register to reply here.
Back
Top