Activate/Deactivate System?

[EricP]

Quote from mhashe:

You've been developing automated systems for quite a while now. Curious to know if any of your old systems from few years back still profitable in this market . Do you develop your own indicators or use what's currently in the public domain?

Some systems continue to be profitable, while others fade away. My main system in the late 1990's stopped working in early 2000. This was an arbitrage strategy that relied on speed, and some trading opponent became faster while his arbitrage machine that I could possibly make mine.

When applicable, I try to make my systems auto-adjusting to market conditions, however, to allow them to last as long as possibly. For example, any system that is affected by market volatility could be fitted with variable parameters than automatically adjust to the volatility of the market. As a result, smaller profit targets and stop losses could be taken when the market is less volatile, and these targets and stop points would automatically widen when the market begins to swing more violently.

-Eric

 

[EricP]

Quote from nitro:

I have tested many systems for twenty years now. In that time, every time I thought I should turn off a system intraday based on either discretion or some clever analysis I had done, it invariably more often than not the wrong decision. I agree that this may be more a statement about my thinking skills than about the feasibility of being able to come up with a good answer, but I doubt it.

Man, I am with you on that one. Seems the best way to turn around a system in a slump is to manually deactivate it. I have had this problem repeatedly over time, and have therefore found that it's best (for me) to make the activation/deactivation decision 100% mechanical. Certainly, every trader MUST have some process (even if it's discretionary) for turning OFF a system that is being traded. This is necessary to protect the trader against the continued trading of a system that is no longer viable and begins to consistently lose money. As with anything else in trading, you must develop an activation/deactivation style that you can be comfortable with. There is no right answer for everyone.

-Eric

 

[nitro]

Quote from metooxx:

Solving the question you are asking for over a year...

Are you saying you are trying to solve it, or that you have solved it ? :eek:

Either way, welcome back.

nitro

 

[akawesson]

In 2004, I changed systems 4 times. My results were good enough, but subsequent testing revealed that if I had used any one of those systems all year, I would have done considerably better!

To further look into this, I added a feature to my testing software that takes a list of systems and a time interval as inputs, and produces a "switch list" as the output. The switch list can be either "switch to what worked best in the prior period" or "switch to what worked worst". The switch list then becomes the input for a new test.

I then did lots of testing of this sort, using different periods and different systems. The results showed that, on average, it is best to change systems as infrequently as possible, and if you are going to change, it is better to switch to a system that was not one of the best in the most recent period.

 

[nitro]

I have done similar studies, but the problem is that I did not have voluminous data to do a "real" test.

How long was your testing period? Was it just 2004?

nitro

Quote from akawesson:

In 2004, I changed systems 4 times. My results were good enough, but subsequent testing revealed that if I had used any one of those systems all year, I would have done considerably better!

To further look into this, I added a feature to my testing software that takes a list of systems and a time interval as inputs, and produces a "switch list" as the output. The switch list can be either "switch to what worked best in the prior period" or "switch to what worked worst". The switch list then becomes the input for a new test.

I then did lots of testing of this sort, using different periods and different systems. The results showed that, on average, it is best to change systems as infrequently as possible, and if you are going to change, it is better to switch to a system that was not one of the best in the most recent period.

 

[akawesson]

What I'm using is daily bars for a large universe of stocks. My systems tend to generate about 1000 trades/year. I do most of my testing using the most recent 3 or 4 years, but sometimes go back 8 or even 12 years.

If I run a really long term system comparison test (say 10 years, 10000 trades), all the systems do about the same, even though in a given year, one might do twice as well as another.

All this points to just picking a system and stiking with it, but I still find this difficult when testing shows dramatically better recent results for a different system. So, I continue to search for a system switch algorithm that beats the long-term average.

 

[nitro]

Ah,

You are using daily data - you are in a totally different place than I am then.

nitro

Quote from akawesson:

What I'm using is daily bars for a large universe of stocks. My systems tend to generate about 1000 trades/year. I do most of my testing using the most recent 3 or 4 years, but sometimes go back 8 or even 12 years.

If I run a really long term system comparison test (say 10 years, 10000 trades), all the systems do about the same, even though in a given year, one might do twice as well as another.

All this points to just picking a system and stiking with it, but I still find this difficult when testing shows dramatically better recent results for a different system. So, I continue to search for a system switch algorithm that beats the long-term average.

 

[mc107]

Quote from EricP:

I'm not sure I understand your question. First of all, if the movement of the market were fully random, then trying to make money by trading the market would be fruitless. Therefore, in order to be trading at all, you must believe that the movement of the market is predictable to some small extent.

Would it be best to deactivate a system if it's confidence level of profitability falls below 99%? This might be nice for some traders. I suspect most traders are like me in that it is difficult for a system to achieve a 99% confidence level. I do have some systems that meet this criteria, but many do not. By restricting my real trading to only systems with 99%+ confidence levels, I might get a smoother equity curve, but would definitely make less money. Instead, I choose to also trade systems with a 90-99% confidence level and smooth out the equity curve by greater diversification among different systems.

On a side point, factors affecting confidence level include the number of trades in the sample (the more the better), the average profit per trade in the sample (the higher the better) and the standard deviation of the profits in the sample (the lower the better). Typically, most systems that I trade do not have sufficient sample size (I limit to only looking at the last 120 trades) or profitability/variability to reach 99%+.

Best of luck,
-Eric

You misunderstanded my question. I mean that you deactivate your system if its confidence level of profitability is ABOVE 99%.

Thanks,
Jim

 

[EricP]

Quote from mc107:

You misunderstanded my question. I mean that you deactivate your system if its confidence level of profitability is ABOVE 99%.

Thanks,
Jim

I'm not sure I understand what you are trying to say. Assuming that their is a statistical confidence level of 99%+ that as system is profitable... => Why in the world would I want to deactivate it from real trading? I wish every single system that I trade had that high of a confidence level. Obviously, I'm using the assumption that my testing was valid and that the results are not "too good to be true" due to an error on my part.

Assuming that there are no significant errors in my testing (which better be the case, or else reviewing the statistics is worthless in the first place), then I would certainly NOT deactivate such a system.

I guess I'm not sure of your point. Under what circumstances would you consider deactivating the systems that appear to be the best ones you have?

-Eric

 

[metooxx]

Quote from nitro:

Are you saying you are trying to solve it, or that you have solved it ? :eek:

Either way, welcome back.

nitro

almost ...