I am also trying to code and backtest strategies from Bensdorp's second book at the moment. I was wondering if you have backtested those strategies yourself and if you have any comments regarding them specially the mean reversion ones. Are there any particular reasons why you would not be following the strategies in the book (apart from your strategies being better of course).
Fully Automated Stocks Trading
- Thread starter ValeryN
- Start date
- Status
I am also trying to code and backtest strategies from Bensdorp's second book at the moment. I was wondering if you have backtested those strategies yourself and if you have any comments regarding them specially the mean reversion ones. Are there any particular reasons why you would not be following the strategies in the book (apart from your strategies being better of course).
A friend of mine did. Neither of us spent much time on this so take anything here with a grain of salt including quality of the test results. It seems most of those strategies are achieving author's goal and are consistent with what he is saying in the book. This is highly unusual based on my experience.
Why I don't use them can be a long answer. To keep it short - performance of mine individual ones and combined is better by any comparison measure I'd use. Once you develop and start trading successfully your own strategy it takes a lot to consider others.
I think they are a great starting point for someone who doesn't have a working mechanical strategy.
PS. Those results do include IB commissions but no slippage. Which might not be unreasonable as author intended them to be using either with LMT orders or traded at the open/close. I ran this test on SP500 data free of survivorship biased and accounting for historical index rebalancing.
Val
I enjoyed Automated Stock Trading Systems...thanks again ValeryN. I am still struggling a bit with the long term TF short system (as the hedge) as results initially appear so poor but the potential benefit is slowly sinking in . The book made me realize I really need to add short systems to the mix (currently just long TF and long MR). Have you found any sig difference in your live results v back test results for the short MR systems please ? I am thinking about potential impact of availability of stocks to short. Thanks in advance
Certainly not every trade my system took or would take. As I believe that would have as much value as another cute kitten video.
Everything else - yes
I'm currently in mountains with a phone only. So will give a shorter answer.
Regarding trading a loosing strategy for hedging - don't trade anything you're not comfortable with. You really need to internalize a strategy before trading live. Normally when combining multiple strategies you are trying to reduce max DD and ideally gaining ARR. From a practical standpoint - for most 1 year periods, not some hypothetical decade. So if after running tests they show to you that's what happens - ask yourself if you're ready to own a result and no matter what. From the moment you run it live it's your strategy. There will be no support line available. If answer is yes - do it and focus on tracking model vs live, evaluate if model is still within your expected params every week or a month.
Regarding my own experience with shorting stocks. Short answer - over time you develop better assumptions and incorporate them in you system model, so eventually they will track pretty closely. I'm pretty happy with my MR short. IB in general is very good with borrow availability but there are brokers who specialize on that. One thing that can help a lot to increase your chances to borrow - exclude stocks under 10$ from your list and have minimum liquidity of at least 200k shares per day on average.
Be careful with shorting bio tech or avoid them all together. I think there are edges with them, but your positions will have to be super small.
IB, unlike many other brokers publishes their daily borrowing rates on public FTP so you can look at them or start capturing automatically. On average they are pretty reasonable and you can develop some general assumptions about them in a month or so.
Occasionally, you'll get into some nasty pump like NKLA, which I had at around 510%/annual borrowing rate when I got my piece. But for MR your hold time is pretty short and that's more of an exception than a rule.
Not such a short answer afterall.
I'll post my NKLA chart when I get back. That stock was a total madeness.
Val
Last edited:
Many thanks Val...I might think more about the hedge system then...appreciate tip re price and volume filters for shorts so will start system design using those minimums and slowly build up to gain experience needed to refine system. I have taken on board the recommendation about bio tech too. Cheers
I wish I would be able to post here a bit more often, but didn’t have much to put together something meaningful since the last post.
Trying to stay on course of writing about something others didn’t already talk about a great deal. Today’s topic will be Outcomes Distribution and Outliers.
There is quite a lot of material out there on Monte Carlo simulations and probability of ruin. While they do make sense sometimes, in my own research I noticed that before using them there are way more significant reasons for why system won't not work well in live trading.
One of them is a possibility of deviating results due to too many entries opportunities present at the same time. It won’t be a problem if a trader trades a single instrument as you’re either in or out. Long or short. Pretty simple. For many multi-instrument systems this might happen due to
Let’s say we’re trading a basket of stocks like NASDAQ 100 and using mean reversion long system with LMT order entry, which is very common. Every day we scan them for setups and might get 10-15 candidates for next day trading and then enter if price goes a bit lower next day. But few times a year volatility spikes and most stocks move in the same direction presenting you with 70 setups and 20 or more actually getting filled next day. For most people that would be way more exposure than they would be comfortable having so we limit it to let's say first 5.
Normally there would be some sort of ranking mechanism to handle that in a backtest. One example - prefer ones with largest distance from previous close. If we don’t have intraday data - no way to know which entries will happen first. Even with intraday data its’ resolution might not be enough to say for sure. But no matter how much data we got there is no way to know since you broker might have not accepted your order for whatever reason or your software went down with "out of memory” error and didn’t send it.
While it’s probably impractical to mitigate all possible reasons one thing we can do is to test system robustness in those cases and look at possible outcomes distributions and then try to mitigate the biggest impacting ones with lowest cost of mitigation.
This year's volatility gives plenty of opportunity to do that on recent data.
My friend came up with a randomized entry test which I love. What we would do is
It is pretty simple and yet extremely powerful. Could be very eye opening.
Before looking into an example here is few more ideas
He is an example of what I would consider an excellent MR Long system from 1 Feb - 1 May of 2020. Remember that SP500 market went down 35% and is still at a loss YTD.
Here is a backtest for one of the most volatile period in market history Feb 1 - May 1 2020:
Might not look like much for inexperienced traders but note that
Either way, looks pretty good to my taste.
But what happens is we run 100 random entries test using EOD data:
Trader A and trader B is mentioned for a reason… This was actually me and my friend who ran very similar version of a system. While some of this can be attributed to a very different execution algos we use - this particular system had pretty wide outcome distribution to begin with.
So where anyone trading it would end up for this period is also somewhat of a luck.
We went different ways about handling this. Which I think both were very viable.
While this time I was on a better side of distribution I decided that am not comfortable with those in a future, so I eventually came up with system version having more a more acceptable range, sometimes giving up a bit of ARR in a backtest. Also checked all other systems for same problem. That proved to be a very good decision so far.
Best predictor of future volatility is current volatility
So those volatility-related deviations have continued in old system version since I made the changes while my live results were very much inline with backtest or better. Which was luck. Or execution. Or both. There is always luck involved.
Takeaways
Val
Trying to stay on course of writing about something others didn’t already talk about a great deal. Today’s topic will be Outcomes Distribution and Outliers.
There is quite a lot of material out there on Monte Carlo simulations and probability of ruin. While they do make sense sometimes, in my own research I noticed that before using them there are way more significant reasons for why system won't not work well in live trading.
One of them is a possibility of deviating results due to too many entries opportunities present at the same time. It won’t be a problem if a trader trades a single instrument as you’re either in or out. Long or short. Pretty simple. For many multi-instrument systems this might happen due to
- Risk management
- Insufficient data resolution
- Data quality
- Very high volatility
- Limitations of your real-time data
- Broker screwing up your order
- You screwing up your order
- Broker outage
- Unable to borrow for shorting
- Unknown unknowns
Let’s say we’re trading a basket of stocks like NASDAQ 100 and using mean reversion long system with LMT order entry, which is very common. Every day we scan them for setups and might get 10-15 candidates for next day trading and then enter if price goes a bit lower next day. But few times a year volatility spikes and most stocks move in the same direction presenting you with 70 setups and 20 or more actually getting filled next day. For most people that would be way more exposure than they would be comfortable having so we limit it to let's say first 5.
Normally there would be some sort of ranking mechanism to handle that in a backtest. One example - prefer ones with largest distance from previous close. If we don’t have intraday data - no way to know which entries will happen first. Even with intraday data its’ resolution might not be enough to say for sure. But no matter how much data we got there is no way to know since you broker might have not accepted your order for whatever reason or your software went down with "out of memory” error and didn’t send it.
While it’s probably impractical to mitigate all possible reasons one thing we can do is to test system robustness in those cases and look at possible outcomes distributions and then try to mitigate the biggest impacting ones with lowest cost of mitigation.
This year's volatility gives plenty of opportunity to do that on recent data.
My friend came up with a randomized entry test which I love. What we would do is
- Look at peaks of potential uncertainty using backtesting software. Plot visually how many setups / fills next day did you have and find large spikes way above your risk parameters. Depending on a software you might need to change your strategy script to temporarily disable number of new/max positions you would take and make set margin available to something very high, so your software doesn’t get rid of trades due to insufficient capital
- Depending on a system you might get few spikes per year or more, or maybe once in 10 years. Modify your entry score / ranking to temporarily use random score. Limit your test dates to 1-2 months period around each of those spike and run a 100 iterations
- Then plot results distribution. Normally you will be looking at return % / drawdown % over the period
It is pretty simple and yet extremely powerful. Could be very eye opening.
Before looking into an example here is few more ideas
- To discover very rare outliers. Some people call left tail risk. Most times when developing a system people run them on “reasonable” number of new/max entries. A rule of thumb is to run not without limits but with ones more than you would be trading in real life. So its enough to see most bad trades. But that still might not show you very rare ones when there were 100 potential entries on one day and one of them lost 100% or maybe 600% in case it’s a short.
- To address this problem I’d run lost of iterations described above as well as few versions with very liberal limits, like 20 new entries, 50, unlimited and see what happened. Normally that happens during periods like 1987, 2008/2009, 2011. If you’re planning to run your systems long term it’s important to find as many as possible “black swans” like that. No guarantees something worse won’t happen but you’re be way more prepared than most professionals out there
He is an example of what I would consider an excellent MR Long system from 1 Feb - 1 May of 2020. Remember that SP500 market went down 35% and is still at a loss YTD.
Here is a backtest for one of the most volatile period in market history Feb 1 - May 1 2020:
Might not look like much for inexperienced traders but note that
- This is ~1:1 return to risk ratio with very small position size and no use of leverage. +4% while loosing ~5%
- One of the worst markets to be long
Either way, looks pretty good to my taste.
But what happens is we run 100 random entries test using EOD data:
Trader A and trader B is mentioned for a reason… This was actually me and my friend who ran very similar version of a system. While some of this can be attributed to a very different execution algos we use - this particular system had pretty wide outcome distribution to begin with.
So where anyone trading it would end up for this period is also somewhat of a luck.
We went different ways about handling this. Which I think both were very viable.
While this time I was on a better side of distribution I decided that am not comfortable with those in a future, so I eventually came up with system version having more a more acceptable range, sometimes giving up a bit of ARR in a backtest. Also checked all other systems for same problem. That proved to be a very good decision so far.
Best predictor of future volatility is current volatility
So those volatility-related deviations have continued in old system version since I made the changes while my live results were very much inline with backtest or better. Which was luck. Or execution. Or both. There is always luck involved.
Takeaways
- You probably wasted your time reading this if you trade a single instrument at any given time
- Consider adding outcomes distribution test into your toolbox and at least stress-testing all your strategies with it
- Consider running your backtest on a few very aggressive max daily/total positions settings including “unlimited"
- If your system is susceptible to a wide range of outcomes - consider changing it in a way that will reduce the range till it’s acceptable to you. Normally you can come up with more strict setup selection on a prior day / consider different ranking algo / increase your data resolution and do more testing to reduce uncertainty / take more but smaller positions
- Study your outlier trades. Especially big losers. Don’t expect you will ever get rid of them by tweaking a system. Most of the time this is just inevitable and it’s better to expect them regardless of what current backtest shows. Testing with unlimited number of entries and looking at outcomes distribution helps to discover hem
- If you don’t have really bad outliers in your backtest - there is probably something wrong with it of your backtest just got lucky. Can’t speak for all instruments but stocks gap down 50% overnight and do go to 0. If you have system with overnight hold and test over sufficient time period - you should see those.
- Consider removing best trades from your back test and see what happens
Val
Last edited:
This one gonna be about feelings, good bad and ugly trades, from a mechanical perspective.
The other day I felt like showing off a good trade here. And I did, in some other thread, NKLA. Even though my trading is automated there are still feelings.
Here it is. Behold!
Personally I believe my biggest achievement has been having very low rate of interference with my systems and always focusing on (a) perfect execution / trailing backtest as close as possible (b) no bugs / software reliability by design (c) staying with “are strategies within parameters for similar market conditions” mentality. So what I “feel” has a little “live” interference.
But emotional noise still exists for sure.
Normally I shield myself from looking at individual trades before they close and then over the weekend I will pull up charts and do some extra assessment to see if all is within parameters. That makes things less emotional. Occasionally I need to troubleshoot something intraday and will have TWS running locally and might peek at something. Then there are more “feelings".
If feels great to see a “perfect” trade. Like getting in at a high/low of a day, out at low/high, or a quick intraday profit target hit (I had some with 10 seconds entry/exit at ##% profit). And the opposite is true - some trades are just so bad that make you question everything or at least leave with a burning desire to filter them out somehow. Over time it gets easier as I see 50%-ers on both win/loss side in live trading, but I’m not sure this emotional component will ever go away completely.
This year’s results so far been pretty outstanding and I think it’s worth to show not so pretty part of it. One example - in early March longs were being absolutely destroyed and caused my cumulative systems DD of -14.86%. It took just few days to get there from near account’s all time high. Might not sound like too much but in $ its like - working a year for free in some occupations, or saving up and NOT buying that new car because you lost your briefcase with cash on a way to a dealership. Garbage way of thinking about it and yet it does creep in. Almost no reports my software generates include $ for that reason.
Some companies went from 30 to 5$ in a few days and probably disappeared from institutional radar once dipping below 10$ making things even worse. Crazy volatility, never seen this in my life trading, only in backtests like 1987, mid 70-ies, 2000-2001, 2008 + few flash crashes. Long before running live with a meaningful account size I did some mental exercises to simulate how it would feel when I eventually get into stock that loses 50% in a day or a short that doubles or triples and sized everything accordingly to reasonably preserve my well-being. What still caught me off-guard this year was how many of those I had at the same time. Emotionally it was difficult to see, though, systems were technically within parameters, but nothing like this happened in a very long time and DD for some were hovering around max historical level.
Here is how March 11/12 long positions were doing and a dent in equity curve they left. Individual positions looked really ugly with some have MAE of ~-70%. Shorts certainly cushioned the blow as well as diverse exits types for longs.
Val
The other day I felt like showing off a good trade here. And I did, in some other thread, NKLA. Even though my trading is automated there are still feelings.
Here it is. Behold!
Personally I believe my biggest achievement has been having very low rate of interference with my systems and always focusing on (a) perfect execution / trailing backtest as close as possible (b) no bugs / software reliability by design (c) staying with “are strategies within parameters for similar market conditions” mentality. So what I “feel” has a little “live” interference.
But emotional noise still exists for sure.
Normally I shield myself from looking at individual trades before they close and then over the weekend I will pull up charts and do some extra assessment to see if all is within parameters. That makes things less emotional. Occasionally I need to troubleshoot something intraday and will have TWS running locally and might peek at something. Then there are more “feelings".
If feels great to see a “perfect” trade. Like getting in at a high/low of a day, out at low/high, or a quick intraday profit target hit (I had some with 10 seconds entry/exit at ##% profit). And the opposite is true - some trades are just so bad that make you question everything or at least leave with a burning desire to filter them out somehow. Over time it gets easier as I see 50%-ers on both win/loss side in live trading, but I’m not sure this emotional component will ever go away completely.
This year’s results so far been pretty outstanding and I think it’s worth to show not so pretty part of it. One example - in early March longs were being absolutely destroyed and caused my cumulative systems DD of -14.86%. It took just few days to get there from near account’s all time high. Might not sound like too much but in $ its like - working a year for free in some occupations, or saving up and NOT buying that new car because you lost your briefcase with cash on a way to a dealership. Garbage way of thinking about it and yet it does creep in. Almost no reports my software generates include $ for that reason.
Some companies went from 30 to 5$ in a few days and probably disappeared from institutional radar once dipping below 10$ making things even worse. Crazy volatility, never seen this in my life trading, only in backtests like 1987, mid 70-ies, 2000-2001, 2008 + few flash crashes. Long before running live with a meaningful account size I did some mental exercises to simulate how it would feel when I eventually get into stock that loses 50% in a day or a short that doubles or triples and sized everything accordingly to reasonably preserve my well-being. What still caught me off-guard this year was how many of those I had at the same time. Emotionally it was difficult to see, though, systems were technically within parameters, but nothing like this happened in a very long time and DD for some were hovering around max historical level.
Here is how March 11/12 long positions were doing and a dent in equity curve they left. Individual positions looked really ugly with some have MAE of ~-70%. Shorts certainly cushioned the blow as well as diverse exits types for longs.
Val
Back in the day, after reading so many books on trading I felt like often they all follow the same general pattern which leads to predictable emotional cycle. Pattern being - author relates to you with a life story (you’re feeling great), share struggles (holy shit I’m not alone, this guy gets me!), show it can be done (wow! There is hope), get some glimpses into what performance can look like, and then either leave you with
Depending on the quality of a book, within a few days to couple of months I fell back to "I’m completely on own in this and all those guys are crooks realization”
It’s kinda funny, didn’t I just use the same trick to “connect" with readers of this journal? Well, at least I’m not selling anything.
When thinking what would be best topic for my next post I figured there only so much theoretical stuff people can take. So thinking about the struggles I had at the beginning I decided to come up with a tradable system example which, hopefully, can give some folks a breakthrough they need in searching for their edge, consistency and profitability. Or maybe enough hope to keep digging till it's found.
Before that a bit more on my own trading - what I do today is not practical or feasible for 10 mil account. Good news - this is not a problem most of us gonna have for awhile. And this is part of the edge.
Understanding this is critical. For the same reason I’m not sharing any exact rules of my strategies since they rely on pretty small inefficiencies and ability to execute nearly flawlessly. I also developed my own software to harvest them at the fullest and manage risk. My execution edge comparing to anyone else running my strategies is +8.5% annual over average number of trades per year. If it’s a volatile year like this one, just execution edge may become 15% of my profits.
Here is an MR Long strategy I’ve put together this morning. This is not something I personally use but I believe this a good start form someone having no working system and a small trading account. 100-200k USD would be just fine. Besides - lots of my stuff IS MR. Just different rules, short version and multiple versions with variations.
It’s based on Connors RSI. Basic rules are - take SP500 stocks above 5$ and with at least 0.5mil 10d average volume, find oversold ones using CRSI, leave only ones with a strong trend using ADX, rank by a creative formula, leave top 10 and send LMT orders for next day 10% below today’s closing prices. Exit with MOC order when CRSI > 50 or held for 10 days.
This system doesn’t generate too many trades and basically takes everything that is offered at a significant discount on volatile days.
In the graph below you can see few key equity curves. I’ve included SPY with dividends for comparison. Red line is one that a trade with a large account would likely experience trading this system (no compounding). Green - smaller trader having an advantage of getting his small positions filled with high probability using simple LMT orders. Blue or whatever color it is included just for example, since most backtests you see out there will be done assuming compounding.
For 2000-2010 I’ve excluded compounded version and it made too much money and makes everything else look indistinguishable. I’m personably not a big fan of testing with compounding, but I do benefit from it in a live trading.
Advantages of this system:
Disadvantages:
The system itself:
Val
- Smart ass philosophical remarks and “now search for your own edge"
- Signup up for my service and follow signals
- A system rules that never works after you test it
Depending on the quality of a book, within a few days to couple of months I fell back to "I’m completely on own in this and all those guys are crooks realization”
It’s kinda funny, didn’t I just use the same trick to “connect" with readers of this journal? Well, at least I’m not selling anything.
When thinking what would be best topic for my next post I figured there only so much theoretical stuff people can take. So thinking about the struggles I had at the beginning I decided to come up with a tradable system example which, hopefully, can give some folks a breakthrough they need in searching for their edge, consistency and profitability. Or maybe enough hope to keep digging till it's found.
Before that a bit more on my own trading - what I do today is not practical or feasible for 10 mil account. Good news - this is not a problem most of us gonna have for awhile. And this is part of the edge.
Understanding this is critical. For the same reason I’m not sharing any exact rules of my strategies since they rely on pretty small inefficiencies and ability to execute nearly flawlessly. I also developed my own software to harvest them at the fullest and manage risk. My execution edge comparing to anyone else running my strategies is +8.5% annual over average number of trades per year. If it’s a volatile year like this one, just execution edge may become 15% of my profits.
Here is an MR Long strategy I’ve put together this morning. This is not something I personally use but I believe this a good start form someone having no working system and a small trading account. 100-200k USD would be just fine. Besides - lots of my stuff IS MR. Just different rules, short version and multiple versions with variations.
It’s based on Connors RSI. Basic rules are - take SP500 stocks above 5$ and with at least 0.5mil 10d average volume, find oversold ones using CRSI, leave only ones with a strong trend using ADX, rank by a creative formula, leave top 10 and send LMT orders for next day 10% below today’s closing prices. Exit with MOC order when CRSI > 50 or held for 10 days.
This system doesn’t generate too many trades and basically takes everything that is offered at a significant discount on volatile days.
In the graph below you can see few key equity curves. I’ve included SPY with dividends for comparison. Red line is one that a trade with a large account would likely experience trading this system (no compounding). Green - smaller trader having an advantage of getting his small positions filled with high probability using simple LMT orders. Blue or whatever color it is included just for example, since most backtests you see out there will be done assuming compounding.
For 2000-2010 I’ve excluded compounded version and it made too much money and makes everything else look indistinguishable. I’m personably not a big fan of testing with compounding, but I do benefit from it in a live trading.
Advantages of this system:
- It is reasonably simple. Though the ones I use live are ever more simple
- Better than pretty much anything I've seen in books and was able to reproduce myself
- It has “safe” liquidity assumptions for small trades. There is lots of room to go for less liquid stocks if desired. The less liquid the more inefficiencies.
- It has a fixed number of outstanding LMT orders. Which makes it usable by even non mechanical traders and allows for a very safe execution
- Big guys can’t use it as slippage will kill the edge
- It is <10% invested on average and never uses margin. In SPY you would have 100% of your money locked full-time
- It does get destroyed during big market downturns. I’d very suspicious if any MR Long wouldn’t. Max DD certainly needs some work though.
Disadvantages:
- It has pretty big max DD during big market downturns. There are many ways to mitigate it. Increasing number or trades, adding short system, positive sizing, different setups ranking, taking less volatile stocks, etc.
- 2010-2014 performance is pretty flat. I’d look into it more
- Max DD duration is more than 1 year on a few occasions. Ideally should be less than a year
The system itself:
Val
Last edited:
- Status