Using machine learning to Improve systematic trading

[fan27]

Excellent research..thanks for sharing!

I am a newbie when it comes to ML. I am currently re-writing my backtest engine and here is what I am going to be able to do. Feedback welcomed.

Let's say I am interested in a strategy that buys an oversold stock or ETF and expects a bounce. For this strategy I know that I expect the stock to be below its 200 day ma. Let's assume I am testing daily data. Here are the features I am interested in testing.

[1DailyClose, 2DailyClose, 3DailyClose]
[1WeeklyClose, 2WeeklyClose, 3WeeklyClose]
[1MonthlyClose]
[1MonthIs12MonthLow, 2MonthIs12MonthLow]
[1WeeklyCloseRelatedToLowerBollingerBand, 2WeeklyCloseRelatedToLowerBollingerBand, 3WeeklyCloseRelatedToLowerBollingerBand]
[1MonthlyCloseRelatedToLowerBollingerBand, 2MonthlyCloseRelatedToLowerBollingerBand, 3MonthlyCloseRelatedToLowerBollingerBand]
[1MonthlyCloseIs12MonthLow, 2MonthlyCloseIsTwelveMonthLow]

Note that weekly and monthly data points always start with the previous week or month relative to the daily data point.

The software I am writing would take in all the feature arrays above and run all of the combinations. For example, let's say I start testing SPY and go to the first testable data point with my first feature combination 1DailyClose and 1WeeklyClose. If the data point was below its 200 day ma, then it would determine Close change value for the current day and the previous week, and would then proceed to test all of the other data points for that pattern. Some exit criteria here would be used (maybe put a stop and a limit in) just to see if the pattern can get a positive expectancy.

Once the software goes through all of the results for the different combinations, it can pick the top performing patterns and then test those patterns against other stocks. Assuming some viable patterns are identified, further tweaking could be made.

I am writing the software in GoLang so it is going to be fast. I suppose this could be considered a "poor mans" machine learning approach

fan27

 

[Simples]

Excellent research..thanks for sharing!
...
I am writing the software in GoLang so it is going to be fast. I suppose this could be considered a "poor mans" machine learning approach

ML is just having the computer making decisions based on past data, so is about automating the analysis / decision-stages using data, but with huge limitations compared to the ideas of "general purpose AI". Humans still need to prepare data, do feature-selection and maintaining process quality and performance. So if you let the runtime analyse historical data and decide based on that, it may be called ML (adaptive). If you use the results to hardcode the logic or as parameters, it's not so much ML (static). For me, the distinction is not so important and don't need ML for its own sake, but would be useful where there's better value versus costs.

1. For trading, the entire system contributes to total performance, so every bit of logic will be a crucial integral part of backtesting results. This means, a simple change in parts of the logic may drastically alter results. Ideally you want to avoid spikes, though trading results may quickly become non-linear, especially from a portfolio-level perspective downwards.

2. Golang is stable, fast and simple, with speed comparable to Java and sometimes approaching C++, while superior in other aspects. However, it's still very easy to come up with workloads that may take several hours, or, if testing all possible permutations on top of that again, days/weeks, even on EOD data. When you've shed the intepreters (again), it's still about the core: algorithms

I like the idea in the paper of using ML for special purposes, ie. QA and verification, although can't really say I understand the specific details of it.

 

[fan27]

ML is just having the computer making decisions based on past data, so is about automating the analysis / decision-stages using data, but with huge limitations compared to the ideas of "general purpose AI". Humans still need to prepare data, do feature-selection and maintaining process quality and performance. So if you let the runtime analyse historical data and decide based on that, it may be called ML (adaptive). If you use the results to hardcode the logic or as parameters, it's not so much ML (static). For me, the distinction is not so important and don't need ML for its own sake, but would be useful where there's better value versus costs.

1. For trading, the entire system contributes to total performance, so every bit of logic will be a crucial integral part of backtesting results. This means, a simple change in parts of the logic may drastically alter results. Ideally you want to avoid spikes, though trading results may quickly become non-linear, especially from a portfolio-level perspective downwards.

2. Golang is stable, fast and simple, with speed comparable to Java and sometimes approaching C++, while superior in other aspects. However, it's still very easy to come up with workloads that may take several hours, or, if testing all possible permutations on top of that again, days/weeks, even on EOD data. When you've shed the intepreters (again), it's still about the core: algorithms

I like the idea in the paper of using ML for special purposes, ie. QA and verification, although can't really say I understand the specific details of it.

Thanks for the post @Simples. The example I posted would require 8191 combinations. If each combination can be done in about 2 seconds on my laptop then that would be 5 hours. I will do some benchmarks on my laptop. I think my approach is valid as it can scale across multiple computers. With EOD data it is easy keep everything in memory.

fan27

 

[indicator777]

Try the Numerai competition with 21 features I believe. ML is overkill for these 10 features. You can use (0,1) weights to optimize. Q: why taking abs value of the MA difference? You lose the direction information. Then you say

"Finally, one must decide the parameters for the classification of the training data. Will only
profit generating trades be classified with output Y=1, or will trades that also contain other
desirable traits be classified as Y=1. For the purpose of this study we have chosen the latter,
with both profitable trades classified as Y=1 and trades with drawdown less than one third of
their run-up also classified as Y=1."

But what determines the value of class 1? The 5-day return of the trade? Or the 2-day return? This is not clear. Could you please clarify because in the training set this must be set in advance?

There is a large volume of papers by academics that have found that the profit potential is limited from commonly used features. There is at least one developer I know that uses ML to first extract features here and here. This is a lot more interesting.

Also useful to have a sense of data-mining bias. Retrying with the data and different features has potential of fooling even the best. You certainly have a good base to start from but the road will be bumpy. Good luck.

 

[Rogerahan]

Try the Numerai competition with 21 features I believe. ML is overkill for these 10 features. You can use (0,1) weights to optimize. Q: why taking abs value of the MA difference? You lose the direction information. Then you say

"Finally, one must decide the parameters for the classification of the training data. Will only
profit generating trades be classified with output Y=1, or will trades that also contain other
desirable traits be classified as Y=1. For the purpose of this study we have chosen the latter,
with both profitable trades classified as Y=1 and trades with drawdown less than one third of
their run-up also classified as Y=1."

But what determines the value of class 1? The 5-day return of the trade? Or the 2-day return? This is not clear. Could you please clarify because in the training set this must be set in advance?

There is a large volume of papers by academics that have found that the profit potential is limited from commonly used features. There is at least one developer I know that uses ML to first extract features here and here. This is a lot more interesting.

Also useful to have a sense of data-mining bias. Retrying with the data and different features has potential of fooling even the best. You certainly have a good base to start from but the road will be bumpy. Good luck.

Hi indicator77, thank you for the feedback, it was clear, concise and informative.

The main objective of this paper was to provide a balanced framework to improve systematic strategies using Machine Learning. The strategy that was chosen, as well as the features, were only chosen because they were simple and the logic would be easy to follow for the reader.

With that said, I believe you are right in saying that there are many different improvements that could be made during the feature selection process which would aid in the improvement of the risk/return profile of the selected strategy utilised.

For example, I used the absolute value of the MA difference since there was no differentiation in the features data between the long and short trades, yet I didn't use the absolute value for the close - 20 MA, or the percentage change since previous close. This could clearly be a useful change.

For the succinctness of the study, I also made no changes during the Cross Validation section of the paper. Clearly, this would be the place to make the appropriate changes to features and check for the improvement in results.

In terms of the determination of the class, I would say that this is a bit more nuanced. As a start I would say class only profitable trades as '1'. However you can alter this and check how this affects your Training and Cross Validation results (check which alteration would give you the highest profit or profit/drawdown). I personally thought trades that had a relatively high run-up (3 to 1 compared to draw-down) were likely to have been whipsawed by a factor like the news, and all things being equal would provide profitable trades in the future. But that was just my thinking behind it.

To tackle data-mining bias, the data had been split, and you really can only use your test set data once. The particular reason it has to be split is to make sure we aren't fooling ourselves in this regard. Since the test set data is 'unseen' it really will reveal whether we have over-fit the data.

I really want to emphasise that no alterations were done (or should be done) to the model once it was run on the test set. At that point it really has to be sink or swim.