Machine Learning Algo for Trading

[jcl366]

This seems to be the typical way of discussions on this forum: Someone utters an opinion, someone else disagrees, and the next argument is "you have no idea what you are talking about". I have indeed no idea where and what you have heard about machine learning. But I have developed more than 40 ML systems so far for financial prediction, with all sorts of algorithms, since this is my job.

Your declaration that the same financial data exists for decades and is used to train ML systems is a trivial statement, since of course all trained features are obviously derived from financial data. But how you derive the features, and which algorithm you use to preselect them for training is essential for success. Lots of papers have been published about this very problem. Feeding raw price to a ML algorithm, no matter which one, and hoping for profit simply does not work. If you don't believe me, just try it. It is not very difficult. Here's some links to recent blog articles that may give you a basic introduction:

http://www.financial-hacker.com/build-better-strategies-part-4-machine-learning/

http://robotwealth.com/machine-learning-financial-prediction-david-aronson/

A list of recent papers:

http://gregharris.info/a-survey-of-deep-learning-techniques-applied-to-trading/

 

[userque]

...But I have developed more than 40 ML systems so far for financial predictions, since this is my job. ...

How well do they perform?

 

[gkishot]

Not sure why that is not apparent. Take the GO competition between human and computer. The input data are merely used by the algorithm (=engine) to train the network. How it trains it and how it used the trained network to make prediction are the real secret sauce. Certainly not the training data.

Same with an image recognition network. The input data are just labeled image data. Everything else, the building of associations and the computed weights to later on recognize images is all accomplished by the actual algorithm.

Same with an algorithm that peruses time series or other financial input data...

Humans can do image recognition without a computer's aid. Can you do market predictions without a computer?

 

[jcl366]

How well do they perform?

60% failed, 40% work.

 

[userque]

60% failed, 40% work.

Great!
Of the ones that worked, about what range of annualized returns did they fall in?

 

[jcl366]

Great!
Of the ones that worked, about what range of annualized returns did they fall in?

In the medium two-digit range. But with many disclaimers. We program the systems for clients, so I can not provide details. Also we're doing this only for three years, and only since 2015 we got systems that really work. They're running live not yet long enough for determining solid performance data. And not all systems are for trading - some predict something else, like trader behavior prior to a market crash.

Generally, systems based on some financial model seem to have still better performance than machine learning systems.

 

[conduit]

Mate, nobody talked feeding couple price series into a network and trying to predict the next price move. Careful data cleansing, pre-election, mapping of labels to data series, and other techniques are important, so are they to feed back tests and all other statistical techniques.

But that does not make the input data more important or more complex than designing and deriving and fine tuning the heart, which is the actual network. I do indeed not know what kind of ML methodologies you apply but they must be laughably simplistic if you are claiming that your data pruning and extraction is the most complex part of the equation. Every other AI researcher would also vehemently disagree with you. I rest my case.

By the way all your referenced papers are retail crowd oriented hacks by wannabes, no single professional, self respecting hedge fund would ever derive any value out of such content. The last link of yours points to relevant papers but some of the techniques described there are quite dated and definitely not considered innovative nor the most computationally efficient anymore. For example support vector machines had their space but are superseded by a number better, more elegant, and efficient approaches.

I don't try to be an ass, I simply tell you how I see it. You can of course disagree and I respect that.

This seems to be the typical way of discussions on this forum: Someone utters an opinion, someone else disagrees, and the next argument is "you have no idea what you are talking about". I have indeed no idea where and what you have heard about machine learning. But I have developed more than 40 ML systems so far for financial prediction, with all sorts of algorithms, since this is my job.

Your declaration that the same financial data exists for decades and is used to train ML systems is a trivial statement, since of course all trained features are obviously derived from financial data. But how you derive the features, and which algorithm you use to preselect them for training is essential for success. Lots of papers have been published about this very problem. Feeding raw price to a ML algorithm, no matter which one, and hoping for profit simply does not work. If you don't believe me, just try it. It is not very difficult. Here's some links to recent blog articles that may give you a basic introduction:

http://www.financial-hacker.com/build-better-strategies-part-4-machine-learning/

http://robotwealth.com/machine-learning-financial-prediction-david-aronson/

A list of recent papers:

http://gregharris.info/a-survey-of-deep-learning-techniques-applied-to-trading/

 

[conduit]

What is your point, you completely change topic here. And you are dead wrong. Neural networks are so good at image recognition that worldwide all postal services fully employ computers to read address labels, printed or handwritten, at significantly lower error rates than a human ever could. And that is currently the most trivial example I can think of

Humans can do image recognition without a computer's aid. Can you do market predictions without a computer?

 

[conduit]

You design them for clients? Why would you do that if they performed so well and were robust. That already makes very little to no sense. It's like a trading mentor claiming he can trade. Why would he then not trade for himself?

Sorry that I come across as being harsh but I have worked in the front office and hedge fund industry more than a dozen years, and half of which in Quant prop trading. I never ever heard of a shop who would sell highly profitable strategies to clients. It's an oxymoron.

In the medium two-digit range. But with many disclaimers. We program the systems for clients, so I can not provide details. Also we're doing this only for three years, and only since 2015 we got systems that really work. They're running live not yet long enough for determining solid performance data. And not all systems are for trading - some predict something else, like trader behavior prior to a market crash.

Generally, systems based on some financial model seem to have still better performance than machine learning systems.

 

[syswizard]

Generally, systems based on some financial model seem to have still better performance than machine learning systems.

Good discussion...and I think the above just confirms that the data being given to the machine learning algorithms is not prepped properly. Just feeding it tick data is stupid IMHO.