Can a retail trader succeed in algorithmic trading? (Kevin Davey vs Ernest Chan)
- Thread starter edward22245
- Start date
Hello fellow traders,
Kevin Davey is one of my favorite traders, authors, coaches in this industry. Kevin, through his books, courses, and seminars, has made algorithmic trading available to any "mortal," non-heavy-duty mathematician.
From the other side, I believe that Dr. Ernie Chan is one of the "sexiest" hardcore scientists of the algorithmic trading business, but for people who can follow his mathematical approach and have extensive knowledge of applications like MATLAB, and computer languages like Python and R.
For me, the discussion here shouldn't be about who is right or wrong about his approach. And there's not such a thing as "Ernie VS Kevin." Both of them are prominent members of the algorithmic community, both of them by their books, courses, and seminars have contributed to the most so this "forbidden" -for most of us retail traders- algorithmic approach of trading the financial markets.
Now, the main question here for us should go directly towards ourselves: "Which of these two successful individuals CAN I follow and try to imitate (in a healthy way) so to have success also? Now, having said the above and according to my humble opinion, Kevin's approach could be much more easily "digested" by a retail trader.
And this is not because it is oversimplistic, but because he managed to absorb all the pitfalls of a hyper analytic mathematical approach by himself, allowing the reader or the student, to get to the core of what algorithmic trading is and how someone can use it.
From the other side, Dr. Chan's approach embodies the beauty of a strict and analytic quantitative method of analysis-synthesis. I cannot deny the fact that also Dr. Chan is trying to not go so far into the territory of Ph.D. mathematics, yet at the end of the day, he is who he is: a "shrewd" scientist.
Concluding my first (and long-I am sorry for that) post here in these forums, I have to end up with a suggestion: Whom of these two gentlemen someone will follow, depends only on how feels that he could have a hands-on approach to algorithmic trading.
Happy Trading to All!
Kevin Davey is one of my favorite traders, authors, coaches in this industry. Kevin, through his books, courses, and seminars, has made algorithmic trading available to any "mortal," non-heavy-duty mathematician.
From the other side, I believe that Dr. Ernie Chan is one of the "sexiest" hardcore scientists of the algorithmic trading business, but for people who can follow his mathematical approach and have extensive knowledge of applications like MATLAB, and computer languages like Python and R.
For me, the discussion here shouldn't be about who is right or wrong about his approach. And there's not such a thing as "Ernie VS Kevin." Both of them are prominent members of the algorithmic community, both of them by their books, courses, and seminars have contributed to the most so this "forbidden" -for most of us retail traders- algorithmic approach of trading the financial markets.
Now, the main question here for us should go directly towards ourselves: "Which of these two successful individuals CAN I follow and try to imitate (in a healthy way) so to have success also? Now, having said the above and according to my humble opinion, Kevin's approach could be much more easily "digested" by a retail trader.
And this is not because it is oversimplistic, but because he managed to absorb all the pitfalls of a hyper analytic mathematical approach by himself, allowing the reader or the student, to get to the core of what algorithmic trading is and how someone can use it.
From the other side, Dr. Chan's approach embodies the beauty of a strict and analytic quantitative method of analysis-synthesis. I cannot deny the fact that also Dr. Chan is trying to not go so far into the territory of Ph.D. mathematics, yet at the end of the day, he is who he is: a "shrewd" scientist.
Concluding my first (and long-I am sorry for that) post here in these forums, I have to end up with a suggestion: Whom of these two gentlemen someone will follow, depends only on how feels that he could have a hands-on approach to algorithmic trading.
Happy Trading to All!
I received below email from Robot Wealth. I thought maybe of interest for some people here. Sounds comprehensive, but I wouldn't know.
We’re excited to announce that our Trading with Machine Learning and Big Data Bootcamp launches on Monday November 11th!
By the end of this 6-week Bootcamp you’ll know:
The success of any research and trading strategy design project comes down to how well you can frame the question you are asking
You'll learn how to do that well. And you'll learn how to use machine learning techniques to make answering that question quicker and easier.
You’ll watch us develop tools and solve problems that we rely on in our own trading book, so you'll know how these hands-on techniques work on a highly practical level.
This bootcamp is not academic. We're solving real trading problems.
First, we're going to look at equity pairs trading. The basic pairs trading algorithm is well known - but how do we decide which of millions of possible pairs we should pick? We'll show you how we use big data processing and unsupervised learning techniques to answer that question.
Next, we'll look at how we can use supervised learning techniques, together with features from the equity options market, to improve an existing alpha trading strategy. This is called "metalabelling" and it's a great skill to have in your arsenal. You can apply it anywhere!
We'll cover a lot of ground together in 6 weeks, more than most solo traders would cover in years of going at it alone.
I've shared the syllabus with you below.
Happy trading,
-Kris, James and Michael
Week 1 - Introduction
We’re excited to announce that our Trading with Machine Learning and Big Data Bootcamp launches on Monday November 11th!
By the end of this 6-week Bootcamp you’ll know:
- How to apply machine learning tools and techniques in a practical way to solve real trading problems, without wasting resources on things that don’t work
- How to prepare and analyse enormous amounts of "big data"
- What research and trading in a professional operation really looks like, for you to apply to your personal trading
- How to deploy tried and tested strategy frameworks and a solid, quick, pragmatic research approach
- How to apply your new strategies, toolkit and approach diversely across a range of financial products and strategies
The success of any research and trading strategy design project comes down to how well you can frame the question you are asking
You'll learn how to do that well. And you'll learn how to use machine learning techniques to make answering that question quicker and easier.
You’ll watch us develop tools and solve problems that we rely on in our own trading book, so you'll know how these hands-on techniques work on a highly practical level.
This bootcamp is not academic. We're solving real trading problems.
First, we're going to look at equity pairs trading. The basic pairs trading algorithm is well known - but how do we decide which of millions of possible pairs we should pick? We'll show you how we use big data processing and unsupervised learning techniques to answer that question.
Next, we'll look at how we can use supervised learning techniques, together with features from the equity options market, to improve an existing alpha trading strategy. This is called "metalabelling" and it's a great skill to have in your arsenal. You can apply it anywhere!
We'll cover a lot of ground together in 6 weeks, more than most solo traders would cover in years of going at it alone.
I've shared the syllabus with you below.
Happy trading,
-Kris, James and Michael
Week 1 - Introduction
- Initiation
- Why classic data science doesn't apply to the markets
- The basics of Unsupervised and Supervised Learning, Tools and Applications
- What we're going to cover in the Bootcamp
- Logistics (Using Slack, Webinars, Resilio Sync, R)
- Starting off on the right foot: How to Frame a Trading Problem
- Framing our pair trading problem
- A pairs trading algorithm and backtest
- Using the simulated random data to understand the dynamics of a pairs trade
- Market noise and the limitations of statistical tests under market conditions
- Market noise and the effectiveness of simple heuristics
- Reviewing the pair trading literature for direction in our research
- What does a real pairs trading portfolio look like?
- Researching and building a filtering and classification model for pair selection - Creating a plan
- A simple proof of concept: clustering on the correlation matrix on a broad equity universe.
- How are we going to know that our classification model has done something useful?
- The feasibility of running millions of event-based backtests
- Simple heuristic backtests and pairwise time-series features on equity spreads
- Using distributed cloud processing to calculate backtest results and features over an enormous data set
- Is there persistence in pair trading returns? Using classical factor analysis to investigate
- Using regression analysis to uncover pairs of stocks with known common economic exposures
- Uncovering latent economic exposures using Principal Component Analysis on cross-sectional stock returns
- Clustering on PCA to make sense of the data
- Tuning and stability analysis on the clustering model, and visualisation using t-SNE and DBSCAN algorithms.
- Mining for conditional dependencies using the sparse inverse covariance matrix and graphical lasso
- Tuning the regularisation parameter in graphical lasso.
- Using the fundamental similarity of stocks to filter for pairs to trade
- Using classical factor analysis to analyse fundamental similarity as a driver of convergence trading returns
- Feature selection techniques
- Incorporating fundamental similarity data in our clustering model
- Tuning the model
- Validating and tweaking the model using our backtest "big data" sets
- Putting together a 3D graphical model to make a huge potential dataset tractable
- Analysis of time-series features as a driver of convergence trading returns
- Putting together a multi-stage statistical arbitrage workflow, including:
- Universe filtering using our machine learning classification model
- Sorting and filtering on time-series filters
- Stability, robustness analysis and manual discretionary
- High- frequency meta-labelling to skip trades under certain conditions.
- Watch us run through the process and put together a portfolio and backtest it.
- An introduction to supervised learning
- Why predicting returns or return signs is very hard
- How to avoid the biases that trip you up using time-series cross sectional
- Clearly define your objectives and trade-offs using cost and hyperparameter objective functions applicable to trading
- How going LARGE on machine learning can fool you, even when you use out-of-sample testing.
- Picking the easy games: Things that are much more predictable than asset returns
- Other tools that you should turn to before you turn to machine learning techniques
- How to use metalabelling to improve an existing trading strategy
- Applying a metalabelling model to a long-short equity strategy
- Using liquidity features and features from the options market in our model
- Putting the "metalabelling" model together
- Cross-validation techniques
- Using metalabelling to cherry-pick the best trades
- Using metalabelling to determine bet sizing
- How metalabelling can rescue a strategy on the edge, or improve a good one
- What we found when we applied our features in a metalabelling framework to the long-short equity strategy
- How you can apply the metalabelling framework to other strategies
- Lessons learned and next steps.
- All data is provided. You'll receive a large amount of price data, fundamental data and liquidity, and options-derived features for over 1,500 US stocks. You'll receive sophisticated time-series features and analysis for millions of US stock pair combinations.
- Access to all RW courses during the Bootcamp
- Our complete Risk Premia Harvesting Strategy (and associated research and webinar recordings) are provided. This is a strategy that has realised a Sharpe Ratio of 3.8 in 11 months of live trading.
- A Gold Seasonality Strategy
- Example analysis of a discretionary macro trade
I received the below email from Robot Wealth. I thought maybe of interest to some people here. Sounds comprehensive, but I wouldn't know.
We’re excited to announce that our Trading with Machine Learning and Big Data Bootcamp launches on Monday November 11th!
By the end of this 6-week Bootcamp you’ll know:
And we’ll do all that step-by-step as part of team who will help you by sharing the workload, as well us their ideas, feedback and experience.
- How to apply machine learning tools and techniques in a practical way to solve real trading problems, without wasting resources on things that don’t work
- How to prepare and analyse enormous amounts of "big data"
- What research and trading in a professional operation really looks like, for you to apply to your personal trading
- How to deploy tried and tested strategy frameworks and a solid, quick, pragmatic research approach
- How to apply your new strategies, toolkit and approach diversely across a range of financial products and strategies
The success of any research and trading strategy design project comes down to how well you can frame the question you are asking
You'll learn how to do that well. And you'll learn how to use machine learning techniques to make answering that question quicker and easier.
You’ll watch us develop tools and solve problems that we rely on in our own trading book, so you'll know how these hands-on techniques work on a highly practical level.
This bootcamp is not academic. We're solving real trading problems.
First, we're going to look at equity pairs trading. The basic pairs trading algorithm is well known - but how do we decide which of millions of possible pairs we should pick? We'll show you how we use big data processing and unsupervised learning techniques to answer that question.
Next, we'll look at how we can use supervised learning techniques, together with features from the equity options market, to improve an existing alpha trading strategy. This is called "metalabelling" and it's a great skill to have in your arsenal. You can apply it anywhere!
We'll cover a lot of ground together in 6 weeks, more than most solo traders would cover in years of going at it alone.
I've shared the syllabus with you below.
Happy trading,
-Kris, James and Michael
Week 1 - Introduction
Week 2 - Big Data Processing and Unsupervised Learning for Equity Pairs Selection
- Initiation
- Why classic data science doesn't apply to the markets
- The basics of Unsupervised and Supervised Learning, Tools and Applications
- What we're going to cover in the Bootcamp
- Logistics (Using Slack, Webinars, Resilio Sync, R)
- Starting off on the right foot: How to Frame a Trading Problem
- Framing our pair trading problem
- A pairs trading algorithm and backtest
- Using the simulated random data to understand the dynamics of a pairs trade
- Market noise and the limitations of statistical tests under market conditions
- Market noise and the effectiveness of simple heuristics
- Reviewing the pair trading literature for direction in our research
- What does a real pairs trading portfolio look like?
- Researching and building a filtering and classification model for pair selection - Creating a plan
- A simple proof of concept: clustering on the correlation matrix on a broad equity universe.
Week 3 - Introducing Fundamental Features and Validating the Model
- How are we going to know that our classification model has done something useful?
- The feasibility of running millions of event-based backtests
- Simple heuristic backtests and pairwise time-series features on equity spreads
- Using distributed cloud processing to calculate backtest results and features over an enormous data set
- Is there persistence in pair trading returns? Using classical factor analysis to investigate
- Using regression analysis to uncover pairs of stocks with known common economic exposures
- Uncovering latent economic exposures using Principal Component Analysis on cross-sectional stock returns
- Clustering on PCA to make sense of the data
- Tuning and stability analysis on the clustering model, and visualisation using t-SNE and DBSCAN algorithms.
- Mining for conditional dependencies using the sparse inverse covariance matrix and graphical lasso
- Tuning the regularisation parameter in graphical lasso.
Week 4 - Graphical Modelling and Building a Statistical Arbitrage Workflow
- Using the fundamental similarity of stocks to filter for pairs to trade
- Using classical factor analysis to analyse fundamental similarity as a driver of convergence trading returns
- Feature selection techniques
- Incorporating fundamental similarity data in our clustering model
- Tuning the model
- Validating and tweaking the model using our backtest "big data" sets
Week 5 - Supervised Learning in a Long/Short Equity Trading Strategy
- Putting together a 3D graphical model to make a huge potential dataset tractable
- Analysis of time-series features as a driver of convergence trading returns
- Putting together a multi-stage statistical arbitrage workflow, including:
- Universe filtering using our machine learning classification model
- Sorting and filtering on time-series filters
- Stability, robustness analysis and manual discretionary
- High- frequency meta-labelling to skip trades under certain conditions.
- Watch us run through the process and put together a portfolio and backtest it.
Week 6 - Metalabelling Models
- An introduction to supervised learning
- Why predicting returns or return signs is very hard
- How to avoid the biases that trip you up using time-series cross sectional
- Clearly define your objectives and trade-offs using cost and hyperparameter objective functions applicable to trading
- How going LARGE on machine learning can fool you, even when you use out-of-sample testing.
- Picking the easy games: Things that are much more predictable than asset returns
- Other tools that you should turn to before you turn to machine learning techniques
- How to use metalabelling to improve an existing trading strategy
- Applying a metalabelling model to a long-short equity strategy
- Using liquidity features and features from the options market in our model
Bonuses
- Putting the "metalabelling" model together
- Cross-validation techniques
- Using metalabelling to cherry-pick the best trades
- Using metalabelling to determine bet sizing
- How metalabelling can rescue a strategy on the edge, or improve a good one
- What we found when we applied our features in a metalabelling framework to the long-short equity strategy
- How you can apply the metalabelling framework to other strategies
- Lessons learned and next steps.
See you on Monday!
- All data is provided. You'll receive a large amount of price data, fundamental data and liquidity, and options-derived features for over 1,500 US stocks. You'll receive sophisticated time-series features and analysis for millions of US stock pair combinations.
- Access to all RW courses during the Bootcamp
- Our complete Risk Premia Harvesting Strategy (and associated research and webinar recordings) are provided. This is a strategy that has realised a Sharpe Ratio of 3.8 in 11 months of live trading.
- A Gold Seasonality Strategy
- Example analysis of a discretionary macro trade
Well...that was precisely what I meant when I wrote that "...Kevin's approach could be much more easily "digested" by a retail trader. And this is not because it is oversimplistic, but because he managed to absorb all the pitfalls of a hyper analytic mathematical approach by himself, allowing the reader or the student, to get to the core of what algorithmic trading is and how someone can use it".
To avoid any possible misunderstandings: There's nothing wrong with the syllabus of the Bootcamp, above. On the contrary, it looks really exciting! However, I do not believe that a successful trader is like a rocket scientist. Or maybe the most successful ones perhaps are rocket scientists, yet my "hunch" is that there are much more "easy" approaches for a retail trader to construct a bunch of robust algorithmic trading systems. And to make a portfolio of them.
Global Macro Long term strategies can offer 10% per year. UHNWI will be extremely happy having this. From the other side in the 20 years, I am in the markets, I've seen many "global macro" strategies to perform lousily (to say the least) and make big banks extremely happy and their UHNWI clients hugely disappointed.
Terminology does not bring profits, buying and selling right, does. But that's just my 50cents word.
If I were to take the approach similar to Kevin's (as oppose to Ernie Chan's) how much mathematics would I need to know to become successful in algorithmic trading?
My suggestion for you to decide:
1)Go to youtube.com and search for both gentlemen. Watch a couple of videos with them. Then pick one of their books. "You need to spend in order to earn," that's my motto.
2) Read both books carefully. Do not stack to a page or two; if you do not understand their context, take notes, and come back later.
3) After you finish them both, return to your records. Compare the number of pages you do not understand what they say, or you partially understand.
4) Try to contact the authors themselves to solve your queries. Both are very kind individuals, and the chances are that you will have answers. Remember: ask them specific questions and not general, for example, "Sir, on page 43, Chapter 1, I didn't understand this: why x(t) = logp(t) + dε?". Do not ask things like, "how am I going to be successful" or "what do I need to succeed in trading," these people are very busy with their matters, and we need to respect their time.
5) Here you are, I believe after the above steps you can have your answer.
I hope I helped a bit.
1)Go to youtube.com and search for both gentlemen. Watch a couple of videos with them. Then pick one of their books. "You need to spend in order to earn," that's my motto.
2) Read both books carefully. Do not stack to a page or two; if you do not understand their context, take notes, and come back later.
3) After you finish them both, return to your records. Compare the number of pages you do not understand what they say, or you partially understand.
4) Try to contact the authors themselves to solve your queries. Both are very kind individuals, and the chances are that you will have answers. Remember: ask them specific questions and not general, for example, "Sir, on page 43, Chapter 1, I didn't understand this: why x(t) = logp(t) + dε?". Do not ask things like, "how am I going to be successful" or "what do I need to succeed in trading," these people are very busy with their matters, and we need to respect their time.
5) Here you are, I believe after the above steps you can have your answer.
I hope I helped a bit.
Is clearly not true (assuming we can agree on what 'classic' data science is)
GAT
Much less important than the *level* of the math is the quality of *your* execution of it. The math is little more than high school stuff. But your execution must be flawless, ultimately. You must put this stuff together as if your account depended on it. Heh.
If you backtested the strategy carefully with attention to common biases, left a safe margin of error and still the strategy shows profits in the backtests, why should it not show profits in live trading? And more importantly, why should the size of the account matter? (as long as we are not talking about gigantic account size differences)
Few reasons:
1) Cost of doing business (slippage and commisions/spread)
2) If forex, Forex brokers are mostly bucket shops who run stops
Plenty of others, these 2 are enough