Can a retail trader succeed in algorithmic trading? (Kevin Davey vs Ernest Chan)

[TommyR]

The reason is bank and large firms algos can only work for them, examples are market making and HFT. Hardware barrier of entry is in 10s of millions.

What you are looking for a strategy can can be deployed on retail level. While those exits, no one in their right mind would disclose it. It is not that difficult to raise funds and trade it, as long as it indeed has positive expectancy.

You will not find it anywhere in public domain.

remember by 'algos' or 'quant' it's actually like 3 people. the 'quantative hedgefund' doing factor rotation and buying the moving average crossover etc does mot make the market move like it moves

 

[kevinkdog]

Kevin Davey, an (apparently) successful algorithmic trader/author claims simple strategy works and will always work.

Ernest Chan, another guru in this field, however, says simple quant strategies don't work anymore and machine learning is a must...

Sorry, I just saw this thread today after getting the weekly "Most Popular New Threads" e-mail. I thought I'd just clarify my philosophy...

I do like simple strategies, and a lot of that is because simple strategies are harder to curvefit. Fitting a strategy to past data is the biggest "crime" I see developers making. And those fitted strategies tend not to work in real time as well as simpler strategies.

But that doesn't mean simple strategies always work, or complicated strategies never work. And it does not mean that simple strategies that do work are perfect (most simple strategies have large drawdowns, for example).

edit: sorry that last paragraph was terribly written. Here is what I meant:
Can simple strategies work in real time? YES
Can simple strategies fail in real time? YES
Can complicated strategies work in real time? YES
Can complicated strategies fail in real time? YES

As far as simple strategies "will always work" - I plan on every strategy I have eventually not working anymore. And the simpler a strategy is, the easier it is for others to find, and that hurts the "simple strategy" developer.

Ernie's approach is much more complicated, and less likely to be copied by a ton of other developers. That is a definite plus. And he does a lot of other really good things in his development. We were actually just e-mailing each other last week, as there are things he does that I am looking at doing.


BOTTOM LINE: I think you can succeed with either approach, and you can fail with either approach (and succeed with other approaches too, like what @globalarbtrader does - his books are worth reading). The key is to find a strategy development approach that correlates with future performance. Easier said than done, of course, but I am amazed with how many people forget this end goal.

BTW, I am disappointed with @kevinkdog as he doesn't seem to contribute to the forum. I understand he is paying to be a sponsor so his choice, but other sponsors do.

Sorry. I typically avoid discussions at ET, as inevitably people will say "oh he is just trying to promote himself." I'm a sponsor, so I usually only post in the Announcements section (where it is clear promotion). People can always reach out to me privately.

 

[d08]

Ok, wait a minute: Everytime traders are talking about big players and market manipulation, everyone seems to agree that algos are running the game.
So, assuming that this is right - someone stating that 85% of market is driven by them - that means, that big banks, players etc. actually have financial algorithms, that work and are profitable.

If this is true, why is there, after decades of banks etc using them, no market for retail traders, for buying them, even if it is not the best/newest version of all?!

Coders working for banks could make a fortune with it/ at least there must have been some leaks...

Furthermore, you say that one can only create a robot on your own. Why? Can't you imagine that developers, that aren't actually into trading, would like to sell their robot for a good price, when if performs well?!
Especially if these developers don't have big money...

The algorithms the largest banks and funds use depend on infrastructure or size. There's almost no way an at-home type trader can participate in this. Many markets have large capital requirements -- 100s on millions just to enter. For others you need best co-location or contacts in the industry.

I've been following people who sell signals for a long time. There are very rarely profitable systems but they always go private fast because selling something that can make millions for thousands is not a business. On top of that you have people looking at it trying to reverse engineer, someone is bound to succeed.
I've reverse engineered one myself a long time ago as I knew what components he was using.

We live in a time where funding is not extremely difficult. From instant loans to credit cards to friends and family to mortgages -- if you have an extremely good algo then you'll get the funds. If you cannot get the funds then you're probably not smart enough to participate in the industry anyway. Consider it a test.

 

[ZTrader888]

From a physicist's point of view:

The misconception of mean-reversion
Journal of Physics A: Mathematical and Theoretical

"The notion of random motion in a potential well is elemental in the physical sciences and beyond. Quantitatively, this notion is described by reverting diffusions—asymptotically stationary diffusion processes which are simultaneously (i) driven toward a reversion level by a deterministic force, and (ii) perturbed off the reversion level by a random white noise. The archetypal example of reverting diffusions is the Ornstein–Uhlenbeck process, which is mean-reverting. In this paper we analyze reverting diffusions and establish that: (i) if the magnitude of the perturbing noise is constant then the diffusion's stationary density is unimodal and the diffusion is mode-reverting; (ii) if the magnitude of the perturbing noise is non-constant then, in general, neither is the diffusion's stationary density unimodal, nor is the diffusion mode-reverting. In the latter case we further establish a result asserting when unimodality and mode-reversion do hold. In particular, we demonstrate that the notion of mean-reversion, which is fundamental in economics and finance, is a misconception—as mean-reversion is an exception rather than the norm."
https://iopscience.iop.org/article/10.1088/1751-8113/45/33/332001/meta

So what does this say about algorithmic strategies? It says that they don't adapt to change very well. A machine can only learn what you teach it to learn or detect.

Now comes the AI which can "think". Changes the game completely. No more black box strategies or machines. But will it predict? Big question. Humans don't predict very well. If they did, humans in general would be great traders - and they're not. We have to resort to complex equations and a mishmash of data to even start the process of prediction - think earthquakes.

As Kevin said: Fitting a strategy to past data is the biggest "crime" I see developers making. But every strategy is dependent on past data at the very least for pattern recognition. When we say a strategy doesn't "work" anymore, what we're really saying is that we can't wait around 50-100 years for it to work again. Because it probably will work again given enough time.

I'm not into writing these types of strategies, but if I was, I would start here:

Uncertainty, Variability, and Earthquake Physics in Ground‐Motion Prediction Equations
https://pubs.geoscienceworld.org/ssa/bssa/article-abstract/107/4/1754/354057

Because the most successful algorithmic strategies will predict in ways perhaps not considered yet.

 

[digitalnomad]

When we say a strategy doesn't "work" anymore, what we're really saying is that we can't wait around 50-100 years for it to work again. Because it probably will work again given enough time.

In my experience, robust intraday strategies tend to come back faster. Of course if you’re highly leveraged, you get cleaned out, but if you’re able to recapitalize at a future point, you can maintain it, along with a basket of other intraday or even overnight strategies.

 

[tommcginnis]

The misconception of mean-reversion
Journal of Physics A: Mathematical and Theoretical

WOW. Those are some extraordinary findings -- mostly in that they should have been screaming out for years and years. In particular, in my memory, is the notion that a normally-distributed perturbation would produce a reversion-to-mean result in the primary variable under study. (An underlying, and perhaps flawed, assumption on my part is that a Brownian Motion/Weiner Process fit that bill......)

I will read this in depth later, but THANKS FOR POSTING.

 

[Real Money]

I'll give you an example of algorithmic trading that you will never be able to do.

Calculate and predict the exact cash value of a basket of ultra liquid rate and index futures. If you could do that and automate then you might have something. The problem is beyond you at this time, of that I am sure.

The pros are using arbitrage to dominate the market. They are using all of these

cash stocks / index futs
cash bond / rate futs
physical commodity / commodity futs

So, you will have some kind of trading algorithm that is using publicly available data feeds and a (static or dynamic) set of criteria to initiate and manage exposures.

This is the situation.

You will have to be an expert trader in order to think like a trader and then to build a trading system. A system that takes other market participants money, reliably.

That, or you will have to be a programming genius that can reduce trading to a simple exercise in full stack implementation. API, database, error checking, status look ups, adjustments, data tagging, ML, whatever (i'm a hacker not a programmer).

Trading is a secret art. An art that only a few dedicated and obsessive people will ever master. I had to scour the internet to find the greatest traders. Academic treatments of trading are necessary but not sufficient preparation.

This is a secret language.

Take the time to learn that language or you will surely fail.

 

[tommcginnis]

You will have to be an expert trader in order to think like a trader and then to build a trading system.

Or in the alternative, you will have had to buy tomatoes at a farmers' market.

Trading is a secret art. An art that only a few dedicated and obsessive people will ever master.

...Although, that list includes every 12-year-old with a collection of Pokémon cards, or every 30-year-old who happened upon a playable Hank Williams Library in the vinyl bins of Goodwill.

This is a secret language.

...which is available to you now, only through the Oliver Velez Secret Language Translator Service (with an annual payment of just $1597, if you call before now)....



["Sir, your snark is showing." Whaaaaat?? ]

 

[Orbiter]

And as a postscript, the strategies in my books still work and I still use them. Because they trade slowly, are based on sources of return that decay extremely slowly, and highly liquid, it's unlikely they will be competed out of existence any time soon. That doesn't apply to faster strategies discovered using ML.

GAT

Brilliant post. I have been using systematic trading for many years and fully automated bots for the last 6 in the same fashion you describe them. I exploit trends, pullbacks, statistical edges mostly on EOD timeframes. The lower the timeframes and less time a strategy survives as that market dynamic changes quicker on those timeframes.

ML is really hard to harness due to really high noise in financial data but statistical edges can still be found. So a retail trader can definitely succeed in algorithmic trading. No doubt about that. I’ll be looking for your book. Hopefully will see a reflection of my 20 year journey. Brilliant post.

 

[nooby_mcnoob]

Wow! So much talking past each other -- all convinced of their correctness, but in various grades ignorant of the rest of the picture. Like the whole Touching The Elephant -- all convinced that they've got the right story.....

FWIW, I like much of what Chan and Davey have put together. But even they speak in absolutes -- and I'm convinced that they're both smart enough to qualify their various statements were you to corner them at a holiday party...

• Finding a decision rule with positive expectancy is a nearly trivial operation.
• Finding a decision rule with positive expectancy on daily data that goes back 20+ years is (IMO) foolhardy.
• Finding a decision rule with positive expectancy on daily data that may be tweaked every 6-9-12 months to continue that positive outcome set is an operation that *may* benefit from pattern-recognition routines that inhabit the machine-learning space, but ...
• machine learning depends on data stability that is at odds with the very nature of time-series phenomena, and
• developing a meta-algo to modify a floor-level algo is *also* a nearly-trivial maneuver -- just google "time-series decomposition" for examples that go back half a century.

In my tick-scalping days, I put T/A up that looked good for 20-30 minutes; I relied on it for the next minute or two. For trend-exploitation, I put up daily candles that go back maybe 6-12 months, and rely upon it for the next week or so. HINT.

In other words, calibrate often. I've come to the same conclusion.

PS: GAT is a baller. Doesn't even have more than one screen.