Go for manual or automated trading simulation?

[MaxPastukhov]

Both is clearly advantageous as when you're polishing something off, it'd be nice to run a ton of sims and walk away, but that's not how you develop a system. Developing a system has to be manual, otherwise it's crap. You start off with an idea, or set of ideas. Whittling away at it can't be automated or guessed on. Automated can also help with you throwing away ideas quicker.

Please correct if I'm wrong, but here is what I learned so far about the problem.

It feels like experienced/profitable traders don't need anything brand new. They either have their own in-house solutions or just stick to a tool of their choice. There is no way for me to develop anything unique here. I can't compete with dozens of existing automated solutons that use the same algoithms in various combinations. And, even if I develop something unique, it will be extremely hard (impossible) to convert those who got used to something else.

If we are talking about newbies, they can use something brand new, but the same problem arises: how to compete in the field where I can't really be different? If I'm developing a UI, I can make it look better, work faster or just make it more intuitive to use. In the strategy development world my options are limited to a combination of a limited number of algorithms everybody knows.

I also know that I can't make automated backtesting work as people may expect. I'm talking about the main goal - becoming a consistently profitable trader. While I'm sure that there are plenty of profitable manual traders (I talked to at least a dozen of them), I see how robot developers struggle. I personally talked to 2 traders who wasted about 10 years of their lives each on this task, without any success. I've never met even a single quant who is consistently profitable. Yes, there are some people who make some money here and there, but it usually doesn't last for long. Maybe they are less visible on the surface, not trying to show off, but I can't base my decisions on this only assumption.

I feel that I should stick to manual backtesting, learning and training system. As for the automated part, a good way to go may be to integrate it with third-party automated solutions so that people can dig deeper manually into results they get either from their in-house or external automated backtesting tools. Something like an easy way to import a list of orders for further manual analysis, or even a direct integration when you can do it with a single click or automatically.

 

[Snuskpelle]

What is your favorite automated backtesting tool, by the way?
What do you like about it? What do you miss?

I will answer this and more generally how my view has developed. And I don't intend to elevate myself in this forum: I am a minor retail trader/educated software eng that fairly recently (past two years) started being consistent; which one might rightfully argue is a too small time to indicate consistency for the kind of trades I'm doing, with holding time of days to weeks.

Automated tools I've used specifically are Quantopian, QuantConnect (the one I still use since QT cut live trading), Metatrader before these, and various home coded tools. QC does not support (although could probably be extended) for manual backtesting.

First, let me state that my trading remains 100% manual at this time, despite that I have been doing automated back and forward testing for years now. Why is this?

I've been searching for algos to work in all market conditions. What I've had to conclude after testing everything under the sun is, these are rather far and few between. [A sidenote: A beginner might get an impression there's a lot of stuff that works... there isn't, and the Internet is overflowing with BS from wannabe traders that are actually losers and would better serve as comedians in the worst cases.] Algos are very easily fooled, and it's difficult to account for all possible circumstances. You will have to be doing some kind of assumption in what kind of your algo is going to operate for most strategies. Of course, you want to make sure it does not go to hell fast if circumstances change before you can react, but...

An awful way of testing is writing some code, running test to get an equity code, then adjusting code to get another equity curve, repeat. Not just due to curve fitting, but you need to see what the algo is doing. You might be surprised to learn that the code you just wrote is doing entirely different trades than you intended, but the equity curve could still improve due to random reasons. Understanding what the algo is actually doing is paramount for it to have any utility. Maybe it's based on something that will not work in a somewhat different market environment, but that's totally fine if you're there to enable/disable on a discretionary basis.

This also brings us to that coding takes quite a bit of time. Sure, in a very productive coding environment this can be comparatively little, but manually backtesting whether an idea can have merit is usually faster than coding it, I find. So the natural progression is then to want to manually and automatically backtest in the same environment. Here one problem of a manual backtesting tool I've used is aimed towards price signals more or less exclusively; in reality trading on multiple correlations or other statistical measures may be far more reliable. Price is a quite fatigued predictor of price since exactly everyone can see it...

Of course, a different approach is full the scientific/quantitative research process in which case the above matters less (although being in the market can still provide ample inspiration for research), and I think this is the one necessary for fully automated systems. I am not a data scientist however as much as I meander in that territory. The full quantitative process is presumably also the goal of QuantConnect, and they do not seem to aim for manual approaches to trading at all. This is a shame, for although I think pure AI may eventually win the battle, for now the machine augmented human stands a chance and is in fact a strong contender in low frequency trading. Although, it may be their specific business case and the fact it's a bad idea to do everything that guides them.

 

[Snuskpelle]

Thank you for the reply.

Can you please take a look at my tool and tell if it's enough user-friendly?

I will some time next week register and have a look, my initial impression is positive and I was definitely not expecting this amount of progress.

 

[MaxPastukhov]

I will some time next week register an have a look, my initial impression is positive and I was definitely not expecting this amount of progress.

You are welcome
Demo requires no registration, by the way - just download and install it.

 

[MaxPastukhov]

I will answer this and more generally how my view has developed. And I don't intend to elevate myself in this forum: I am a minor retail trader/educated software eng that fairly recently (past two years) started being consistent; which one might rightfully argue is a too small time to indicate consistency for the kind of trades I'm doing, with holding time of days to weeks.

My congratulations on finally becoming profitable!

Automated tools I've used specifically are Quantopian, QuantConnect (the one I still use since QT cut live trading), Metatrader before these, and various home coded tools. QC does not support (although could probably be extended) for manual backtesting.

What do you like in these tools? Should I invent the bicycle or should I better integrate my tool with these solutions in some convenient way?

First, let me state that my trading remains 100% manual at this time, despite that I have been doing automated back and forward testing for years now. Why is this?

I've been searching for algos to work in all market conditions. What I've had to conclude after testing everything under the sun is, these are rather far and few between. [A sidenote: A beginner might get an impression there's a lot of stuff that works... there isn't, and the Internet is overflowing with BS from wannabe traders that are actually losers and would better serve as comedians in the worst cases.] Algos are very easily fooled, and it's difficult to account for all possible circumstances. You will have to be doing some kind of assumption in what kind of your algo is going to operate for most strategies. Of course, you want to make sure it does not go to hell fast if circumstances change before you can react, but...

Yes, I came to the same conclusion. While there are plenty of indicators and various systems, it all comes to a limited set of extremely simple math equations. Why? My idea is that it all begins with what humans can actually see on the screen. People can't do complex maths on the fly, that's why any algorithm you can ever invent must be based on what an Average Joe may notice and act accordingly.

An awful way of testing is writing some code, running test to get an equity code, then adjusting code to get another equity curve, repeat. Not just due to curve fitting, but you need to see what the algo is doing. You might be surprised to learn that the code you just wrote is doing entirely different trades than you intended, but the equity curve could still improve due to random reasons. Understanding what the algo is actually doing is paramount for it to have any utility. Maybe it's based on something that will not work in a somewhat different market environment, but that's totally fine if you're there to enable/disable on a discretionary basis.

Here is yet another important idea: you need not only to see what the algo is doing. You must also understand if what it's doing has any meaning in the real world. For example, MA and Bollinger Bands may work just become plenty of traders place them on the charts. But, when we are talking about some exotic ideas, the first question is: "Why they should work?"

And, after you found that something actually works, the next question is: "Did I learn enough about curve fitting?"

This also brings us to that coding takes quite a bit of time. Sure, in a very productive coding environment this can be comparatively little, but manually backtesting whether an idea can have merit is usually faster than coding it, I find. So the natural progression is then to want to manually and automatically backtest in the same environment. Here one problem of a manual backtesting tool I've used is aimed towards price signals more or less exclusively; in reality trading on multiple correlations or other statistical measures may be far more reliable. Price is a quite fatigued predictor of price since exactly everyone can see it...

As I told before, price may be the best predictor for this exact reason. If everyone looks at something, finds the same reasons for entries and exits, it's where market activity happens.

I have an idea that you are successful in trading not because of amount of time you invested into backtesting but because you finally started to trade manually, based on price action. Nothing beats screen time, even if you spent it developing and debugging various algos

Of course, a different approach is full the scientific/quantitative research process in which case the above matters less (although being in the market can still provide ample inspiration for research), and I think this is the one necessary for fully automated systems. I am not a data scientist however as much as I meander in that territory. The full quantitative process is presumably also the goal of QuantConnect, and they do not seem to aim for manual approaches to trading at all. This is a shame, for although I think pure AI may eventually win the battle, for now the machine augmented human stands a chance and is in fact a strong contender in low frequency trading. Although, it may be their specific business case and the fact it's a bad idea to do everything that guides them.

My maths teacher in the university always told us that it's much more important to deeply understand what we are actually measuring and calculating, than the numbers themselves. I'm a data scientist by education, my core expertise is "The Theory of Information". I spent about 3 years learning really complex maths/statistics/human-machine-human communication and I'm sure that it can't be applied here unless you take into account human psychology. Market isn't "maths", market is made of humans.

While idea behind QuantConnect may look solid at the very first glance, using this tool from a technician point of view may be not the best idea.

 

[MaxPastukhov]

Thank you all for you votes!

I feel that I'm ready to make the final decision. It's not that easy, I've already spent 2+ days weighting all possible outcomes. I may disappoint somebody, sorry in advance.

While the results are slightly in favor of automated backtesting, I would prefer to go for manual backtesting, learning and training. Here is why:

Yes, it will not be hard for me to develop yet another automated strategy builder, add some code to run it automatically and fill it with various random inputs from indicators. Easy interface, cool charts, loud sales letter... But I will get just yet another piece of code nobody will ever use to make money on a regular basis. I will earn more money, but it will never translate into my customers' success.

Before I came to the market, I spent several weeks trying to find profitable traders. I finally found several dozens of them but nobody is a pure quant. All of them trade manually, some use quantitative technologies to get deeper insights about the market, some partially automate market research and trade execution. Nobody makes their living from pure AI or any other type of algorithm.

The strongest trading mathematician I knew, who invested 90% of his time into statistical analysis and internal software development, still made final decisions by himself, using his gut feeling of the market. He made $2,000,000+ out of $20,000 in 12 months in 2001, publishing each and every trade he made in a Yahoo group. While his trades were mostly maths-based, they had a solid trading psychology background. He clearly understood currently active groups of traders and was able to predict their actions if something goes right or wrong for them in the nearest future.

Maybe quants are just secretive, maybe there are plenty of them, but I failed to find anyone who is consistently profitable, if not take into account a number of self-promotion articles/websites/blogs without any long-term proven track record. Yes, there are huge quant funds that may be profitable, but I my goal was to find an independent retail trader or a small group of them. Even if we are talking about quant funds, I can't be sure that there are no human intervention, especially if taking into account a team of humans that constantly updates all these algorithms and decides whether they are good or not for current market conditions.

I personally met 2 traders, each of whom spent 10+ years on this problem without any success. They were persistent in making negative talks about how hopeless and time-consuming it may be

I have enough education and experience in data science but I still don't believe in long-term success of quantitative trading, especially when we are talking about small retail traders. Maybe some big guys doing HFT using computer farms at exchange ground floors have a chance to earn big, but I don't want to give even a tiniest piece of hope to a regular computer-literate guy who will ever come to me for the solution.

Yes, I will lose big money by ignoring this market. I'm sure that those who will follow the "please them all" route will earn much more than me serving regular human traders, not star software developers. I'm ready to accept this loss, but I'm not ready to develop and sell something that I'm not confident about. I would be happy about average outcomes for the users of my products, not for top ones. I hate trading competitions because they almost always put the least lasting trader on top. Yes, there are rare good exceptions, but not to be happy about the idea itself.

I'm sure that there are plenty of profitable manual, human traders. I've found a lot of them, several dozens until I felt that it's enough. A lot of them are 20-30+ years in the business which means a lot for me. I prefer measuring probabilities based on long-term data sets, not trying to make assumptions on just 3-5 good years. I came into the trading niche because it's at least a thousand years old which means that it will last for the next 1000 years, at least. It may evolve, but humans will always win against computers, especially when we are talking about human+AI synergy.

So, I will focus on manual backtesting, learning and training. I'm sure that I can develop the best product which will really help people to trade better. Nothing beats screen time and experience, but I can try to help them to earn it faster, easier and less risky.

I may also add some quantitative technologies in the future but only as a way to assist human traders, not to replace them. I feel that I can borrow a lot from the quant universe that may actually help human traders. I'm not a luddite. Anyway, my focus will always be on people, not algorithms.

What's more important is that I will develop something that I will be confident about. Something that will not make me feel bad when I sell it because I will know that I've just helped somebody, not only "made more money". It may sound naive, but it's my personal choice. My wife supports me this time, which is a good sign as well