Backtesting

[shMark]

I don’t post much, because I have very little of value to say. Here is something that I think is worthwhile.

Very long backtests that were the bread and butter of the quant’s approach I consider more or less irrelevant. Markets change too much based on macro factors to consider them to be predictive. What works in one macro environment falls completely apart in another. Consequently, investing has to be approached with a more nuanced strategy that considers these factors. Which factors exactly?

* Interest Rates

* FED's policies - quantitative easing and tightening

* Inflation Rates

* Multiple macroeconomic indicators that identify recessionary pressures

* Consumer sentiment

I could go on and on, but you get the idea. A long-term backtest based on fundamental data of stocks doesn't take any of these things into consideration. This is where quant investing has failed and will continue to fail.

 

[Drawdown Addict]

A backtest is like an indicator, it can explain the past but it will never predict the future.

 

[trend2009]

But if a strategy fails in backtest, it is not usable

 

[Businessman]

it will never predict the future.

But nothing can predict the future.

Back tests work well if you don't over fit and are not expecting to find the holy grail.

Make sure you test over a very big sample size as well. You want to see some outlier drawdowns in the back test as well as upside outliers to make sure the back test isn't overfitted.

 

[shMark]

But if a strategy fails in backtest, it is not usable

You are both right. I was once a quant, and I have seen so many things work well in backtesting, and then fail. I have even seen things work well in backtesting, and then work well for ten years after backtesting, and then fail. The stock market is a very humbling place.

 

[shMark]

But nothing can predict the future.

Back tests work well if you don't over fit and are not expecting to find the holy grail.

Make sure you test over a very big sample size as well. You want to see some outlier drawdowns in the back test as well as upside outliers to make sure the back test isn't overfitted.

When I used to do it, we took overfitting into account, and had about 10 statistics we used to make sure they were valid tests, and still failure. It could have been as simple as more sophisticated people with better backtesting software started doing it, and arbitraged any advantage away. I did it for stocks not options. Maybe it works better for options.

 

[Businessman]

The gold standard of quant trading is Rentec Medallion. Sharpe ratio about 2.5, and i around 88% profitable months.

While avoiding systems that appear to be really good but eventually blowup, Victor Niederhoffer & LTCM style.

But you don't need to be as super powered as Medallion, even a sharpie of 1.5 and 65% winning months is still really good.

If Medallion can average 70%+ returns on $10 Billion+. Mere mortals should be able to figure out how to make 20% on $10million with a decent reward to risk.

 

[Slow Learning Elf]

What does this have to do with options?

 

[Businessman]

What does this have to do with options?

Sound like the OP is looking for Option trading systems.

 

[dorietrading]

I have written my own backtest software and thanks to this backtesting (not overfitting) ik know how to deal with crashes (most of the time i sell options). 2008, 2015, 2018, 2020 are all years where you can learn a lot from backtesting. A lot of option writers went broker because they didnt know how to handle a crash, backtesting can help you with that...