Proper back and forward testing

[QuantWizard]

Quote from Allistah:

I've had a number of strategies that have worked for a while and then they stop working. Should you try to make it continue to work so that it keeps going or do you just scrap it and start over with something else?

I would think that you would want to develop a system that has worked for at least 5 years in the past and also does well with walk forward testing. You'll never know but I would think it would continue to work for a while after that.

You also touched on an interesting point. If multiple people start using a system will it cause it not work anymore? Is this why when people have a working system that they guard it with their life (so to speak)?

You'll always have occasional drawdowns. As long as you have these also in your back tests and they recover afterwards so that you can still see a long term positive trend in your equity curve, then it's nothing to worry about. But if there seem to be a more prolonged, long-term downward trend in your equity, I'd be more worried.

If you look at the equity curve of my trading, I've had some really serious drawdowns, but the long-term trend is still around appr. +100% per annum, hence I'm not worried about sharp, short-term retracements.

 

[KMeriwetherD]

You'll always have occasional drawdowns. As long as you have these also in your back tests and they recover afterwards so that you can still see a long term positive trend in your equity curve, then it's nothing to worry about. But if there seem to be a more prolonged, long-term downward trend in your equity, I'd be more worried.

If you look at the equity curve of my trading, I've had some really serious drawdowns, but the long-term trend is still around appr. +100% per annum, hence I'm not worried about sharp, short-term retracements.

I was wandering around on the forum and came across this thread which includes a chart of the long term performance of a system (QuantWizard). I was wandering if this is a typical example of what some of you experienced algo traders would see. I noticed it does indeed have a solid net positive trend but it also not only has draw down, but whole years where it just loses all kinds of money. It does not scare me to see that. I imagine systems trading is about waiting for the odds to give you what you want in the long run. Also, ideally you are running a few non-correlated systems at once. Hopefully they are not all drawing down.

What do you all think of what QuanWizard posted? What do your multiyear P/L curves look like?

 

[bashatrader]

Read about data-snooping bias. A good article from the price action lab blog: http://www.priceactionlab.com/Blog/2015/12/momentum-survivorship-bias/

 

[KMeriwetherD]

Read about data-snooping bias. A good article from the price action lab blog: http://www.priceactionlab.com/Blog/2015/12/momentum-survivorship-bias/

I did read that article the other day. I having trouble assimilating the difference between what I see in the chart above and what is depicted in the article. Both fall apart..... one recovers, the other does not. Perhaps the difference is in the expected volatility of returns. The drawdown depicted in the article was anomalous and out of place with what the past would indicate. QuantWizard's performance show a consistancy in it's non-robustness. (If that makes any sense) Looks to be a prime example if extraordinary returns that are paired with equally dramatic drawdowns. However, it appears that can work in the long run as well.

 

[conduit]

I attached an introduction to strategy testing and development which I found pretty realistic.

Hi there..

I hope this is the right forum for this question.

Lets say that my strategy runs on a 5m chart and it only trades during regular trading session hours.

How do you properly back and forward test this? Optimize on 4 months and then forward test the 5th month? The move up to the next 4 months that include the prior forward gest and optimize that. Then forward again the next month forward.

How do I know what the right thing to start off with is? Is there any good books that can teach you about proper back and forward testing?

Thanks,

-Alli