I'm a remote futures algo trader seeking backing/prop firm

[chris500]

Say, for example, you had an intra-day strategy that averaged 75 trades per year and you had data going back to the year 2000. How would you construct your rolling windows? Would they overlap?

thanks

The point of walking forward a strategy is to mimic a live trading environment without actually putting money at risk. You travel back in time and pretend the year is 2000, 2005, 2010, etc. and then pretend that you're live trading.

Rolling windows have increments that the expert/inventor of the term "walk foward" says should be around 25 - 35% of the backtest/parameter optimization period. So if you're "live trading"/walking forward the year 2004, your parameters should be based on a backtest/parameter optimization of the years 2001-2003 (3 years/33%) or the years 2000-2003 (4 years/25%). Then when you're walking forward the year 2005, your backtest/parameter optimization windows should be the years 2002-2004 (3 years/33%) or the years 2001-2004 (4 years/25%). etc. etc.

The above is what Robert Pardo says in his books. The real answer is that it depends and that no one is going give away their secret recipe on a silver platter. You have to put in the hours and figure stuff out for yourself. The other answer no one is going to give you is when/how do you know a strategy has gone into or out of profitability and whether you should stop trading a strategy or start trading a strategy.

 

[sle]

The other answer no one is going to give you is when/how do you know a strategy has gone into or out of profitability and whether you should stop trading a strategy or start trading a strategy.

That really depends on your process.

Most of the pros (real ones, not the ones that write books, with some notable exceptions) start with a prior hypothesis, establish statistical significance and then convert it into a strategy/signal. That process is very similar to a process of scientific discovery. Your back-test (which is, essentially, an experiment) is likely to be curve-fit but if it's based on a good prior and you understand the significance of your parameters it's not a big deal. Needless to say, in this setup most ideas either "do not work" a priori and/or stop working sooner or later, so a good quality research pipeline is key to survival.

My impression is that most retail systematic traders, unfortunately, simply throw shit at the wall to see if it sticks and do it many times over. In that case, they really do need some form of statistical validation - either by out-of-sample testing (not necessarily a walk forward, but any form will do) or some form of significance correction (e.g. do a basic Bonferroni correction for every data pass).

PS. my former description does not include HFT - their process is very different, for variety of reasons and I am not proficient enough within that world to comment

 

[chris500]

That really depends on your process.

Most of the pros (real ones, not the ones that write books, with some notable exceptions) start with a prior hypothesis, establish statistical significance and then convert it into a strategy/signal. That process is very similar to a process of scientific discovery. Your back-test (which is, essentially, an experiment) is likely to be curve-fit but if it's based on a good prior and you understand the significance of your parameters it's not a big deal. Needless to say, in this setup most ideas either "do not work" a priori and/or stop working sooner or later, so a good quality research pipeline is key to survival.

I think Robert Pardo is the real deal, it's just that he suffered a big drawdown around the 2008 timeframe, and that's probably why he published the 2nd edition of his book in 2009. But he recovered from his drawdown pretty quickly. This is probably wildly inaccurate, but I think his drawdown was 40% one year and then next year he was up 160%. Again, I don't know if those numbers are accurate, so don't quote me on that.


With regards to backtesting, the key is how much predictive power does your set of parameters/variables have. Generally speaking, using a large number of parameters will give you a result that is strongly curve fit and that has very little or zero predictive power. It will only have descriptive power - it will perfectly describe what has happened but not what will happen. A too small number of parameters will be neither descriptive or predictive. The goal is to achieve symmetry: the backtest should have equal descriptive and predictive power (what the backtest described what has happened is also what actually ended up happening in the future). It's a balancing act between too little and too many parameters.

 

[userque]

A too small number of parameters will be neither descriptive or predictive. The goal is to achieve symmetry: the backtest should have equal descriptive and predictive power (what the backtest described what has happened is also what actually ended up happening in the future). It's a balancing act between too little and too many parameters.

This is a matter of semantics. Also, what evidence is there that "too few" parameters and "too many" parameters, as a rule, can't work. I submit that ... as usual ... it depends.

Generally, systems with many parameters fail.
Generally, systems with few parameters fail.
Generally, systems fail.

Suppose there is a simple moving average system with 'one parameter:' the number of periods/days it averages.

Now suppose we code this system to automatically determine this 'parameter' based upon recent volatility.

Now, does this new system have any parameters?

 

[fan27]

The point of walking forward a strategy is to mimic a live trading environment without actually putting money at risk. You travel back in time and pretend the year is 2000, 2005, 2010, etc. and then pretend that you're live trading.

Rolling windows have increments that the expert/inventor of the term "walk foward" says should be around 25 - 35% of the backtest/parameter optimization period. So if you're "live trading"/walking forward the year 2004, your parameters should be based on a backtest/parameter optimization of the years 2001-2003 (3 years/33%) or the years 2000-2003 (4 years/25%). Then when you're walking forward the year 2005, your backtest/parameter optimization windows should be the years 2002-2004 (3 years/33%) or the years 2001-2004 (4 years/25%). etc. etc.

The above is what Robert Pardo says in his books. The real answer is that it depends and that no one is going give away their secret recipe on a silver platter. You have to put in the hours and figure stuff out for yourself. The other answer no one is going to give you is when/how do you know a strategy has gone into or out of profitability and whether you should stop trading a strategy or start trading a strategy.

Thanks for the info! I have seen some videos with Robert Pardo and will get his book.

 

[chris500]

This is a matter of semantics. Also, what evidence is there that "too few" parameters and "too many" parameters, as a rule, can't work. I submit that ... as usual ... it depends.

Generally, systems with many parameters fail.
Generally, systems with few parameters fail.
Generally, systems fail.

Suppose there is a simple moving average system with 'one parameter:' the number of periods/days it averages.

Now suppose we code this system to automatically determine this 'parameter' based upon recent volatility.

Now, does this new system have any parameters?

A too small number of parameters is 0, in other words random. Perry Kaufman has said that a single parameter is best and that he made tons of money using a single parameter. If you can make money using a single parameter then it's neither too little or too many.

 

[userque]

A too small number of parameters is 0, in other words random. Perry Kaufman has said that a single parameter is best and that he made tons of money using a single parameter. If you can make money using a single parameter then it's neither too little or too many.

Thanks, but you never answered my question.

 

[userque]

A too small number of parameters is 0, in other words random.

LOL ... a system with zero parameters in not necessarily random.

 

[chris500]

Thanks, but you never answered my question.

Oh, I thought it was a rhetorical question.

The answer is that I have no idea. Look at the code you wrote and count the number of parameters. If you haven't translated your idea into working code yet, then you need to do that first and the answer will reveal itself.

 

[userque]

Oh, I thought it was a rhetorical question.

The answer is that I have no idea. Look at the code you wrote and count the number of parameters. If you haven't translated your idea into working code yet, then you need to do that first and the answer will reveal itself.

Thanks again.