Your Thoughts On Walk-forward Testing To Assess Robustness

Edges that cannot be negated by others.

Structural edges work in this way. Informational and structural edges can't be negated by others, and they can't be arbed away. In fact, they usually involve arbitrage of some kind.
 
Hi,

Just wondered what your thoughts were on walk forward optimization as a way to test the robustness of a system?

I have done conventional walk-forwards, so for example build a systems based on 4 years of data (sample of around 350 trades) and then test that system on the most recent two years of data. I have systems that work on this basis which is a good sign you have robustness in your strategy. However if I try a walk forward optimization (so for example optimize every 30 days then restest 10 days out of sample) over the same period it destroys the backtest.

Part of me thinks this means the system is not robust, the last 6 weeks or so of optimized settings should have some predictive value on the next 2 weeks.

However, the same system is profitable on a basic 4 year in-sample versus 2 year out of sample test with no parameter changes. Which is a positive sign.

I find it very hard to get any walk forward optimization backtests to produce profitable systems. I don't want to cheat myself by trading non robust systems but at the same time don't want to set an unrealistic bar whereby I throw out good systems.

What are your thoughts?

Not sure I understand.

Your walk-forward is optimizing using only 30 days/6 weeks of data? Why aren't you using 4 years like your other test?
 
Not sure I understand.

Your walk-forward is optimizing using only 30 days/6 weeks of data? Why aren't you using 4 years like your other test?

Fair question. But it’s a technique some people use both for robustness testing and also staying on top of the system so it adapts with current markets. Idea is in real world trading you can keep the core system the same but have a short term optimisation regularly to keep the parameters as close to current market conditions.
 
generally speaking. Yes.

these are attempts to find the holy grail that doesn’t exist. What it may find is some temporary market behavior skewed away from randomness. But these skews will be discovered and arb’ed away quickly anyway.

you need a true edge like PTJ who can see the order flow faster than anyone else. Edges that cannot be negated by others.
richDude,

Stop with the bullshit man. I do not know if you being serious or trying to just fuck with people on this forum because you bored or something. If you can't post nothing with logical proven stats, just don't post man before you confused some innocent person. Its enough clowns on youtube doing that. Just leave with that bullshit man. Go find you a woman to play with or something.

You need to be very careful sir on this forum without logically expressing your thoughts. People are working spending lots of time in trading. PLEASE stop with the Bullshit or explain yourself if you going to go against the grain.

Let me make sure I understand you sir. And take time to explain your point.

1. So back testing an idea or trading method for xxx to xxxx is a waste of time, right?

2. Forward testing and idea or trading method for xxx to xxxx is a waste of time, right?

So, what should a trader do? Just trade an idea straight up.

I don't understand what you are saying. In another thread you said back testing is a waste of time. Now you saying walk forward is a waste of time. So what is a good use of time?
 
Last edited:
Pondering topic of "Walk Forward Optimization" a bit more... If you replace the word "Optimization" with correction as in correct the error(s) in your system, the concept becomes palatable. The changes should be made once you understand how the prior system was flawed, and NOT from merely turning the knobs on your mechanisms! -- The phrase "As simple as possible but no simpler" may help as long as emphasis is placed on the "no simpler" part. Determining "no simpler" is a holy grail.
 
Things have changed. It's a bit more difficult to manage trump tweets these days.

And I'm not even joking, it's a big problem for automated strategies these days.
 
Things have changed. It's a bit more difficult to manage trump tweets these days.

And I'm not even joking, it's a big problem for automated strategies these days.


Guess it depends what side of the fence you sit on. I know a couple of guys up close to a million this year mostly trading Trump tweets. Which I guess ties into my question of whether a strategy that reoptimizes regularly starts to adapt to changes in the market such as random spikes off of tweets that didn’t appear in the long term backtest
 
Guess it depends what side of the fence you sit on. I know a couple of guys up close to a million this year mostly trading Trump tweets. Which I guess ties into my question of whether a strategy that reoptimizes regularly starts to adapt to changes in the market such as random spikes off of tweets that didn’t appear in the long term backtest

How can you adopt a strategy to accommodate random? Especially when the random can cause move of such varying intensity?
 
what we really need, which I don't see many (even any) people doing, is analyzing the price actions and news stories and reach some kind of conclusion on how to approach the market.

back and forward test... well, I haven't seen anybody making money with it.... well, maybe you did, but I bet you there is no proof any of this helped at all.
 
Back
Top