Absolute need for walk-forward testing? (If given enough data)

SimpleMeLike said:
Hello RStrauss,

Can back test back to 2006?

Why are you only limited to 3 years of data?

Most platform come with minute data back to 2006.

Unless you are using tick data.
More...
I am not using tick data as I don't need to trade intrabar. All my trading considerations were based on the close of a candle.
 
A forward test is always a good idea, it should give you a different result and you could measure how much are you overfitting to your data.

Another good technique is to use a Random Forest.

In this video, there are a few more ways to test your data.

Essentially every test should evolve around overfitting in order to understand how happy is your algorithm with unknown data.

 
You said you live traded this for a bit less than a year. Does this mean the last 400 trades you plotted were live trades? If yes, then that instill confidence and just keep trading it.
If the live trades pnl do not looke like your backtested pnl for the last year then you have a problem. In that case the optimized in sample backtest is likely just overfitted data and your plotted equity curve does not reflect what you will see when going live.

Bottom line is you cannot trust in sample backtests no matter how many trades or how long the backtest is.
 
RStrauss said:
I am not using tick data as I don't need to trade intrabar. All my trading considerations were based on the close of a candle.
More...
Hello RStrauss,

Yes, that makes sense.

If you are using candle bars, then back test far back as you can go for confidence in your strategy

Even if results are not looking good from 2006 to 2019, at least you know what market conditions systems will not work well in.

But for what you are showing, I would definitely proceed running it live.
 
RStrauss said:
For some reason my setup (Multicharts open API to Interactivebroker) can only get meaningful data back to July 2021. Those before July 2021 were choppy and without much trading volume.
More...
Hello RStrauss,

I understand.

I am with NinjaTrader and I get free data back to 2006.
 
RStrauss said:
Hi folks,

I started building my algo trading strategy 3 years ago trading the HSI futures and have started paper-trading about a year ago and a while after that with real money. My strategy generates around 10 trades a week (around 30-40 trades a month). I have almost 3 years of data (around 1200 number of trades). I have backtested and optimized the whole data set and found a positive net profit each and every month. One could say this is curve-fitting - i don't deny it but since it gives me a positive net profit every month for 3 years and the actual trades executed have a >95% match with simulated results, do i have good enough reason to believe that it will continue to do so in the future?

Given the tenure of my data (going back 3 years) and the no. of trades going above 1200 which I think is not a small sample, do you guys think there is still a need for walk-forward test? How often should I re-optimize or should I re-optimize at all given the consistency of my strategy?

I attach the strategy performance summary and equity curve with close-to-close drawdown trading 1 contract of HSI futures. As you can see I am currently in the midst of a long flat period...one that has not been seen in my entire data set before.

Any advice would be appreciated. TIA.

View attachment 338480

View attachment 338479
More...

4 years before making a trade? I would have blown up 3 accounts by now. You can't learn anything from algos or paper trading...other than getting used to the platform which is important.
 
maler said:
You said you live traded this for a bit less than a year. Does this mean the last 400 trades you plotted were live trades? If yes, then that instill confidence and just keep trading it.
If the live trades pnl do not looke like your backtested pnl for the last year then you have a problem. In that case the optimized in sample backtest is likely just overfitted data and your plotted equity curve does not reflect what you will see when going live.

Bottom line is you cannot trust in sample backtests no matter how many trades or how long the backtest is.
More...


Yes and no. The equity curve graph shown is all from backtest. I started paper trading and real money trading for about a year, I get a ~95% match of real trades vs simulated trades but my strategy was not able to tackle some months in the past year (while the other months had good performance around the historical monthly average) so I have made enhancements to the strategy to try to improve on those poor performing months.

I take note of your last sentence - I guess what I can do moving forward is to continue to forward-test it with a small contract.

Tks.
 
Drawdown Addict said:
A forward test is always a good idea, it should give you a different result and you could measure how much are you overfitting to your data.

Another good technique is to use a Random Forest.

In this video, there are a few more ways to test your data.

Essentially every test should evolve around overfitting in order to understand how happy is your algorithm with unknown data.

More...
Tks for the pointers. i kinda skimmed through the entire video - a bit too advanced for me from a general economics&finance background.
 
You must log in or register to reply here.
Back
Top