Yes. That is exactly it! If one plans on re-calibrating models going forward, the re-calibration needs to be included in the backtest.Short post as I am on holiday (as you know) but I could not resist:
As long as you have a static methodology how you evolve new strategies over time you backrest over any dataset of your choosing. So, for example, if you decide to evolve new strategies every 6 months or upon a certain data trigger then have your back tester do that and generate your performance and risk metrics off generated trades. In market parlance this is called model re-calibration. Whether you do that manually or fully automated makes no difference. In essence your trading strategies are just another adaptive set of model parameters. Is that somewhat answering your question or did I miss the mark?
Enjoy your vacation!
)