All back testing platforms or methodologies I have seen involve some sort of testing (AI, ML, if then else, etc) on historical data and perhaps followed by some sort of walk forward and or live trading analysis. The result is one to many strategies to trade. Let's say I have data from 2000 to 2018 and let's also say I tested on several sub data sets and I have 18 years worth of simulated results. The results are flawed because the earlier results are from strategies that were formulated after that slice of data. For example, if we were to go back in time to 2010, the group of strategies that would pass testing and be selected at that time would be different than the strategies that would pass testing and be selected today.
Here is a more realistic approach:
Determine the minimum amount of data you need to test with and split it up and test accordingly. Let's say it is 10 years (2000 to 2010) and we arrive at a group of strategies. In simulation, we trade those strategies for the first quarter of 2010. At the end of the first quarter, we now perform the same analysis on the expanded set of data and perhaps wind up with a different set of strategies. Those strategies are traded in simulation for the next quarter (I am arbitrarily picking a quarter. Could be another time frame). To me, this seems like a true simulation. Is anyone else doing anything similar or know of any research platforms that can do this?