what features would make the test as reasonable and fair as possible..
It has to be a playback, so there is a continuous updating aspect to it. It is not realistic to have a singular time slice.
Every bar that is put out there, has a chance to change EVERYTHING or nothing and everything in between. Which bars merit evaluation and which don't is a serious question to anyone who actually trades. When is the previous analysis invalidated? What if I can't finish my analysis before the next bar happens? What happens if the previous bar and the current bar contradict the analysis? Are you paralyzed?
All this "look at this chart" and tell me xyz is armchair kiddie stuff and not at all serious. One has to get real about it, as in REAL-time. There are examples EVERYDAY in thousands of charts. Just get to work and place your entries and exits.
Also I don't know about the concept of "fairness", as I have never really encountered that preparing for an entry or exit. If some Fed president makes a comment and the entire chart blows up in a double whipsaw during lunch hour, is that "unfair"? Was it fair the morning of 9/11? It is ThunderDome.
Trading requires a lot of experienced based skills, keen concentration, precise actions, and relentless effort. It is not a matter of being "good" as the OP wants to know, it is about doing all the parts well over and over without messing up.
I am out.
