In-Sample? Out-of-Sample? "Simple" Sample?

[abattia]

Quote from Random.Capital:

If you tested against out-of-sample once, it's not out-of-sample the second time.

I think I could have been clearer in my description of what I was doing ...

In my outline of METHOD B, when I referred to optimizing "over the first group", I was referring to optimizing over the "in sample data", rather than to over the METHOD A dataset (i.e. the whole sample).

Does this address the point you are making?