Jeff, Does your system use cointegration or correlation? Log returns or non-stationary price series?
In my tests, historically cointegrated series (Granger-causality test, error correction model) consistently broke down out of sample. Secondly, choosing an alpha to input to the regression...