When to let it in

[Rapunzel]

Hi guys, quite new here.

I am just curious to know; in the process of trading signal development, how do you ultimately decide which signals to include in your repertoire.

I am currently working on some models, where I ultimately decide to include the signal where I find that it replicates in 2 separate forward datasets. But the original process of developing the model included multi-fold cross validation on the training data, before the check on 2 data sets.

So I have to ask myself why the signals didn't work completely on the first dataset (in spite of multi-fold validation) and in some case why those that did went on to fail on the second dataset. That is; are the successes on test data set 1 and then test data set simply incidental?

 

[Peter8519]

Use your charting software to run a screen and not backtest from 2007 to 2021. This should cover the bull and bear market condition. A screen will let you see the entry and exit. Choose a stock that you expect of your model and see what is your model action.

 

[maxinger]

Use your charting software to run a screen and not backtest from 2007 to 2021. This should cover the bull and bear market condition. A screen will let you see the entry and exit. Choose a stock that you expect of your model and see what is your model action.

It might not work if you backtest far too much.

about a decade ago, there were major currency wars.
currencies were moving massively.

Nowadays, currencies move just a little.

The environment has changed very significantly.