Ranking system alternatives / objective functions -- your thoughts?

[tradegee]

I'm at the end of an optimization and testing period.

I'm coming up on a tough problem. I have a large number of candidate systems that have passed a high bar for trade performance, expectancy, and risk measures.

Comparison seems asymmetrical. For example, three candidate systems might have very similar expectancy ratios, percent of profitability, Sortinos, good Monte Carlo performance, but one could have a much different skew measure. Each tests relatively well out of sample. So, in general the CAR for each is near the other, but one might be a different animal in some way.

I've been reading a lot on ways to rank candidate systems, but nothing's gelling yet.

Any suggestions on what I might look into to get a new take on an objective function? I had been trying CAR/MaxDD as a simple calculation to start with.

 

[southall]

Pick the system that seems least curve fit.
Also pick the system you are most likely to stick with during drawdowns i.e. feels the most right.
All those metrics wont mean much when you suffer a drawdown that is bigger than anything in your tests.

 

[tradegee]

Pick the system that seems least curve fit.
Also pick the system you are most likely to stick with during drawdowns i.e. feels the most right.
All those metrics wont mean much when you suffer a drawdown that is bigger than anything in your tests.

The system has very low time in drawdown and recovery time in general.

I have a specific deviation calculation I will use for determining whether the system is tracking adequately or not, so out-of-bounds drawdowns mean the system is turned off for monitoring or shutting down totally. I expect a future drawdown to be bigger than tests.

The only ways I can think of to determine extent of curve fit with the tools and metrics I have are WFAs and parameter shift analysis.