Optimization

[elitetradesman]

It appears that the way NinjaTrader does optimization is simply running multiple backtests with a range of values for each parameter in a brute force way. Is this correct? Then, is optimization just running a series of backtests and picking parameter values from the backtest with the best results? Does that mean a trading platform does not make use of any linear optimization library?

Thanks.

 

[LeeD]

The problem is non-linear for most non-trivial trading strategies.

 

[stevegee58]

That's the usual way backtesters work. Same for WealthLab and MetaTrader.

 

[VisualQuant]

We have stochastic optimization (simulated annealing) in OpenQuant. Check it out...

Cheers,
Anton

 

[jimbojim]

Quote from elitetradesman:

It appears that the way NinjaTrader does optimization is simply running multiple backtests with a range of values for each parameter in a brute force way. Is this correct? Then, is optimization just running a series of backtests and picking parameter values from the backtest with the best results? Does that mean a trading platform does not make use of any linear optimization library?

Thanks.

Good point and I don't think many here will understand it because I suspect very few in these threads have had formal studies in optimization theory.

To make a long story short, never, I mean never use products like that. This is not optimization but plain undermining of the intelligence and knowledge of people. They call it optimization to impress idiot traders but in the best case it is more of a crude perturbation technique with no measure of stability and convergence.

I have worked in the past for a quant firm and there they used non-linear optimization algorithms to find the optimum parameter set for strategies and they also calculated metrics to ensure that was a true optimial set.

 

[newDave]

Quote from VisualQuant:
We have stochastic optimization (simulated annealing) in OpenQuant. Check it out...
Cheers,
Anton

Unfortunately,
It is not possible to make a good use out of this feature in this product.
Despite the “engine” and the idea looks quite good but any customizing is missing for this simulation process, no reporting on the process and what is more important - there is a restriction that you can optimize only Profit, or Profit/DrawDown which is quite insufficient.
Maybe guys will improve it someday…

 

[newDave]

Sorry, I made a typo

Quote from newDave:
customizing is missing for this simulation process

I meant really that customizing for simulated annealing optimization is missing. They have a lot of simulation customizing...

 

[Eight]

Quote from VisualQuant:

We have stochastic optimization (simulated annealing) in OpenQuant. Check it out...

Cheers,
Anton

That is the best thing around for a test with a really large parameter space.

IMO most systems fail going forward not because they were not tested properly, they fail because the creator of the system did not understand how changing market conditions would affect a system.

I've used the simulated annealing by doing the sop thing that people do, test on out of sample data and view the in-sample and out-of-sample parameter space results in a 3d viewer... it's probably not going to get better than that for me unless I can hire a Math PhD to guide me into something better...

Also, learning to use successive approximation is a simplified way to reduce the necessary test results.

 

[newDave]

agree... generally, simmulated annealing seems to be quite good.

 

[newDave]

Quote from Eight:
That is the best thing around for a test with a really large parameter space.

BTW, do you mean Stochastic methods (e.g. annealing) in generall or you speak exactly about the feature of OpenQuant ??