Backtesting optimization software

[nonlinear5]

I have extensive backtesting optimzation experience, albeit 20 years ago. Again, with 25 vars, my typical run might be 300-900 iterations.

1. There is "backtesting", and there is "optimization". But there is no such thing as "backtesting optimization". There is a concept of "portfolio optimization", and "trading strategy optimization", but perhaps what you mean is entirely different.

2. To statistically justify the necessity of 25 variables in a model, you would need many, many millions of trades. It sounds to me like many of your variables are fixed, in which case they are not variables, but constants.

 

[lindq]

Check out InvestorRT at Linnsoft.com. Contact them about optimization.

 

[Q.E.D.]

1. There is "backtesting", and there is "optimization". But there is no such thing as "backtesting optimization". There is a concept of "portfolio optimization", and "trading strategy optimization", but perhaps what you mean is entirely different.

2. To statistically justify the necessity of 25 variables in a model, you would need many, many millions of trades. It sounds to me like many of your variables are fixed, in which case they are not variables, but constants.

Re #1: I've seen software to backtest, but only test one fixed set of vars. So one might test a 10 & 20 SMA, going back years. But the software is not able to vary the short & long MA. I add the word optimization to make sure the software can run thousands of iterations, in one test, and output the results in a spd sheet, to permit logical evaluation.

Portfolio is completely different, as my initial work will be backtesting optimization ONLY with SP futures.


re #2 While one or more of the 25 vars may be a constant in one run, it may vary in the next. Simple case: One var is day of week. I run Monday with say 5-20 other vars, changing or on/off, and still generate say 500 iterations. Then I run a 2nd set with day of week = Tuesday. In that run, it is a constant, but among different runs, it changes.

I assume trading strategy optimization will be combining/ optimizing among many systems.

When I managed money RT, I used approx 50 different trading systems, each with 10-50 vars. I did have a separate portfolio module, but not for futures, but rather than we carried small positions in 200+ securities.

 

[d08]

Re #1: I've seen software to backtest, but only test one fixed set of vars. So one might test a 10 & 20 SMA, going back years. But the software is not able to vary the short & long MA. I add the word optimization to make sure the software can run thousands of iterations, in one test, and output the results in a spd sheet, to permit logical evaluation.

Portfolio is completely different, as my initial work will be backtesting optimization ONLY with SP futures.


re #2 While one or more of the 25 vars may be a constant in one run, it may vary in the next. Simple case: One var is day of week. I run Monday with say 5-20 other vars, changing or on/off, and still generate say 500 iterations. Then I run a 2nd set with day of week = Tuesday. In that run, it is a constant, but among different runs, it changes.

I assume trading strategy optimization will be combining/ optimizing among many systems.

When I managed money RT, I used approx 50 different trading systems, each with 10-50 vars. I did have a separate portfolio module, but not for futures, but rather than we carried small positions in 200+ securities.

Even an old version of WealthLab can do this, software from over a decade ago. What you're asking is very basic.

 

[Q.E.D.]

Check out InvestorRT at Linnsoft.com. Contact them about optimization.

Thanks, but I looked quickly at Linnsoft site, & although it seemed program does optimize (via posting in forum,) it was certainly not featured by the company. And just the number of postings, lead me to think it is not a wildly used program. There seem to be a number of open source programs that offer more of what I need, such as BackTrader, Quantconnect, etc, which I will investigate first.

 

[Q.E.D.]

Even an old version of WealthLab can do this, software from over a decade ago. What you're asking is very basic.

Years ago, before WealthLab was bought, I had the owner customize WealthLab for me. I worked closely with him for I think a few months. It was an impressive program, but as I recall had many shortcomings, for me, but I'll check it out again; thanks.

But what I'm looking for is not at all simple. My old Linux program, I think with 100.000+ lines of code, but also including many open source modules, was superior to all but the $50,000+ programs used by hedge funds. It should have been, because our development cost was well over $250,000. Part of that was because to manage money, it was impt to own the source code.

Now I am only looking to trade for myself. Five years ago I was unable to find anything, so had two programmers from Brazil (much less expensive then) to try to upgrade my old Linux, but ultimately were unable to do so. But today there seem to be a number of platforms that might accomplish what I'm looking for, and some suggestions on this thread have been very helpful.

 

[userque]

My background:
1. Full-time trader, mostly futures, for 30+ years;
2. Managed money in futures, 20 years ago, as CTA, using proprietary software platform, under Linux.

I'm looking to get back into testing strategies, initially on futures. I'm working with a very experienced programmer, but I'm starting investigation for platforms.

Thru Elite, I learned about & registered for trial with Quantconnect, & watched a couple of videos, & scanned documentation.

HOWEVER, I did not yet find info re backtesting optimization. Anybody have experience with Quantconnect?

Questions
1. Can one run, say few hundred optimizations on SP e-mini, with 25 variables, and control the output results, in spd sheet format?
For instance, can one list iteration 1 to 500, sort by # trades, then % profitable, then another 2-3 levels?

The main question is how one runs optimization runs, including var input. It appears that var range, which I assume is possible, would need to be hard coded, vs read in from separate file/spd sheet

I've written to Quantconnect support, but no answer yet.

Appreciate any feedback. In exchange, again, I'm very experienced, profitable, active discretionary technical trader (I use IB, Sierra Charts, 12 monitor screens)

If you ultimately find you need a custom-built solution:

RapidMiner and Knime can both optimize parameters. At least one can even do so via an evolutionary algo (as I recall).

Your solutions can be coded using visual building blocks; so it may be faster and less expensive to employ an experienced RapidMiner/Knime developer for one of these platforms instead of coding from scratch using a general programming language.

Whichever road you choose, your choice of 'which search/optimization algorithm to use' is very important.

 

[Q.E.D.]

If you ultimately find you need a custom-built solution:

RapidMiner and Knime can both optimize parameters. At least one can even do so via an evolutionary algo (as I recall).

Your solutions can be coded using visual building blocks; so it may be faster and less expensive to employ an experienced RapidMiner/Knime developer for one of these platforms instead of coding from scratch using a general programming language.

Whichever road you choose, your choice of 'which search/optimization algorithm to use' is very important.

Thanks

 

[Q.E.D.]

My background:
1. Full-time trader, mostly futures, for 30+ years;
2. Managed money in futures, 20 years ago, as CTA, using proprietary software platform, under Linux.

I'm looking to get back into testing strategies, initially on futures. I'm working with a very experienced programmer, but I'm starting investigation for platforms.

Thru Elite, I learned about & registered for trial with Quantconnect, & watched a couple of videos, & scanned documentation.

HOWEVER, I did not yet find info re backtesting optimization. Anybody have experience with Quantconnect?
For the record, Quant Connect support got back to me: backtesting optimization is on their "to do" list. I removed that from consideration, because we all know, that even if it is done soon, it would be quite some time before bugs, etc., corrected.

Questions
1. Can one run, say few hundred optimizations on SP e-mini, with 25 variables, and control the output results, in spd sheet format?
For instance, can one list iteration 1 to 500, sort by # trades, then % profitable, then another 2-3 levels?

The main question is how one runs optimization runs, including var input. It appears that var range, which I assume is possible, would need to be hard coded, vs read in from separate file/spd sheet

I've written to Quantconnect support, but no answer yet.

Appreciate any feedback. In exchange, again, I'm very experienced, profitable, active discretionary technical trader (I use IB, Sierra Charts, 12 monitor screens)

 

[Aquarians]

Levenberg Marquardt is a standard way of optimizing a large set of parameters, provided that you already start wih a good approximation, which in my experience is pretty much what you'll get.

Also, how much you're paying the Brazilian super-programmer?