in sample and out of sample testing

I just glanced at their backtesting results, their systems have between 200 & 600 trades over a 6 years period, this isn't enough for me to get interested ... to easy to have (intentional or not) over-optimization on such a small sample set.
 
good point. what would you consider a minimum number of trades for 1 market futures system over 1 year?

did you check the video? how could it be over optimized if the out of sample performance is as good as the in sample one?
 
I no longer bother with anything less than a few thousand trades in backtesting, and minimum 5 years back (preferably 10 years).

WRT out of sample testing, it doesn't mean much to me, as obviously every one discards systems with bad out of sample results, so this is no better IMO. The only thing that counts beyond backtesting is live - audited - results.
 
thanks for the clarification.

the only issue i see with live, audited, results is that you need to move from the space of "systems vendors" to the space of registered CTAs (commodity trading advisors).

there are still 2 issues i see:

1. for registered CTAs normally starting capital minimum are higher... almost always over 100K
2. the comment you made about survivorship bias for outofsample results still applies here: non performing CTAs go broke so any list you see suffers by definition from survivorship bias

i am still looking for a honest system vendor that offers decent systems in the retail space that i could add to my list.

if you know any i'd be interested in knowing your experience.
 
Any systems that is sold to the public, and has no control over the number of lots traded will fail.

Get a good system from someone who actually trades or not available on the shelf for everyone to grab. Try your broker, some do offer good systems not available for the public to "buy" and would lease only.

RG
 
You must log in or register to reply here.
Back
Top