Why Optimization Won't Kill My System and Will It Work In The Future?

Originally posted by bozwood
dottom,

Do you use TR?
More...

Boz, I use Wealth-Lab for all my testing. It is an extremely robust platform, and can even do such things as portfolio management, overlay testing of different time-frames, analysis of other instruments/indexes (e.g. VIX, TRIN, etc.), and adaptive adjustment based on max adverse/max favorable excursions, to name a few.

If you can clearly describe what you want to do, you can program it.
 
dottom,

Thanks. Was just curious because TR was the only other platform I had heard of regarding the severe slippage test. One other question, is Wealth-Lab resident on your computer or do you access it online?

Thanks
 
I have a question for all you guys out there trading eminis intraday systems... what is the best Profit Factor you have achieved, BEFORE slip and commish?

I have tried different filters on my trendfollowing system, which trades 1-3 times a day, the best I can come up with is a PF of 1.5 for NQ. (for the last 4 months or so testing).

Is anything higher than 1.5 even achievable ???
 
Good question dozu. I also want to know what is your system's ( not only day trading systems on eminis ) best profit factor?
Please add information about # of trades. - it's easy to have PF=2.5 with only 8 trades...

DT-waw
 
is available both online and a desktop version. The online version is free and fully functional, except that it is limited to testing on only a rather small basket of stocks due to timeouts and such. You can get started online and then move to desktop version once you are comfy.

And it is unbeatable. I've done a lot of looking and nothing else I've tried is even in the same universe as wealth-lab for building and backtesting mechanical systems.
 
Originally posted by DT-waw
Good question dozu. I also want to know what is your system's ( not only day trading systems on eminis ) best profit factor?
Please add information about # of trades. - it's easy to have PF=2.5 with only 8 trades...

DT-waw
More...

well, it trades 1-3 times a day, so it could easily have about 200 trades in 4 months.

I also have some long term commodity systems with PF over 3, but of course the intraday stuff generates more revenue. I guess it's a fact that TA works better with longer time frame. hence the higher PF with long term systems.
 
Just some questions about t-testing. I have run the results for my trend following system on the attached t-test excel sheet. This and other helpful sheets can be found at both www.futuresmag.com and www.turtletrader.com. My system received a t-test score = 1.77 and a probability of average profit (µ > 0)= 96.11% (1-TDIST(ABS(t),d.f.,1). From Daven's analysis this would tell me that the probability of my profits being produced from randomness would be 3.89%. This all seems to make sense to me until I plugged in a random series from 0 to 1 using Excel's RAND() function. To my surprise the t-test value returned a value = 18.11 and a probability of average profit (µ > 0)= 100%. This would imply just the opposite of my first interpretation. Am I going nuts or does something not make sense? Is it possible that my profits are 96.11% random? Does this have to do with the fact that in a trend following system you have many small losses and a few super large gains. In this way the trades are highly removed from the mean making them seem average? If anyone can put me straight on this please do as it would be much apprechiated. Check out the attached excel sheet to do your own analysis.

For free futures data go to www.geocities.com/opmtrader/
 

Attachments

"In this way the trades are highly removed from the mean making them seem average?"

What I meant to say was:

In this way the trades are highly removed from the mean. Does this phenomenon make the individual trades seem random?

Please excuse my error
 
Originally posted by opmtrader
This all seems to make sense to me until I plugged in a random series from 0 to 1 using Excel's RAND() function. To my surprise the t-test value returned a value = 18.11 and a probability of average profit (µ > 0)= 100%. This would imply just the opposite of my first interpretation. Am I going nuts or does something not make sense
More...

The series you plugged in was indeed random, but consisted of numbers from 0 to 1, which for this zero-based comparison is one bloody good system. Try plugging in random numbers between -1 and 1 (or -x and +x), and you should get a result somewhat closer to 50%.

Disclaimer : I have not tested the spreadsheet for accuracy, nor have I expressed any opinion regarding the contextual validity of this test.
 
Thanks Tom,

You are correct. I was confusing the elements of randomness with the way the data values were distributed. Basically as I now interpret it the t-test probability used above says that given the example distribution of profits and losses what are the odds that the next sample will be net positive (i.e. a determination of a repeatably profitable system in the future). Thanks for helping with the clarification.
 
You must log in or register to reply here.
Back
Top