Confidence based on a couple O' points....

Quote from amigasearch:

John - as far as watching it - it tests well over 10 years. The backtest results instill alot of confidence.
More...
If your backtest was performed properly, with strict in-sample/out-of-sample separation and honest estimates of commissions and slippage, then by all means start trading. Based on the fact that you are here seeking reassurance instead of confidently trading your system based on the backtested results, I can only assume that your backtesting was less than rigorous to some extent. (This is not meant to be disparaging; it's just a fact of life.) If my assumption is correct then prudence is in order. If I were in your situation (and I have been many times), I would simply watch the system for a while. I've never regretted it.

jj
 
My testing is simple, at best. Here is what I do (the method I use):
I b-test over the last 6 months (a theory that i think of), modifiying my parameters (the constant value, stops, etc.).
Then, when I am happy with results, I run a whole b test over the ten years. Thats it. If the ten years tests well, i consider this positive, since i make no changes to the ten year results (not even tweaking stop val. or times, or anything).
Commisions are factored in. Slippage is factored in.
Is my method viable? (is this considered out of sample testing and such?). Is this similar to your methods? Thanks for all the feedback.
 
Quote from amigasearch:

My testing is simple, at best. Here is what I do (the method I use):
I b-test over the last 6 months (a theory that i think of), modifiying my parameters (the constant value, stops, etc.).
Then, when I am happy with results, I run a whole b test over the ten years. Thats it. If the ten years tests well, i consider this positive, since i make no changes to the ten year results (not even tweaking stop val. or times, or anything).
Commisions are factored in. Slippage is factored in.
Is my method viable? (is this considered out of sample testing and such?). Is this similar to your methods? Thanks for all the feedback.
More...
Yes, this is fine as long as you don't play around with the strategy at all after the first time you run it on all 10 years. In fact, I'm not entirely sure why you're here asking what we think! :) If you developed a system on 6 months of data that held up on 10 years of data, then you should be very happy. My methods are indeed similar.

In a nutshell...
I develop systems on 5 markets between 1980 and 1990. I use all markets from 1980 through 1990 as my test set. I use those same markets from 1990 through present as my walk-forward set. I use markets that were not available until after 1990 as a "double-blind" walk-forward set. My strategies tend to be long-term, and are applied with the same rules and parameter settings to over 40 futures markets (no exceptions to this rule).

No single system applied to a single market produces tradable results (most are not even close). However, once I apply my position sizing algorithm and add enough markets to the portfolio, the results are more than acceptable.

In the end, I have created a portfolio of 12 systems that I can trade with confidence, along with an honest assessment of the behavior of those systems on a large collection of unseen data. My mission in life is to keep negative surprises to a minimum, and my development philosophy is guided by this quote:

“It’s amazing how rich you can get by not being perfect.”
- Larry Hite, quoted in Market Wizards by Jack Schwager

The results so far speak for themselves. Using 3 of my systems I was able to win the RQSI/Access CTA Star Search trading competition for 4Q2003 and I'm currently ranked 6th of the 50 participants in this quarter's competition.

jj
 
You are correct in your assumption of "Why ask here". Mostly, because i am nervous! When I look at the history, i have confidence. But, when I trade it (soon to be automated) I get scared.
I think if i had a 99.9 percent gainer to loser sytem, 1 to 1, I would still think todays the day it stops working!
Thats why I like this board and the members. Reminds me to put aside the irrational fears (based on past results of course!). I thank you taking time to help.
 
Amigasearch,

two years ago I made a system, intra day position trading, for the ES, 100% mechanical, 100% automated.

It was backtested & walk forward tested over three years, 2300 trades. P/F was around 1.36.

It traded well for two months in real life, then the ES range contracted and it has not worked ever since the start of last year April.

Since then I have been working on educating myself to become a discretionary trader going into areas that are not easily transformed into mechanical systems. That study is going well, but it has been slower than anticipated. Presently results are what was within the expectations of a beginning discretionary trader.

PS. Coming from 17 years of computer industry background and having worked on some very complex systems I am not quick to say that something cannot be automated. I have had some first hand experience of what the "big" money can do.

The next level of profitability will be reached when I augment my existing system with a trade (position) management system. Unfortunately I have not been able to find anything on the market that can do what I want to do, so I'll have to write my own in C#. I have been lucky to find a component supplier that will take care of 80 - 90% of the tasks that I need to accomplish. Otherwise I would be programming for a long time to come.

If I was going to go the 100% systems way again then I would definitely have multiple (diverse) systems and multiple (uncorrelated) markets and then try to make sure that they do not all break down at the same time.

Unfortunately these days the markets are more correlated than one may think.

Good luck.
 
without any fitting ? If it falls apart ... :)

Quote from amigasearch:

The system backtests very well, without any fitting - you just happen to like how 2 points works (the results are nice). BUT, adjust that 2 points - play with that constant value, and the system falls apart.
More...
 
You must log in or register to reply here.
Back
Top