Posting System Results - WHY!

T

twalker

I have seen quite a few posts here that ask opinions about systems and post some results whether in the form of an equity curve or stats.
This is a total waste of time and unfortunately only shows the ignorance of the poster.
Any idiot who has a rudimentary knowledge of a package such as Tradestation can create a system using optimisation on a single market that will show a respectable equity curve and produce great stats. As anybody who has been doing this any time at all will tell you however these systems will never stand up to real-time trading. If you put your money on them in real-time you will lose.
So why post this shite?
The only respectable results are those which have been achieved with real trading in real time. Unfortunately nothing else is of any relevance since readers on this forum cannot be sure how backtesting of any historical results were achieved.

Sorry, I know this is a rant and really I contribute nothing here but over time I will.
 
Quote from twalker:

I have seen quite a few posts here that ask opinions about systems and post some results whether in the form of an equity curve or stats.
This is a total waste of time and unfortunately only shows the ignorance of the poster.
Any idiot who has a rudimentary knowledge of a package such as Tradestation can create a system using optimisation on a single market that will show a respectable equity curve and produce great stats. As anybody who has been doing this any time at all will tell you however these systems will never stand up to real-time trading. If you put your money on them in real-time you will lose.
So why post this shite?
The only respectable results are those which have been achieved with real trading in real time. Unfortunately nothing else is of any relevance since readers on this forum cannot be sure how backtesting of any historical results were achieved.

Sorry, I know this is a rant and really I contribute nothing here but over time I will.
More...

I completely agree with you, i posted this in a thread about backtesting:

You optimized the parameters for backtesting. This always gives much greater results than real trading because you fitted the system to maximize profits in the past.
But for the future you might need a completely different combination or the combination can change each day. All depends of what kind of market you are in. For each kind of market you need a different approach.
What you can do, to see the effect of your system, is to take each time parts of the total period. You will see that the optimized system per period will have constantly other setups.
This confirms that what was optimal isn't optimal anymore afterwards.
I know someone who constantly optimized his system to always have the optimal setup. He never made money because he was in fact always running behind the facts.

---------------------------------------------------------------------------------------------------------
Backtesting on 1 indicator is the first stage a newbie has to go through. He is still convinced at that time that you can find a good system ( if not the holy grail) with 1 indicator. They don't realize that all the known indicators have already be optimized at least a million times by others. Once the newbie realizes that this is not going to work he will be ready for the next phase in his evolution to a real trader. That's why it takes years to make something out of nothing.
 
that's why i never optimize anything
if you come up with an idea and it doesn't work, it doesn't work.

if it works, it works. no optimization
 
"optimization".................what does this mean? Let's see what different traders give in their answers...........may be we will reach root cause why backtesting fails in real time.
 
IMO backtesting and optimisation are very valuable tools. I would not dismiss either as they are an excellent way to test and improve a model. Problem is that people new to this think that by searching for the highest return over 20 years of data by testing umpteen variables for hours on end until you get one result is the way to future riches, which of course it is not. It is actually a financial disaster of misconception.
Once you have a system however that you think will work, you do ideally need to be able to test that it is robust. What the testing and optimisation software does allow you to do is to find points of relative stability in your system variables. If these points of stability persist over tested sets and untested sets of historical data then you are likely to have a more robust system than you have if you do no testing at all.
I think that due to this sort of testing it is better to keep the number of variables to a minimum and rather try to make values as generic as possible i.e. Rather than using a fixed figure for something try to make it dynamic by relating it to price volatility or volume or some other direct market characteristic. Harder said than done. The reality is that coming up with a system which has a positive expectancy over un-optimised historical data is really just the tip of the iceberg.
 
"I think that due to this sort of testing it is better to keep the number of variables to a minimum and rather try to make values as generic as possible i.e. Rather than using a fixed figure for something try to make it dynamic by relating it to price volatility or volume or some other direct market characteristic"

Could not agree with you more on the above. Dynamic variables that adapt to the market conditions are the efficiency generators. I try to go back into charts day for day, signal for signal for atleast 4 years and test system reaction to market behavior, and this gives a good idea of what system is doing visavis the market. Optimization with blind number punching and declaring win ratio is 75% instead of 55%, is like stupidity.

There is a saying that price charts play out the same for any commodity but inherently commodities change over the years and that's what busts most of the systems.
 
Quote from twalker:

IMO backtesting and optimisation are very valuable tools. I would not dismiss either as they are an excellent way to test and improve a model. Problem is that people new to this think that by searching for the highest return over 20 years of data by testing umpteen variables for hours on end until you get one result is the way to future riches, which of course it is not. It is actually a financial disaster of misconception.
Once you have a system however that you think will work, you do ideally need to be able to test that it is robust. What the testing and optimisation software does allow you to do is to find points of relative stability in your system variables. If these points of stability persist over tested sets and untested sets of historical data then you are likely to have a more robust system than you have if you do no testing at all.
I think that due to this sort of testing it is better to keep the number of variables to a minimum and rather try to make values as generic as possible i.e. Rather than using a fixed figure for something try to make it dynamic by relating it to price volatility or volume or some other direct market characteristic. Harder said than done. The reality is that coming up with a system which has a positive expectancy over un-optimised historical data is really just the tip of the iceberg.
More...

Are you the same twalker from trade2win.com? I appreciate the posts there. I thought you were yield curve spreader? this sounds like systematic directional trading, please explain :)
 
I am one in the same yes.
I trade to make a living. Whether this is yield curve spreads or other markets I use tools at my disposal. Been many years in business and spent last 3 mainly doing STIR spreads but been working on directional models since trading for futures fund in 1994-8 when I first came across Tradestation and hence the mechanical system idea .
Recently I have found risk reward profile of the STIR spreads was becomming much less in my favour. Used to be case of 50lots on a spread was enough. Now you need 200 lots on outrights to get a reasonable fill but rewards are about the same. As a result I have been applying more margin to systems which I have had to keep under capitalised for a while due to the increasing STIR trading margin requirements.
I run a number of models mainly on FX and Indices but also across the board commodities. I did have automated models on the STIRS running through the TT API but problem is that the exchanges started to sting me for non-executed orders which meant I had to put those to bed.
Right now I am in the process of moving my office so out of anything but a few longer term fx positions. Will be back in new place early in July.
So have a little time to read and post.
 
ok thanks for the explanation. I am more of a discretionary stock guy myself, things that used to work some years ago there, don't work as well lately anymore neither, that's the marketplace. Hence I like to stay on top of everything in the financial markets. whenever you find yourself bored with the fx and fi, apply the same buying power to illiquid stocks and you'll have no problem seeing them go up when you buy and go down when you sell :D
 
Quote from twalker:

I have seen quite a few posts here that ask opinions about systems and post some results whether in the form of an equity curve or stats.
This is a total waste of time and unfortunately only shows the ignorance of the poster.
Any idiot who has a rudimentary knowledge of a package such as Tradestation can create a system using optimisation on a single market that will show a respectable equity curve and produce great stats. As anybody who has been doing this any time at all will tell you however these systems will never stand up to real-time trading. If you put your money on them in real-time you will lose.
More...

I'm not sure such a sweeping statement has any validity.

You have to start from the hypothesis that there is recurrent market structure. If you don't believe this, you quit and go home because it's impossible, system or no.

So given that you believe in recurrent market structure, how do you discover it? Obviously it has to be discovered by examining the historical evidence.

If you take a system that has, say 1000 trades in the 2002-2003 time frame and calibrate it for that data, and it continues to show comparable performance through another 1000 trades through 2003-present data, you are saying that somehow magically this structure that this system captures cannot persist into the future? That's absurd. If you want to say it may not persist into the future, ok, fine. But to say that it won't is just wrong.

If you're railing against testing that has no statistical merit such as poor sample sizes, then fine, sure, some people don't understand what they're doing. But sweeping away the idea of backtesting in general just seems as ignorant as optimizing a 10-parameter system based on 25 trades.
 
You must log in or register to reply here.
Back
Top