Avoiding Curve fitting

Quote from Random.Capital:

Price action is determined by comparing prices at multiple points in time. Right there you have a parameter - the time delta between observations.
More...
Sorry but I disagree with your usage of the word parameter as it applies to a programming language used to code a trading strategy. A definition can be found online

http://en.wikipedia.org/wiki/Parameter_(computer_science)

"In computer programming, a parameter is a special kind of variable, used in a subroutine to refer to one of the pieces of data provided as input to the subroutine.[1]. These pieces of data are called arguments. An ordered list of parameters is usually included in the definition of a subroutine, so that, each time the subroutine is called, its arguments for that call can be assigned to the corresponding parameters."

As a professional programmer in the early 1970s when parameters first evolved it was first used for business subroutines that did common functions in languages like COBOL. For example a parameter was input into date conversion sub-routines that could be called or included in your program. You would give a special variable in the subroutine called a parameter a Gregorian date and the subroutine would process it an out came a Julian Date.

Today parameters or input parameters are supplied to a coded strategy. Using optimization multiple values are supplied these input parameters. Each parameter(s) value or set of values proposes a trading solution.

I would not interpret comparing prices at multiple points of time or a time delta as a parameter. It would be a form of storage either as a variable or a matrix. Parameter implies in most programming languages coming (or going) from an external source. If time in your example comes from an external source it would be the parameter.
 
Quote from TSGannGalt:

Because anyone can run stuff like above... I would consider any kind of object within a trading rule as a parameter. Obviously, this is regardless of Price Action or not. So what about the rest of the guys? What's your definition of a parameter?
More...
I'd consider any "magic" numbers to be parameters. Things such as thresholds values, time ranges and intervals, or anything else that over the life of a particular strategy would be a constant. You can certainly argue that different methods can be represented as parameters too. It does eventually get pretty complicated though. For the sake of curve fitting, I suppose that doesn't really matter; somebody could come up with a strategy that involved looking at one symbol in a narrow window and then infer rules from that. That would be curve fitting too, but there isn't much there they could have for parameters under such narrow conditions.

So I guess pragmatically, a parameter is anything that, when refined, runs the risk of curve fitting.

Just about anything, I suppose. :D
 
Quote from Rabbitone:

I would not interpret comparing prices at multiple points of time or a time delta as a parameter.
More...

An all too common error. Magic numbers are magic numbers, regardless of where they come from, or how they are encoded or interpreted. The fact that one might be a constant in one implementation or another simply means the parameter has already been fit to the curve.

Cheers.
 
Quote from TSGannGalt:

Guys...

Let's try to clear each other up with what you consider a "parameter"... Do you consider a "parameter" as a numeric variable? control variable? process? etc. etc.?
More...

I would consider anything that can affect the trading system's ability to generalize well to be a parameter. Apart from the explicit parameters of the model, this would include the selection of symbols, the range of dates for training/back tests etc.

We rely heavily on optimization methods which means that we need to be a bit paranoid when it comes to "curve fitting" (not too fond of the term, I think "selection bias" generally is a better term, at least for our systems).

When we consider a trading model we usually write down everything that could have impact. Then we try to assess the impact of each and how to mitigate the problem.

IMO it is better to be on the conservative side and rather look at something as being a potential problem than ignoring it and as a result having a defunct system.
 
First a thank you to all have contributed. I certainly got a lot more than I bargained for on this one.

A bit more info as to what I am doing, I use wealth lab. I am testing a breakout system with a volatility filter. I wrote the system using a 5 year period of data and 100 random stocks from SP 500 and optimized on a portfolio level.
I then backtested on out of sample data (10 years) on SP 500.
Ive read the posts on peoples differing ideas of a paramter, but in my case I am refering to the period for ATR as part of my entry filter, a period for timed exits and one more for ATR based profit target.
each parameter has relatively good profitablity across the range I optimized, which I think gives me a pretty good margin for error when choosing my parameters to run

I ran the parameters I decided on on this years data and I am now forward testing on a live demo through TWS.
Performance is definately below what I backtested, but its within acceptable levels.

What I am interested in now though, is on any given period one set of parameters will outperform others, and this will change, so if I took 3 or 4 sets of parameters (for example, a 5, 10 and 15 ATR period) and split up my positioning across these 3, would that not help smooth out the equity curve slightly more??

I understand I may be over simplifying the whole process more than some may like, but I am trying to keep the whole process simple, just like my systems. Ive read some of the discussions on testing systems and for the moment its just way over my head.
 
Look around and see if you can find another measure other then straight ATR to measure your breakouits. That alone will be a significant boost to your results.
 
Quote from Random.Capital:

Price action is determined by comparing prices at multiple points in time. Right there you have a parameter - the time delta between observations.
More...

I agree. Let's say you buy if the close is greater than the previous close. Now you need to determine what the *previous* close is. As said earlier, any delta amount of time or volume or price is a parameter. If you adjust that parameter somehow to improve any performance measure, you are already curve fitting.

I think one should not try to avoid any type of parameters or curve fitting at all cost. It is more about basing the strategy on solid and profound tendencies that are likely to remain. The underlying market dynamics and principles you want to exploit are the key to a true edge. The exact way to quantify that market feature is important as well but rather secondary.
 
Quote from Muskoka Joe:

Look around and see if you can find another measure other then straight ATR to measure your breakouits. That alone will be a significant boost to your results.
More...

Are you referring to finding another way to measure Volatility?
 
You must log in or register to reply here.
Back
Top