There's nothing absurd about what he mentions.
Open up your pre-Calculus (maybe Geometry) book. You'll see where he's coming from in the first 1-2 chapters...
Avoiding Curve fitting
- Thread starter maninjapan
- Start date
Open up your pre-Calculus (maybe Geometry) book. You'll see where he's coming from in the first 1-2 chapters...
Nobody selects a model that leads to negative results on historical data. The model itself may not be an exercise in curve-fitting, but the act of selecting the model most definitely is.
++++
"Proper" pre calculas labels or the mathematians definition of "absurd" is irrelevant.
What is relevant is that in developing trading strategies, any time you introduce an input with a range of possible parameters you will be optimizing your trading system. Under the above scenario "model selection" and "parameter selection", come with the SAME risks as they apply to the practical development of trading systems.
If you don't agree with that, your welcome to that view.
You are now twisting things around, talking about risk rather than curve-fitting. The issue was whether what you described in your posts was curve-fitting or not. You got your answers.
Model selection and parameter selection are two completely different processes. You cannot place those two on the same level. Model selection is a very broad, difficult and vague subject that has no fixed rules and mainly depends on experience. Parameter selection is a specific operation that is well-understood. There is nothing magical about parameter selection. Everyone can do it. There is a lot involved though in model selection. Winners know how to select models, losers know how to adjust parameters. If you cannot see the difference, you are missing a lot
Quote from intradaybill:
You are now twisting things around, talking about risk rather than curve-fitting. The issue was whether what you described in your posts was curve-fitting or not. You got your answers.
Model selection and parameter selection are two completely different processes. You cannot place those two on the same level. Model selection is a very broad, difficult and vague subject that has no fixed rules and mainly depends on experience. Parameter selection is a specific operation that is well-understood. There is nothing magical about parameter selection. Everyone can do it. There is a lot involved though in model selection. Winners know how to select models, losers know how to adjust parameters. If you cannot see the difference, you are missing a lot
============
The actual point I made was on optimization, not curve fitting, and they are two different things as it applies to trading system development.
I have attempted to explain that while model selection and parameter selection mean different things to you, they in fact come with the same risks as it applies to strategy development. As I said you are free to disagree with that.
If it makes you feel better to believe your smarter then someone on an anonynous message board then you can have that distinction. I have considerable experience in trading, and during that time I have seen many quants roll though the door with your same attitude, so it neither surprises or upsets me. I don't have an ego to protect here.
I am going back to observer role now.
I think your assessment of
- what values can be changed "afterwards" and what values cannot
and
- when to view something as one system that gets adjusted and when to label it different systems you select from
is mostly arbitrary.
I could as well say that changing the lookback period of a moving average to improve backtest results is not curve fitting. "(c(1) + c(2) + c(3) + c(4)) / number of summands" is one particular model you select and cannot be changed afterwards. This absurdity can be expanded to all popular indicators that involve any kind of smoothing and even further.
Another example. Let's say you buy if the close is greater than the previous highest high since 1843 or 1844 minutes ago. I think these two entries are so similar, they just have to be viewed as the same model with adjusted parameters. It makes no sense whatsoever to proclaim different, specific and unchangeable models here.
just going to throw this out there guys, but from a trading point of view does it matter what you call it? If I can change a number then I can optimize it, right? Wether its X or N (or A ,B, or C for that matter). if I test a range of 1-5 on any of them, wether its which close to compare to or what % of the ATR to use, Im effectively doing the same thing from what I can see.....
I don't think anyone wants you to go to observer status. You raised some important issues but you should always be prepared for criticism and different points of view.
What is a variable, parameter, dimension, etc. is well defined in math and physics. Optimization and curve-firring are also well-defined. Loose usage of these terms leads to confusion.
That's exactly right. On one side you have a single black box with a parameter covering a range. On the other side you have a large collection of unparameterized black boxes covering the same range of models. It doesn't matter whether you pick one-from-many, or tweak the parameter on the single, the end result is the same.
Curve-fitting to the past.
In one case you do it in the time-honored way, by tweaking a parameter. In the other, you do it with the selection process, which also involved tweaking a parameter, but in a less obvious way.
This is classic getting lost in the trees and missing the forest stuff...