Degrees of Freedom

[nononsense]

Quote from stephencrowley:

That's a pretty amazing feat nonsense, I believe you are the first person ever to exist without making assumptions.

Maybe you should let everyone know that Gödel was wrong..

Stephen,

I don't like to post too much, only when things get really crazy. You understand the art of loading up big piles of fuming pretentious nonsense - supposedly being related to speculation of course. Skipping the long list starting with neural networks supposedly helping you understand your brain and purportedly helping you trade and terminating now with Einstein's buddy, Kurt, one of the two titans of last century. I was obviously poking fun at your self admitted lack of rigor in constructing your money making gadgets.

PS: you probably meant to type Goedel which is correct and reproduces on all PC screens instead of your Gödel. Given the circumstances, I can almost assure you that Kurt would have liked Goedel better in this spot.

 

[stephencrowley]

Huh? Lack of rigor... thats funny I'm far more rigorous than most, where did I say I wasn't rigorous?

All the rigor in the world isn't going to help if your assumptions are flawed from the beginning.

Again, assumptions must be made at some level, even if you dont consciously realize you are making an assumption. The assumptions that I finally come to, I thoroughly investigate to see if the assumptions match empirical observations.

Even when designing adaptive strategies, you are making assumptions that the method of adaptation you are using is a good enough to match reality.



But no, I meant to type Gödel because everyone should be using unicode by now. I have more than a passing familiarity with some of his stuff.. try reading g.e.b. sometime, not that it will help you in trading, but if you are really a thinker then some of the stuff in it can be insightful, although I found the obsession with sequences to be a little strange.

Quote from nononsense:

Stephen,

I don't like to post too much, only when things get really crazy. You understand the art of loading up big piles of fuming pretentious nonsense - supposedly being related to speculation of course. Skipping the long list of neural networks helping you understand your brain and purportedly helping you trade and terminating now with Einstein's buddy, Kurt, one of the two titans of last century. I was obviously laughing at your self admitted lack of rigor in constructing your money making gadgets.

PS: you probably meant to type goedel which is correct and reproduces on all PC screens instead of your Gödel. Given the circumstances, I can almost assure you that Kurt would have like goedel better in this spot.

 

[mahras2]

I am definately glad that we are getting some responses flying.

Overfitting can be avoided by carefully understanding how you are optimizing, using regularization techniques, bayesian methods, reinforcement learning, etc.

Stephen> Is there any good literature in this subject that may be recommended? That would be interesting to review.

 

[stephencrowley]

Hmm, I get most of my stuff from journals.. and none of the papers really jumped out at me. It was more of a cumulative understanding of hundreds of papers, a few books , coding, playing around with software and developing and intuitive feel of 'how stuff works'.

Just do a google search for 'bayesian' and 'regularization', also, you might look into monte carlo methods.

Be sure not to get a false sense of security based on the confidence intervals you can get from these methods though.. it's still possible to mess stuff up, it just makes it harder.

Learning about different 'information ratios' like AIC, BIC, etc will be helpful. Instead of just using the mean-square-error to determine the fitness, it will penalize the use of extra parameters that dont increase enough relative to other parameters.

stats is only one way to approach the problem

Quote from mahras2:

I am definately glad that we are getting some responses flying.


Stephen> Is there any good literature in this subject that may be recommended? That would be interesting to review.

 

[Thunderdog]

I'm just wondering if all this "science" discussed in this thread is getting in the way of finding a successful trading strategy for some people. I am vaguely familiar with the subject matter being bandied about, and I am certainly a proponent of using the so-called "scientific method" in assessing alternatives. However, I find it interesting that the concepts discussed in this thread have virtually no bearing on how I trade. If I may resort to a rather threadbare analogy in respect of this discussion, in the scientists' quest to understand the forest, is it possible that they have lost sight of the trees? To each his own, I suppose.

 

[vikana]

Quote from mahras2:

I would like to develop some discussion on the optimal number of degrees of freedom a system may have which would not be curve fitted. What has your experience been in playing with different degrees of freedom? How did you go about obtaining the right balance of being optimized so that it can extract profits the system is capable of but not so much that its curve fitted and will fall apart in the future?

I look at it a little bit in reverse (from your question). The data is what determines how many degrees of freedom you realistically can have in a system.

So if you take e.g. the SP various measurements of dimension will give you an answer from 1.5 to about 4. I'm not sure there really is any one answer btw.

So if the the SP lives in a 4 dimensional space, you should limit the number of independent conditions to 4 in order to stay safe. The good news is that many types of conditions are "roughly" identical, such as macd() and stoch(), so they probably don't count fully as independent variables.

Personally, I always try to keep no more than a few conditions per signal - that simply seems most safe.

 

[horribilicus]

Quote from Thunderdog:
I'm just wondering if all this "science" discussed in this thread is getting in the way of finding a successful trading strategy for some people. I am vaguely familiar with the subject matter being bandied about, and I am certainly a proponent of using the so-called "scientific method" in assessing alternatives. However, I find it interesting that the concepts discussed in this thread have virtually no bearing on how I trade. If I may resort to a rather threadbare analogy in respect of this discussion, in the scientists' quest to understand the forest, is it possible that they have lost sight of the trees? To each his own, I suppose.

The ultra-pragmatist wing of the church of mechanical trading systems often uses a rough rule of thumb as follows:

• Make sure your simulation tests produce AT LEAST 200 trades per parameter in your system's rules.

Don't trust a 3-parameter system unless you've got >600 trades of backtesting results.

Different priests recommend different coefficients (some say 100, some say 300) but the basic methodology is used surprisingly often.

 

[nitro]

The people that could answer that question in earnest don't come and post on ET.

The stuff being discussed here is completely elementary. The tools and ideas needed are far beyond, and I mean faaar beyond, what anyone has ever discussed on ET.

nitro

Quote from Thunderdog:

...However, I find it interesting that the concepts discussed in this thread have virtually no bearing on how I trade. If I may resort to a rather threadbare analogy in respect of this discussion, in the scientists' quest to understand the forest, is it possible that they have lost sight of the trees? To each his own, I suppose.

 

[stephencrowley]

Quote from nitro:
The stuff being discussed here is completely elementary.

Of course it is, who in their right mind would want to discuss the real details of their system?

Basic questions need basic answers, it helps to understand this stuff but it's not gonna get you there by itself.

And yes, ET is a bit too low-brow (not aimed at any of you guys in this thread) and full of trolls to discuss anything even remotely complicated. There are better forums for that, and a lot of the time stuff like that is discussed privately between individuals.

Think about it, no one gets into the details of string theory when they are writing some news blurb about some new research, to get to the guts you gotta have a more one-on-one type of interaction with people who really know the details.

 

[Thunderdog]

Quote from nitro:

The people that could answer that question in earnest don't come and post on ET.

The stuff being discussed here is completely elementary. The tools and ideas needed are far beyond, and I mean faaar beyond, what anyone has ever discussed on ET.

nitro

Not everyone wears the same hat in the same way. If that's what works for you, then that's just peachy.