Optimization, curve-fitting and probability
- Thread starter NinjaTrader_Dierk
- Start date
NinjaTrader_Dierk
ET Sponsor
Easy guys, please ...
See, I'm a real freshman concerning trading. So I appretiate any comment related to this topic. I understand, that oppinions and experiences are quite different, but that's what I appretiate here.
If I only want to see my own opinion confirmed, I just have to look into the mirror every morning - provided I can stand the view
Dierk
See, I'm a real freshman concerning trading. So I appretiate any comment related to this topic. I understand, that oppinions and experiences are quite different, but that's what I appretiate here.
If I only want to see my own opinion confirmed, I just have to look into the mirror every morning - provided I can stand the view
Dierk
Don't forget to turn it one when you start trading. You need it.
LOL!
So I AM the big fool now, am I? And I didn't read properly???
Actually, it's exactly the other way around. This has actually been said by this nononsense:
"The talk about fast scalpers, I hate this fuzzy term - my computer does not understand it - talking about their 1, 2 and 3T's in trying to "squeeze" out of something does not sound right to me. I can tell you that I indeed look only at very short trades and very many of them. To give you an idea: 500 RT's in a typical day. If I try to do really what my simulations showed me to be interesting, I go nuts after only 5 minutes sporting on my keyboard." - nononsense
If you need a little more help with your mild delusion, here's the link; http://www.elitetrader.com/vb/showthread.php?s=&threadid=21340&perpage=6&pagenumber=8
Does any of this stuff make sense to anybody here? Come on! Honestly!
Stop posting nonsense, pul-lease. Stick to just reading for 1/2 year minimum, and when you know a little more, you can start posting. Look at my history - You'll find that I did my first post 6 months after registering! You started posting straight away - As a valve for your own frustration.
Have you started your psychotherapy yet? It has been widely recommended that you should do so. Don't just ignore it, for your own health's sake. Also, I told you several times now that you shouldn't start war with me. I uncover your lies in a flash everytime - You don't seem to learn very fast. You better mess with someone like FPC. You have no chance with me, you just embarrass yourself even more everytime. My god, You should know that by now. Make the best of it.
Best Wishes and Peace,
~The Scientist
The following was put on ET by "scientist".
"I've actually got some good relations to the moderators, and I get them to delete your posts all the time - You didn't notice? "
I refuse to have any further dealings with such a character.
nononsense
As a further footnote, "scientist" once explained us that he was sent over by his parents from Australia to Germany to get what he called an "elite education" (sic). Judging by his writings this German education could not fully remedy his earlier flaws. This is undisputable from a recent sample of his puerile logic:
"Knowledge is knowing what you don't know "
["Started using Pivot Points ..." thread, 08-29-03]
Ever since he hangs around ET as a kid trader.
"I've actually got some good relations to the moderators, and I get them to delete your posts all the time - You didn't notice? "
I refuse to have any further dealings with such a character.
nononsense
As a further footnote, "scientist" once explained us that he was sent over by his parents from Australia to Germany to get what he called an "elite education" (sic). Judging by his writings this German education could not fully remedy his earlier flaws. This is undisputable from a recent sample of his puerile logic:
"Knowledge is knowing what you don't know "
["Started using Pivot Points ..." thread, 08-29-03]
Ever since he hangs around ET as a kid trader.
Remember one thing. Although you believe that once you code a system anew and performed no optimization in the sense of a walk forward type analysis, that would be incorrect statistical thinking.
Every time you build a new system, or in some way optimize against a data set, ALL NEW SYSTEMS AND ALL NEW OPTIMIZATIONS SHOULD PAY A PENALTY.
The reason is obvious. Even when you write a new system, althought the system has never been optimized against the data, every time YOU have written a system against the data set, you are optimizing YOURSELF, which in turn are becoming more and more curvefitted to writing winning systems on that data. The correct way to really test this is not only to hold back data from the system, but hold back data FROM YOURSELF that you will never see. Once you have seen that data, you should have some sort of utility penalty function that subtracts some profit from the system if you write a new system against that data, or optimize against it in any way. The idea is ALWAYS to have at least a year of data that you have NEVER seen.
Ideally, you should walk to a dataset that you have never seen before with a system already in hand, run it against the data, and show a statistical significance. Otherwise, t-test and everything else that show that "This result has a 1.84% probability of being random" is an abuse of statistical inference.
nitro
Every time you build a new system, or in some way optimize against a data set, ALL NEW SYSTEMS AND ALL NEW OPTIMIZATIONS SHOULD PAY A PENALTY.
The reason is obvious. Even when you write a new system, althought the system has never been optimized against the data, every time YOU have written a system against the data set, you are optimizing YOURSELF, which in turn are becoming more and more curvefitted to writing winning systems on that data. The correct way to really test this is not only to hold back data from the system, but hold back data FROM YOURSELF that you will never see. Once you have seen that data, you should have some sort of utility penalty function that subtracts some profit from the system if you write a new system against that data, or optimize against it in any way. The idea is ALWAYS to have at least a year of data that you have NEVER seen.
Ideally, you should walk to a dataset that you have never seen before with a system already in hand, run it against the data, and show a statistical significance. Otherwise, t-test and everything else that show that "This result has a 1.84% probability of being random" is an abuse of statistical inference.
nitro
Quote from droth:
Thanx for all your replies. I like "extended backtesting is mental masturbation ..." most
I have to admit that I'm a bloody newbie, so my points could be somewhat academic ...
What distracts me most is, that I don't know when I cross the small border line between proper system setup and
system over-optimization. In fact, I dislike the term "optimization", since it draws the attention to the wrong point.
I mean: Trading systems should reflect market characteristics and "hopefully" they are profitable. I have the feeling, that I would be better off
looking for approaches, that reveal some kind of "persistence" in the markets. That's where the idea of probability fits into the
picture. If I was where able to find high probability systems (systems that yielded their results - negative (!!) or positive - not just by chance),
"rational" decisions could be made to trade them or not (based on the performance they showed in past). It would be interesting to see if there is
an edge in this idea.
I feel uncomfortable to just "select" some kind of parameter set for a strategy and reject others. I asked myself if there are better (or additional!!) ways then checking by
out-of-sample tests -which are still absolutely necessary. Parameter sets may show past profit or not, as long as the results are not significantly different from random,
they are meaningless. Am I wrong ?
@GIG
Do you have the feeling binom CDF shows more than the method Katz/McCormick offer ?
@CalTrader
The idea of incrementally improve analysis based on seen/real results stands. I will try to stick to it.
The thread
http://www.elitetrader.com/vb/showthread.php?s=&threadid=1019&perpage=6&pagenumber=1
is useful. I noticed it before and just am rereading it.
Dierk
Words from a successful optimizer:
1. minimize the number of variable(KSS)
2. very the length of the in-sample date
3. always walkforward realistically
1. minimize the number of variable(KSS)
2. very the length of the in-sample date
3. always walkforward realistically
LOL! Now you've repeatedly posted this ... HOW many times? 5? And how many times did I tell you that I or anybody else cannot do this? Equally often.
I repeat this again, to hammer this into your numb skull: Your posts were constantly being deleted because 95% of them were FLAMES. The moderators DELETE THEM. Not me. You're ET's #1 flamer today, and you still wonder??? Jesus, how thick must you be?
Haha. No, this is actually a part-result of my "elite" education. It's called "literacy". You (as we know) obviously don't have much of it, otherwise you'd know that this wasn't even said by me.
You have just AGAIN embarrassed yourself badly, because there are millions of people who know this wisdom.
I told you not to try to defend yourself by attacking me, since it always backfires, laying bare your low intellect. You're doing it again. You're a real pain in the rear man, but at least your consistent blue-eyedness makes me laugh. You're like a slapstick comedian. You get slapped everytime again, and yet still not get it, it's so funny. The difference is that Chaplin was actually smart in real life, and he got paid well for it. You aren't, and won't.
I wish you the best on your quest to bettering.
Be Well, Brother
~Scientist