Value of Backtesting and Stops

If you do proper and rigorous testing of your strategy, then yes, the probability that it will work in a real market is very very high. The 99% figure is a guess and the exact figure is not proveable. But I think you get what I mean.

100% of the systems I have developed so far have been profitable in a live situation. So, so far, im 100%.

If you simulate a system, and it does not work in a real market, then that means your simulation was in error and you missed something or have a typical "peeking" bug that you didnt detect in time.

But like I said before, markets do not suddenly change the very day you fire up your system live. They tend to change slowly over time (some notable exceptions, like when nyse trading rules change, etc, hopefully your system doesnt depend on one of these).

Therefore, I can claim, that if you simulated correctly, the odds are, the edge you detected will in fact be there the next day when you go live, and the next and the next. It may fade away over a period of months or years, but not in a single day.

So in summary, if it simulated well, and trades live like shit, you screwed up your simulation.




Quote from pairsarb:


This is the most important component of a backtest. The poster who said his backtest is 99% guaranteed to work in the future, that could be true (did i just win the lottery?) . But I think there is really no way to prove it, unless he traded all 100% of his 99% accurate systems, in real time, with real money. And what are the odds of that? I smell hyperbole here.
More...
 
Fair enough...I thinks it's easy to say those are your experiences.. Maybe I can learn from what you say. My experience is a little different, and those of some of the better traders that I know or talk to frequently.

Maybe i can open my mind again to a little more testing, but I am not sure if it will lead to a better return.

Good thread. But honestly, I really think if we want to improve our trading results, we might want to rethink attacking everything that is different than what we currently believe. I know this is what I learned from this post. I think this would add credibility to this website as well. I hope to follow this plan in my future posts.
 
Quote from traderdragon:

Anyone who pooh-poohs backtesting is ignorant of good mechanical systems development methodologies.

If you know proper backtest methodology, you can absolutely backtest a system which WILL work in the future 99% guaranteed. Good edges dont suddenly disappear the VERY DAY you start trading them (rolling eyes).

If they do, its because the edge was never really there and you dont know how to backtest properly.

Noobs assume that if you backtest and it fails live, that this somehow proves that backtesting doesnt work. This is purely ignorance and inexperience. There is a lot more to backtesting than simply running a fricken backtest. Christ.

This is like someone picking up a book on how to play a flute, reading chapter one, picking up the flute, making horrible sounds, and then proclaiming that flutes dont work.

Well guess what? Just because you buy tradestation, learn how to load some data, and backtest your first strategy, doesnt mean you know a damn thing about proper backtesting.

I already know how to drive a car, but Im no Mario Andretti. Just because I cant race a ferrari as fast as Mario can, do I have the right to proclaim that ferraris are not fast? Of course not.

I really should shut up and instead, convince everyone that backtesting is a stupid waste of time. I need more ignorant people pooh poohing on backtesting so the edges in the market remain there.
More...
Well said! Testing (back and fwd) is immeasurably useful, if and only if performed properly. Warning: KIDS DON'T TRY THIS AT HOME!!! I say that because the vast majority are unwilling and/or unable to learn how to do it right. Kind of pathetic but that's why there are so many losers... all the comfort of opinion without the rigor and work of analysis.
 
Those of you, who perform tests back and forward may I ask you what your returns are over the last two to three years?

Proper or no proper testing is not sufficient and from what I can see (1mln+ test permutations) majority of programs are deep in a DD and very few combinations are somewhat mildly in the positive land, so please don't be kidding yourself that back testing is the holly grail and easy way to golden temple. Me, thinks it is an illusion!

I don't know why but my returns are much higher discretionary trading then mechanically and I am up over 10 x since 2000.

Don't underestimate my efforts and my due diligence tough. I hired top programmers to work for me costing me thousands of dollars a day to code up the entire testing harness/multi strategy trade entry package, which very few here at ET can afford to do.

Nevertheless I can confirm it is extremely difficult to make it worth trading money mechanically!!! As an order of any significant size (50+ contracts) hits the market I get a feeling of like a ton of bricks are trying to pool it down and this is my conclusion of years in the market watching prices day in and day out. I suspect there are quite many market players who only trade individual positions rather then multiple or portfolio level as many of the so called guru's on this board want you to believe

Buying a testing harness of the shelf for $2,000+ (and I suspect few on this board have interest in selling them) are just not telling the whole true as it is. Some of them would rather start business ventures derived from the sales of their back-testing software rather then put those profits back into trading. Sorry guys but I just can’t stand the whole ponzi scheme?

Hope that helps

:D
 
Quote from WinDiff:

Proper or no proper testing is not sufficient and from what I can see (1mln+ test permutations) majority of programs are deep in a DD and very few combinations are somewhat mildly in the positive land, so please don't be kidding yourself that back testing is the holly grail and easy way to golden temple. Me, thinks it is an illusion!
More...
You can't do it so it can't be done? Now who's deluding who about the "holly grail?"
 
Can someone from the purely discretionairy camp who does ZERO backtesting please tell me what it is they do or look at before putting on a trade???

I am a derivatives trader who only looks at backtests to get the distribution of prices,MAE and MFE.Not concerned with anything past that

I would love to be enlightened...This is a very interesting debate
 
Coming from both backgrounds... (Systematic and Discretionary)

What do you look at when you drive?

Do you just look at the lines on the road?

Do you just look at the stop signs?

Do you just look at the street lights?

=================================

Now... if you had to teach a 16 year old cousin how to drive... how would you?

Would you be on the internet or on the phone talking to her about how to drive?

or...

Would you be in the car next to her as a navigator?

==================================

Finally... when your cousin gets her license, is it because:

You taught her how to drive?

or...

She learned how to drive?
 
Post your returns (all of you) or get lost kids!!!


Quote from Trader666:

You can't do it so it can't be done? Now who's deluding who about the "holly grail?"
More...
 
Quote from WinDiff:

Post your returns (all of you) or get lost kids!!!
More...
Are you really as simple as your posts imply? What would anonymously posting "returns" on the internet prove? Not to mention, who would EVER feel obligated to prove ANYTHING to a jackass like you? Does your mother know you're on ET?
 
You must log in or register to reply here.
Back
Top