Backtesting and Strategy Validation - How Much is Enough?

[DT-waw]

and how about central banks with their vast resources?
they defend currencies rates and lose huge amounts of money due to political insanity.

how about 30,000 corporations who trade on markets like crazy clowns? people with big money usually are the worst traders. i've seen private clients lose tens of millions $ trading like a total jerks.

and you're trying to tell me i cannot find simple rules which beats these crowds of delusional greedy and fearful manual traders?

 

[Buy1Sell2]

Quote from Algo_Design_Kid:

There are so many inherent flaws in backtesting.

??? huh ???

 

[intradaybill]

Quote from Buy1Sell2:

??? huh ???

Kid is correct, although just a kid. I know of 8 major backtesting flaws and about 4 royally flawed backtesting programs. There was a paper posted once here but I can't locate the link about major flaws in backtesters that fooled traders big time. There is a company that made a name and a fortune out of selling a screwed up backtesting program.

 

[Algo_Design_Kid]

Quote from Buy1Sell2:

??? huh ???

Slippage, orders being filled, incorrect data, slippage not correctly being accounted for is probably the biggest killer. Don't forget about adding commissions etc..

If you account for even an average of .5 tick in the ES slippage it will most likely turn any short term trading program upside down. Good luck in the live environment. It will happen to you.

 

[Buy1Sell2]

Quote from Algo_Design_Kid:



If you account for even an average of .5 tick in the ES slippage i

??? huh ???

 

[Buy1Sell2]

Quote from intradaybill:

I know of 8 major backtesting flaws

??? What ???

 

[Random.Capital]

Quote from Buy1Sell2:

??? huh ???

I may be able to answer your question, but you'll have to be slightly more specific about what it is you're trying to ask.

 

[Algo_Design_Kid]

Quote from intradaybill:

Kid is correct, although just a kid. I know of 8 major backtesting flaws and about 4 royally flawed backtesting programs. There was a paper posted once here but I can't locate the link about major flaws in backtesters that fooled traders big time. There is a company that made a name and a fortune out of selling a screwed up backtesting program.

Backtesting really hurt my entire psyche. I spent years on this, bragging to people I know, only to fall flat on my face and well in debt.

Most of it was CFitting. After I overcame that aspect I came to find out that the smallest problem in your code or market conditions will manifest itself x100000 in the live environment. Don't think you will have 0 problems. That is truly laughable.

Trust me. I am not saying it isn't possible. It is, maybe .0005% of all my work over the past 4-5 years ( many hours a day ) actually still makes money. Not the programs that I tried to stake my claim though. Those failed miserably and put me in a tight spot in life.

It isn't that great to have to move in with your parents after totally falling flat on your face. I am working my way out now.

Best thing you can do for yourself is learn how to program, I mean be a professional, and just make your 250,000/k a year at a HFT firm that doesn't mind losing money with venture capital.

At least you will get paid. Let them take the risk. There are a lot of people with a lot of money that think the money automatically makes them smart. That story is a dime a dozen in this industry, trust me I have ran into those types as well.

Sorry to lay it out like that. Trying to save others the pain.

 

[Argent]

Quote from Algo_Design_Kid:

Backtesting really hurt my entire psyche. I spent years on this, bragging to people I know, only to fall flat on my face and well in debt.

Most of it was CFitting. After I overcame that aspect I came to find out that the smallest problem in your code or market conditions will manifest itself x100000 in the live environment. Don't think you will have 0 problems. That is truly laughable.

Trust me. I am not saying it isn't possible. It is, maybe .0005% of all my work over the past 4-5 years ( many hours a day ) actually still makes money. Not the programs that I tried to stake my claim though. Those failed miserably and put me in a tight spot in life.

It isn't that great to have to move in with your parents after totally falling flat on your face. I am working my way out now.

Best thing you can do for yourself is learn how to program, I mean be a professional, and just make your 250,000/k a year at a HFT firm that doesn't mind losing money with venture capital.

At least you will get paid. Let them take the risk. There are a lot of people with a lot of money that think the money automatically makes them smart. That story is a dime a dozen in this industry, trust me I have ran into those types as well.

Sorry to lay it out like that. Trying to save others the pain.

You've heard of survivorship bias? Well, you're the opposite: failure bias. Don't lay all your flaws on the method. It works if used properly.

 

[Buy1Sell2]

Quote from Algo_Design_Kid:

Backtesting really hurt my entire psyche. I spent years on this, bragging to people I know, only to fall flat on my face and well in debt.

This an issue with your risk/money management. Not with backtesting