Value of Backtesting and Stops

[HGTrader]

To say backtesting market data is not that important in "discovering" a career making profitable edge, is like saying a large quantity of data to be used in research is NOT that important in "discovering": the cure for cancer/how to clone animals/the causes of shizophrenia/etc.


If someone is going to rely on forward testing alone, it will take 10 years to accumulate 10 years of Data, and they have to forward apply dozens and dozens of strategy ideas concurrenty.

Backtesters on the other hand, don't have to wait 10 years, they can instantly drugde through years of data in a few hours/days/or weeks, and they can focus on checking ONE strategy/idea at a time.

That being said, because of factors such as 1)humans percieving meaning in random chaos, 2)the absence of realtime factors/behaviors
3)waning attentiveness to pertinent detail while looking at chart after chart, there WILL BE PLENTY of strategies that will seem
to have backtested successfully, but turn out to be a major dud once tested with real money.

 

[traderdragon]

First he states that proper testing is not sufficient, and then states he spent thousands of dollars a day to get programmers to test for him.

Ummmmm ok???

Another poser who doesnt know what he is talking about.

Quote from WinDiff:

Those of you, who perform tests back and forward may I ask you what your returns are over the last two to three years?

Proper or no proper testing is not sufficient and from what I can see (1mln+ test permutations) majority of programs are deep in a DD and very few combinations are somewhat mildly in the positive land, so please don't be kidding yourself that back testing is the holly grail and easy way to golden temple. Me, thinks it is an illusion!

I don't know why but my returns are much higher discretionary trading then mechanically and I am up over 10 x since 2000.

Don't underestimate my efforts and my due diligence tough. I hired top programmers to work for me costing me thousands of dollars a day to code up the entire testing harness/multi strategy trade entry package, which very few here at ET can afford to do.

Nevertheless I can confirm it is extremely difficult to make it worth trading money mechanically!!! As an order of any significant size (50+ contracts) hits the market I get a feeling of like a ton of bricks are trying to pool it down and this is my conclusion of years in the market watching prices day in and day out. I suspect there are quite many market players who only trade individual positions rather then multiple or portfolio level as many of the so called guru's on this board want you to believe

Buying a testing harness of the shelf for $2,000+ (and I suspect few on this board have interest in selling them) are just not telling the whole true as it is. Some of them would rather start business ventures derived from the sales of their back-testing software rather then put those profits back into trading. Sorry guys but I just can’t stand the whole ponzi scheme?

Hope that helps

 

[science_trader]

Quote from bluedemon77:

I was very impressed with Van Tharp's book, but he kind of pooh-poohed the value of backtesting. I guess people assume that any system that backtested well will do well in the future and he wanted to destroy that illusion. However, without backtesting and paper trading, what else can you do besides plunk down your cash and hope for the best?

Anyway, I'm working on a system that looks very promising based on backtesting. My problem is I am faced with conflicting philosophies. I mean my backtesting indicates that applying stops at any level will produce worse results in both bottom line and system drawdown than just following the buy/sell signals of the system. However, the "2% rule" says you shouldn't lose more than 2% on a trade and in some cases without stops the trade will lose 4-5%. But if I use stops, it reduces the expected profits by like 10-20% depending on how tight the stops are.

Any thoughts?

Chuck

It is normal that stop-losses destroy value. In order to enhance a strategy using stop-losses, you need a distribution of strategy returns that is bi-modal (i.e. with two peaks) or very strong auto-correlations.

If you don't want to take risks, don't trade ! The "rules of thumb" have just no value.

 

[science_trader]

Quote from WinDiff:

Those of you, who perform tests back and forward may I ask you what your returns are over the last two to three years?

0.4% a day, Sharpe ratio (if one can rely on that) around 5...

Any more comments about backtesting ?

 

[bluedemon77]

Quote from science_trader:

It is normal that stop-losses destroy value. In order to enhance a strategy using stop-losses, you need a distribution of strategy returns that is bi-modal (i.e. with two peaks) or very strong auto-correlations.

Sci, please explain further. When you talk about bi-modal, what variable is on the x-axis assuming profit is on the y-axis? Also, I understand the concept of autocorrelation , but what is the significance of that in this case?

Chuck

 

[AskQuestions]

Quote from bluedemon77:

Sci, please explain further. When you talk about bimodal, what variable is on the x-axis assuming profit is on the y-axis? Also, I understand the concept of autocorrelation , but what is the significance of that in this case?

Chuck

if i may answer.
he means that if the profit histogram (i.e frequency of profits) is bimodal u would benefit from using stop - because u'll be able to "stop" the lower peak of the histogram.
on a gaussian distribution u'll probably get worse results using stop.

 

[slowBear]

traderDragon:

For someone who is trying to learn how to backtest properly, could you suggest some books, etc.?

 

[waxwing]

Quote from science_trader:

It is normal that stop-losses destroy value.

I agree with this both empirically (have observed it often) and philosophically (imposing a target price on the market has no value *to the market* (only helpful to your risk management calculations), so just imposes transaction costs).

But:-

In order to enhance a strategy using stop-losses, you need a distribution of strategy returns that is bi-modal (i.e. with two peaks) or very strong auto-correlations.

This is interesting to me; on what basis do you make that statement? Is it empirical or theoretical(mathematical)?

For example if we have a histogram of returns with one peak at, let's say, +$1000 and one at -$1000, it doesn't follow that by setting a stop somewhere around, let's say, -$500, we will increase the expectancy of the system, as far as I can see - the price paths are hidden in histograms.

I also don't know exactly what you mean by saying that autocorrelations (in time series of returns?) give value to stops, but maybe I'm being dumb there

 

[pepper_john]

Quote from science_trader:

0.4% a day, Sharpe ratio (if one can rely on that) around 5...

Any more comments about backtesting ?

Vow. Yearly return of more than 200% with Sharpe ratio of 5. Is this for single-strategy or a portfolio of strategies?

 

[taowave]

Quote from pepper_john:

Vow. Yearly return of more than 200% with Sharpe ratio of 5. Is this for single-strategy or a portfolio of strategies?

If i am not mistaken that would be apx 100% returns unless you are factoring in compounding...

none the less,outstanding