Value of Backtesting and Stops

Quote from inflector:

Anyone who says that testing using historical price data has no value is not thinking clearly.

How does one arrive at any strategy without knowledge of the past? How do you determine when to buy or sell?

What are the alternatives? Guessing?

All we have is what the markets have done so far. Even if we are trading live as a discretionary trader using only our personal experience as a guide for what to do in the future, we are relying on subjective back-testing.

It's not hard to argue that computers can more reliably test ideas using the same data. Backtesting works for the same reasons that smart traders can develop systems after years of experience in trading. They note repeating patterns that offer opportunities for profits. They then trade using ideas designed to capitalize on those opportunities.

You can do the same thing with a computer but with a more rigorous analysis. You will often find that ideas that appeared promising won't work because of something that you did not anticipate. It is much better to find this out using a computer than an actual account.

The reason that some people shy away from backtesting is that there are many ways to screw up backtesting and to find methods that appear to work that won't actually work in real markets. These problems are tractable if you pay attention to avoid the most common pitfall: over optimization.

Just because someone can use a knife to slit their wrists doesn't mean you don't want to use one in the kitchen when cooking. You just need to be careful with sharp instruments.

- Curtis

Before you open your mouth Mr. Turtle post your trading performance over the last2 years!


:D
 
Quote from traderdragon:

When someone knows they are beat, and cant defend their position, he often just runs away like a child :D

Actually, youre wrong, he only had 1 million in the trust. So yeah, he didnt exactly die broke, he only lost the majority of his fortune.

But more importantly, you didnt bother to address any of the real points I made, because you cant.

People who dont posses the skill to backtest properly often scoff at it and try to convince themselves its worthless. The blind trying to convince themselves that being blind is ok.




:D

What do you know about if I can backtest or not? Before YOU open your mouth about that - I made a profession out of that and have forgotten more than that you'll ever know.

Now, if you want to know why backtesting is useless then you have obviously not even realised the automated order flow analysis systems will completely throw havoc with your results.

And then I am not even talking about the autospreaders and spoofers / flippers and the games that the hedgefunds play with the spoofers / flippers.

The markets are an interactive system that morphs the moment you put a trade of decent size on. Or you think the market is not affected when GS puts an order of 10000 cars in the Bunds?

Keep on dreaming with your one lot trades!

I run circles around guys like yourself but you'll never know.

Holmes (and this time definitely over and out)
 
I think there needs to be a distinction made between backtesting and being a 100% pure mechanical trader...

First and foremost.Everyone, in every profession performs some sort of backtest if they have a modicum of intelligence.To say backtesting is unneccesary or useless is naive at a minimum,and pretty dam close to deusional..

I cant think of a professional sports team that doesnt employ a statistician is some manner to "scout" the opponent...

Eyeballing charts is a crude form of backtesting..

Trying to discern patterns or trading off candlesticks should involve a form of backtesting....


I am not even sure as to what the debate is about
 
Quote from inflector:

The reason that some people shy away from backtesting is that there are many ways to screw up backtesting and to find methods that appear to work that won't actually work in real markets. These problems are tractable if you pay attention to avoid the most common pitfall: over optimization.

- Curtis
Curtis, this describes my experience exactly. I guess it's because I'm a very analytical person and the computer lets you try a large number of variations in a short period of time, which I became fascinated with. The problem I ran into was that after optimizing several variables, I lost a bunch of money paper trading. So I took a look at the trades the optimized system executed and they were dumb, but lucky, if you know what I mean. I would triple my money in six months, but only if the conditions were repeated exactly as in the past, which is pretty unlikely. As a result, this weekend I went back to the drawing board, threw out the multiple indicators and started using a very simple system that seems to be working so far paper trading. It back tests ok, but more importantly I understand it and I can see relatively soon when it's not working. This is another thing Van Tharpe suggested--keep it simple and don't use multiple indicators that just present the same data in different ways.

Chuck
 
When it comes to short-term trading backtesting has value but I dont recommend going too far back. Once I anticipate an edge then I forward test for at least 6 months to check the validity. Worse case scenarios (slippage/commisions) are also priced into the tests. I am also a firm believer in hand testing. It really beefs up your chart reading skills. People have been trading the same patterns for years (Dips, Breakouts, Narrow Range Bars, etc....)
The key is knowing when and where to find the action and trading multiple vehicles as well.

This is what Traderdragon is referring to when he mentions proper backtesting. Maybe he does it a little differently then I do but whatever it is we do seperates us from the backtesting masses. Backtesting is an art and takes years to learn. One thing I stay away from is backtesting Long Term Trend following systems since it takes years to forward test them. These systems are usually pretty simple since all you are using is technical data (MA's, RSI, MACD, ADX, Charts, Candles,etc...) and probably explains why most fail and have unbearable drawdowns. Most of the backtesting public want to sit on their hands and make money but unfortunately there is no free ride in the markets. A good tester/trader can make the same returns (or more) in the short-term as someone trading longer term systems but has to really put the time in and will also benefit from a very low drawdown. He also has the luxury of knowing early when to pull the plug and has a back-up system/plan ready to go. This is what they call shifting your style.

TraderDragon

All I can say to you is that most of these guys will never get it. Dont bother trying to explain it to them. No offense to anyone here. Different strokes for everyone I guess. I calculate risk and testing is the best way for me to do this. It has made me a better trader and also keeps my stress levels down :)
 
Quote from bluedemon77:

I was very impressed with Van Tharp's book, but he kind of pooh-poohed the value of backtesting. I guess people assume that any system that backtested well will do well in the future and he wanted to destroy that illusion. However, without backtesting and paper trading, what else can you do besides plunk down your cash and hope for the best?


Not backtesting is silly. Remember, even something as basic as "buy and hold" is based on a backtest.

Fletch
 
I think you folks have to understand.

Backtesting is problematic, because the majority of ET members lack the skills to properly execute tests, and evaluate the result.

In the hands of an incompetent amateur, woodworking tools will produce a piece of wooden crap. In the hands of a skilled cabinet maker, the result is a piece of fine art.

I could say more, but it is about that simple.

Steve
 
Everyone keeps going on about backtesting but no-one has mentioned the benefit of forward testing.

Forward testing to me is the most important element in developing a profitable strategy.

At any one time, I am forward testing at least 7 or 8 strategies. If a system is profitable in real time with market entries, then I will look at refining it further through methods such as integrating multiple parcel entries back from the current bid/ask in an attempt to get a better average entry price on the total position.

Obviously, the longer time frame strategies are better for this but I'm mostly looking to take the major daily swings.

Runningbear
 
Like it or not Back testing and Forward testing are no different and both activities fall under the umbrella of so called historical data fitting. My advice to you is, be very very careful with the machinery, it may eat into your pockets rather quickly, much quicker then you can think!!!

:D


Quote from Runningbear:

Everyone keeps going on about backtesting but no-one has mentioned the benefit of forward testing.

Forward testing to me is the most important element in developing a profitable strategy.

At any one time, I am forward testing at least 7 or 8 strategies. If a system is profitable in real time with market entries, then I will look at refining it further through methods such as integrating multiple parcel entries back from the current bid/ask in an attempt to get a better average entry price on the total position.

Obviously, the longer time frame strategies are better for this but I'm mostly looking to take the major daily swings.

Runningbear
 
Unfortunately for some amateurs computer backtesting is all they think they have to get a handle on the market. They believe in it regardless.

The point is that basic OHLC data is skeletal and primitive as an input. But that is what is basically being relied on.

Incidentally few or none are suggesting that historical market data is not important. Even if you only look at back charts that is what, underlyingly, you are looking at.
:)
 
Back
Top