Backtesting

T

traderich

I am reading several posts on here about backtesting set-ups and so forth.

Is there any credibility to backtesting to determine if the same set-ups or strategies will work in the future?

Having spent his fair share (or perhaps way too much!) time in a casino or 2, I understand full well what the experts mean when they say "the dice or roulette ball have no memory" .

Is the same true for the stock market?

Can you look at past results and charts etc. etc. and determine with any kind of reliability of what will happen in the future?

I need to do more research on this, but was curious what others on ET think about this, first hand or otherwise.
 
Quote from traderich:

I am reading several posts on here about backtesting set-ups and so forth.

Is there any credibility to backtesting to determine if the same set-ups or strategies will work in the future?

Having spent his fair share (or perhaps way too much!) time in a casino or 2, I understand full well what the experts mean when they say "the dice or roulette ball have no memory" .

Is the same true for the stock market?

Can you look at past results and charts etc. etc. and determine with any kind of reliability of what will happen in the future?

I need to do more research on this, but was curious what others on ET think about this, first hand or otherwise.
More...

I think you should look at backtesting as a way to develop or fix your methodology instead of trying to use it to determine what will happen in the future.

First, backtest any strategy ideas you may have.

Next, after you think you have a something that looks promising...

Trade with a simulator just long enough to get some decent statistical data...make more corrections or go back to backtesting.

Next, you do the same with real money via a small size and keep the same type of statistical info as you did for the simulator trading and backtest results.

* Backtest Stats

* Simulator Stats

* Real Trading Stats

The three above when combined should help you determine with more reliability about the future in comparison to backtest stats all by itself.

Simply, you'll be way behind the game if you rely on backtesting results all by itself.

P.S. My opinion above is that of someone that's not using an automation trading system.

Mark
 
I forget the name of the book, but I read about soem MIT/Stanford (don't remember which), who went to a casino, figured out when they rebalanced the roulette wheels, then charted teh outcomes for 4 days and then play for teh next 3. After charting the wheels for 4 days they were able to pick up the slight imbalances in tgeh wheels. Then they would just play the ones withe the best odds. (I think they rebalance the wheels much more frequently now.

I guess what i'm saying is your right taht the market has no memory, but all you are trying to do is find the situation where you have a 3% edge and then exploit it.
 
Backtesting will never replace REAL experience in the mkt.

Backtesting is not even a shortcut. Now experienced traders might realize more about what to look for in backtesting, but a newbie is still a babe in the woods .
 
Quote from traderich:



Can you look at past results and charts etc. etc. and determine with any kind of reliability of what will happen in the future?

I need to do more research on this, but was curious what others on ET think about this, first hand or otherwise.
More...
=============
T-rich

People in markets have memory;
even the old traders have old charts/data, so thier memory doesnt have to be exactly photographic. LOL:D

No , back testng hints or probabilities are not predictions;
but a 50% off sale can bring/bring in sellers/buyers nicely.
Wisdom is profitable to direct.:cool:

Murray TT
 
I think ther real question regarding backtesting is how do you backtest all the emotions that were also tied into the past historical data.

How do you backtest the theory about " you are your own worst enemy ".

backtest, bah humbug .. :)
 
I think some of you are missing the main point here. System developers and Traders don't say that just because a system backtest well it will work in the future. There is a lot of different types of analysis you can do which will help you analyze the probability that it will work in the future.

Here is point number 1 about backtesting

The first and most important point is that if the strategy has not worked well in the past it is not wise to trade it in the future. Many strategies which non system traders trade with blind faith , when backtested do not make money!.
 
Backtesting can't tell you what will work in the future, but it can tell you what did (and didn't) work in the past. I don't think it's useful beyond an automated system -- you can't simulate emotions but you can simulate a mechanical system (provided you are willing to trade it mechanically and not mess with it). But a good backtest doesn't mean it will work in the future.....
 
roulette wheels are designed and tested specifically so there is no memory.

the market however is designed to faciltate trade, not gambling. if there are opportunities to benefit from the push and shove of the market, why wouldn't backtesting allow someone to see those opportunities?

the real issue with backtesting is doing it in a way that gives you some idea of how the system will actually trade. there are many issues which can distance backtest results from reality, just one of them is if the market has no memory of the system. the great results you got were fleeting and dont continue into the future, which is very familiar in backtesting.

thats why most backtesting relies on 'out of sample data', ignoring the results you got from the data you used to create the system. instead if your system has memory then it will work just as well on data that wasn't used to create the trading rules.
 
toe and Murray's posts are the only one's that have sound logical to me here.

For sure, a back-tested idea may or may not have sufficient edge in the market going forward but yet it is absolutely essential practice to back-test (I'll explain my position in a sec)

For sure, an idea that after (thorough) back-testing shows to be inadequate should be discarded (or amended) and there is real value in knowing what you thought "worked" really doesn't.

The counter point that other factors come into play in real life trading that can't be re-enacted in a back-test no doubt is true. But, if you are hanging your hat on those non-quantifiable variables to trade with IMO, you are in dangerous territory. If you can't translate those "je ne sais quoi" things into a testable statement to backtest then you can never know what is actually giving you your edge, and therefore you can't know if that edge is going away over time as edges do erode (gut felt ones as well as back-tested ones)

The "my $ P&L tells me" doesn't cut it. You never know during a specific window in real time trading how much of your $ P&L is due to random noise in the market versus real edge. That info only becomes clear over a longer time period. My 2 cents.
 
You must log in or register to reply here.
Back
Top