You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
if backtesting works, what could go wrong in real trade?
- Thread starter traderzhangSan
- Start date
NinjaTrader's simulation engine apparently factors in bid/ask size, last trade, time and order delay to determine fill probability.
If you were consistent on the demo in ninjatrader then Is it only the emotional aspect that changes from demo to live? If so, why do people say demo trading means nothing? If not, what other factors come into play (bearing in mind the first paragraph)
cheers.
NewbieBrian
If you were consistent on the demo in ninjatrader then Is it only the emotional aspect that changes from demo to live? If so, why do people say demo trading means nothing? If not, what other factors come into play (bearing in mind the first paragraph)
cheers.
NewbieBrian
Quote from traderzhangSan:
if a backtesting shows a statistically significantly winning odds, you still could lose using the strategy. what could go wrong?
I can think of the following
1. volatility changed
2. you strategy might only work in bull market, now market direction changed.
You are beginning threads frequently.
Volatility and market direction are major considerations in trading.
Therefore, they are always included in any viable trading strategy.
For example, in your last thread you proposed a strategy and left out both of these considerations. As it turned out when they were added to you strategy it became very functional for the reason that including them became an advantage for both the target and the protection you envisioned.
Most people order the degrees of freedom in their models and use that recognised design advantage when the modelling goes into the development stages.
As you begin to learn about markets and trading, think also about what is more important and how it gives you advantage.
As it stands now, apparently, you do not use very many measures of the market, nor do you know that there is an order of events in the markets.
At this point your hypothesis is not in effect in any way based upon this thread and your last thread.
Quote from Brianharvey:
NinjaTrader's simulation engine apparently factors in bid/ask size, last trade, time and order delay to determine fill probability.
If you were consistent on the demo in ninjatrader then Is it only the emotional aspect that changes from demo to live? If so, why do people say demo trading means nothing? If not, what other factors come into play (bearing in mind the first paragraph)
cheers.
NewbieBrian
Your question is a parallel one to the OP's
The OP doesnot include important market parameters in his work (as posted).
During a demo experience, you probably would do the same whereby your mind is just as limited.
When you go live, the market is operating in an unfamiliar way to you. As a consequence when you do not recognize this, you trade and are unsuccessful.
How does one tell if they are equipped or they are doing quality back testing? This is off topic here, of course. Quickly stated it is best for both caes to compare to a standard and I recommend using the market's offer as the standard. you can take a comprehensive list of meausers and put the market's offer into them to get a valuable standard against which to measure.
A major example of the failure to do this are the posts of T zones. He has no viable standard as the basis for his criticisms.
Skills alone can be measured in this way It is a good way to show where and how to focus on improving skills through iterative refinement.
It is very easy to divide the day into the common levels of the market's offer. Knowing the transition poioints is very instructive. The OP seems to think that throughout a given day the paramaters of a market are unchanging. This is far from the facts.
Execution risk, liquidity, etc.
Example: (not taking liquidity into account)
I once saw a back-test for options trading that did extremly well at capturing volatility on spread-trades. The data was solid and everything looks excellent. I could NOT find a single issue with the back-test or model. We tried to break-it, but couldn't.
Then they went into production with the program trading of this strategy and it traded for such a small amount it was useless.
It seemed that when the volatility skew anomaly occurred it was only on a small trades (usually retail small size orders) that there was really very little liquidity at those levels. There was no way to deploy any reasonable amount of money behind the strategy because the order flow wasn't there.
The problem was they didn't take liquidity into consideration. It worked, but only on a fraction of the expected capital that needed to be deployed.
===============================
Example II: (not taking special terms into account)
I help design a covered-call scanner for a trading firm. It worked perfectly, we back-tested the model and it seemed to meet with their expectations. Then it went into production and we took some bad hits on a few trades. Because we had forgotten to eliminate spin-offs, splits, certain deal terms. We corrected the (stupid) error - that we should NOT of missed. However, when we revisited the back-tested results it made some incorrect assumptions on static yield because of special deal-terms. That changed the results considerably that covered-call scanners quickly went into the circular file and I stop working on them in the late 90s.
===================================
Example III: (It works until it doesn't.) I help work on a very large pair's trading system based on spread value. There was actually a pure arbitrage on paper. Unfortunatly the spread continued to widened because the terms of the "b" shares had a provision that allowed for no conversion. Again - this was a fundamental fall - not in the data - but in the legal aspect of the relationship.
-------------------------------------------------------------------------
We try to back-test in the blind.
Start with a hypothsis and then test for it.
However, we also forward test everything as well - to account for execution risk, etc.
I have worked on everything from complex OTC trades, arbitrage, convertable trades, etc. I have made every kind of mistake imaginable and I expect to make more in the future. I do learn from them.
We have some successful trading systems, but for each one we have 10 failures. We keep those failures to learn from them, see WHY they fail, and how to plan for the next test.
Some strategies only work for a particular deal, time frame, or condition - but are EXPECTED to fail over time - because those variables that create the situation are not expected to last.
Example: (not taking liquidity into account)
I once saw a back-test for options trading that did extremly well at capturing volatility on spread-trades. The data was solid and everything looks excellent. I could NOT find a single issue with the back-test or model. We tried to break-it, but couldn't.
Then they went into production with the program trading of this strategy and it traded for such a small amount it was useless.
It seemed that when the volatility skew anomaly occurred it was only on a small trades (usually retail small size orders) that there was really very little liquidity at those levels. There was no way to deploy any reasonable amount of money behind the strategy because the order flow wasn't there.
The problem was they didn't take liquidity into consideration. It worked, but only on a fraction of the expected capital that needed to be deployed.
===============================
Example II: (not taking special terms into account)
I help design a covered-call scanner for a trading firm. It worked perfectly, we back-tested the model and it seemed to meet with their expectations. Then it went into production and we took some bad hits on a few trades. Because we had forgotten to eliminate spin-offs, splits, certain deal terms. We corrected the (stupid) error - that we should NOT of missed. However, when we revisited the back-tested results it made some incorrect assumptions on static yield because of special deal-terms. That changed the results considerably that covered-call scanners quickly went into the circular file and I stop working on them in the late 90s.
===================================
Example III: (It works until it doesn't.) I help work on a very large pair's trading system based on spread value. There was actually a pure arbitrage on paper. Unfortunatly the spread continued to widened because the terms of the "b" shares had a provision that allowed for no conversion. Again - this was a fundamental fall - not in the data - but in the legal aspect of the relationship.
-------------------------------------------------------------------------
We try to back-test in the blind.
Start with a hypothsis and then test for it.
However, we also forward test everything as well - to account for execution risk, etc.
I have worked on everything from complex OTC trades, arbitrage, convertable trades, etc. I have made every kind of mistake imaginable and I expect to make more in the future. I do learn from them.
We have some successful trading systems, but for each one we have 10 failures. We keep those failures to learn from them, see WHY they fail, and how to plan for the next test.
Some strategies only work for a particular deal, time frame, or condition - but are EXPECTED to fail over time - because those variables that create the situation are not expected to last.
you are right.
There is much more to backtesting in order to make your strategy work.
what I am saying is that there is something you CAN do to improve back testing.
for example, to see if your strategy only works in certain market condition, you can do some quantitative analysis, since I have some basic statistic training, you can try to find if your strategy returns has high correlation with market. or you can find if your strategy returns has correlation with vol.
what's max draw down, etc etc
There is much more to backtesting in order to make your strategy work.
what I am saying is that there is something you CAN do to improve back testing.
for example, to see if your strategy only works in certain market condition, you can do some quantitative analysis, since I have some basic statistic training, you can try to find if your strategy returns has high correlation with market. or you can find if your strategy returns has correlation with vol.
what's max draw down, etc etc
Quote from jack hershey:
You are beginning threads frequently.
Volatility and market direction are major considerations in trading.
Therefore, they are always included in any viable trading strategy.
For example, in your last thread you proposed a strategy and left out both of these considerations. As it turned out when they were added to you strategy it became very functional for the reason that including them became an advantage for both the target and the protection you envisioned.
Most people order the degrees of freedom in their models and use that recognised design advantage when the modelling goes into the development stages.
As you begin to learn about markets and trading, think also about what is more important and how it gives you advantage.
As it stands now, apparently, you do not use very many measures of the market, nor do you know that there is an order of events in the markets.
At this point your hypothesis is not in effect in any way based upon this thread and your last thread.
There's no warranty that if an "strategy" worked in the past, it will continue to be profitable.Quote from traderzhangSan:
if a backtesting shows a statistically significantly winning odds, you still could lose using the strategy. what could go wrong?
I can think of the following
1. volatility changed
2. you strategy might only work in bull market, now market direction changed.
As simple as that.
Welcome to the real world.