Limitations in backtesting

Quote from hypostomus:

...thanks for explaining. I think I understand what you mean. I do all my backtesting so that the entry/exit decisions are made at the end of the current bar, and trade with market orders at the open of the next bar. I once modified one of my profitable codes for mid-bar decisions and entries/exits using limit orders, and got better results, assuming that the limit orders would be filled.

However, if I am understanding you correctly, the discrepancy between the backtested performance of a system and its real-time performance with mid-bar entries could be resolved by coding and testing mid-bar entries and exits, trying as best you can to write algorithms for where your intuition/experience tell you to trade.

Also, can you elaborate on signals that appear in real time that don't occur in the data feed? If you mean those due to one tick variations due to data base time differences, then I understand.

Apologies if I come across as dense and argumentative (it's hard-wired in). I always enjoy exchanging messages with you because I learn more from you than you learn from me!
More...

Monday, put up a stochastics oscillator. Use any time frame you like. Note the numerical position of the osscilator 1-2 sec. before the end of the bar, and then note it at the beginning of the next bar. There will be dramatic displacement.

This will adversely affect a backtest.

Connie Brown discusses this in her book.

Regards
Oddi
 
...thanks for the explanation. Does that have to do with the famous end-of-bar follies, allegedly designed to fake out EOB traders (perhaps it was you who first noted that here)?
 
Imposing an end of bar trade on a market is probably not going to represent reality, the market is pretty fluid and bar intervals are dogmatic and arbitrary. The dogma is bound to diverge from the reality all the time.

 
...the reason for decision making on the the end of the bar is that the transition from one bar to the next is the only time you can get a good handle on how your fill related to the decision to make the trade. In all sincerity, being a domschitz, I am all ears if you have a better way to relate reality to backtesting results.
 
...thanks. I forgot it because Harry posted, which sends me screaming the other way. Going back through my signals for the last 60 days looking for cases like this (I found plenty) suggests that the use of supporting but unrelated systems may help to solve the problem. If they agree, no problem. If they don't, abort the trade.
 
Quote from hypostomus:

Going back through my signals for the last 60 days looking for cases like this (I found plenty) suggests that the use of supporting but unrelated systems may help to solve the problem. If they agree, no problem. If they don't, abort the trade.
More...

Glad to learn it seems you've got something you want (although the something you found would be beyond my comprehension!). :p
 
Quote from hypostomus:

...thanks for the explanation. Does that have to do with the famous end-of-bar follies, allegedly designed to fake out EOB traders (perhaps it was you who first noted that here)?
More...

Sorry hypo, can't take credit.

I actually found this out while trying to mechanize and optimize Jack Hershey's ES method. The system performs much better in real time than it does on the backtest because of the nature of the stochastics indicator and the way it behaves in real time vs. delayed time.

It is a safe bet that the CCI oscillator and moving average crossovers have the same phenomenon.

I am leaning towards discarding oscillators as not backtestable altogether, and focus strictly on price, time and volume.

Regards
Oddi
 
You must log in or register to reply here.
Back
Top