Backtesting
- Thread starter shMark
- Start date
I used to backtest a lot.
My personal experience is that there are too many factors rendering backtesting invalid (the OP mentions some).
Now I just focus on trying to 1) determine if we're trending or moving sideways; and 2) finding the sweet spot of prudent risk management where risk of blowing up is nearly non-existent, but you're still making out like a bandit when you're right.
My personal experience is that there are too many factors rendering backtesting invalid (the OP mentions some).
Now I just focus on trying to 1) determine if we're trending or moving sideways; and 2) finding the sweet spot of prudent risk management where risk of blowing up is nearly non-existent, but you're still making out like a bandit when you're right.
how do you determine #1 if you don’t look at history?
how do you determine #2 without looking at how things have worked out historically?
at least informally.
Both good points.
Well, I do look at history, I'm not trading blind.
My meaning, which I should have probably articulated better, is that after going from very simple strategies to more complicated (and possibly overoptimized) ones I backtested to death, trying to take into account a myriad factors, I've gone back to the basics.
We've all seen traders that were very successful under some market conditions, only to go bust when the market changes but they stick to their "edge" - that is not there any more.
That is not to say, of course, that backtesting has no value, and I probably still benefit subconsciously or not from my previous experience.
And I'm sure that (way) smarter people that me can find and renew one edge after the other.
[
1) For trending, EMAs and higher highs and higher lows, this is not normally hard to see - once it gets going. Lateral moves are a bit more problematic (unless they're extended over time) and inflection/exhaustion points are a b*** (but I look for extreme volatility as a sign that the trend is exhausted).
2) Nothing fancy, if my 3 timeframes are aligned, I add twice to winners in (what I think will be) the first 20% of a move; I trade half size if it's going sideways or I don't see a clear trend and I stay out when I'm confused (which is about half of the time
). The normal position size is a % of my trading account, which is a percentage of my liquid assets (this last percentage fluctuates, depending on the overall economic situation).]
That's pretty much it. Remember that brokers employ people and you can bet the employees look at your trade data and try to reverse engineer it. Brokers have nothing to lose. You think IB cares if some employee steals your ideas and gets rich with it? Not one bit. Their whole business is about churning the traders because there's another naive hopeful at the door waiting to lose their money. Overall trading only works for short term and you need to consistently change and change your methods. Over time this is extremely tiring and will wear anything out.
yeah, IB does care. It is a breach of their fiduciary obligations to their customers. They could be sued or lose their licenses as a broker dealer.
further it’s a breach of trust and confidence which can hurt their reputation. In the end IB is some network connections and their reputation.
This thread contains a bunch of misconceptions about backtesting and it's role in systematic trading. But I still think it does not belong in the options section, so I refuse to contribute.
If you expect alpha to exist for a period of time in the present and future, how can you think it hasn’t existed in the past? I mean, how can one exist without the other?
Time is a sliding variable in the lifetime of existence of the alpha. Your discovery is somewhere on this timeline, you just don't know where.
Time is a sliding variable in the lifetime of existence of the alpha. Your discovery is somewhere on this timeline, you just don't know where.
Last edited:
All of that sounds good but it's nearly impossible to catch anyone doing it. Even if they did, it would never come out and the employee might be terminated quietly, at most.