Quote from dom993:
I bought a few weeks ago a "30 year-symbol" pack from TickData, which for a reasonable price gave me 9 years of the best historical data for CL / ES / 6E + 3 years for GC ... I would suggest you get down this path, and backtest your systems on much longer periods than 6 months to 1 year.
The problem with avg-risk >> avg-reward is exactly what you are experiencing ... at some point you hit a string a losers which sets you far back.
I have a very simple test that I use to assess if something has a directional edge ... I use target = stop minus 1-tick (so that a move of exact same size will trigger either), no trade management at all, and simply look at the win% ... anything above 60% for a sample set large enough (500+ trades) shows a viable directional edge (the stop/target combo can vary from trade to trade based on market conditions, it just need to be target = stop minus 1-tick for each & every trade).
Yes, I was aware that I probably would see degradation from the test and that there are always deviations from the backtested samples and that those deviations can be for the worse and that the skewed risk to reward ratio made the accuracy of the win percentage more important than usual.
The problem with backtesting further back is that I'd have to do it manually. The algorithm logic is so complicated that I haven't been able to automate it, so unless I am following a market in real-time or near real-time, it's practically impossible to get trade data.
Regarding your test, I don't think that would work for me because my trades can have enormous theoretical risk, based on volatility. I say theoretical because in actuality, the average loss is 1/3 the size of the average amount at-risk at the start of the trade, due to the way I move stops. For this reason exactly, I am extremely concerned with understanding why a trade might results in me not being able to move my initial stop. If my actual risk ever began to equal my theoretical risk, there would be no way the system could survive. It is the confluence of events that trigger entry which give me some confidence that this will never be the case. At the base level, about 14% of all trades end by being stopped out for the maximum loss. But, by segmenting trades, I have been able to find trade criteria where, at low end, only 2-3% of trades take the maximum loss and, at the high end, 20-25% take the maximum loss. Obviously, I don't take the trades which fall into that latter bucket.
I gave the example of "signals within signals" because those are the bulk of the trades I filter. 20-25% of those signals end at the maximum loss. My hypothesis is that due to the increase in volatility necessary to trigger a signal to begin with, that volatility has the "spill over" effect of triggering a signal in the opposite direction, since the market is just moving that much more. We've all seen markets that, once they start moving big-time in one direction, move wildly in the other as well, just as a compensating action to the overall increase in movement. So, I don't think I'm wildly off-base with this hypothesis and why it creates "false positives" for my system.
Setting those trades aside, this "risk greater than reward" system has a profit factor of about 1.7 on the Euro with a 75% win rate and a PF of 2.3 on the ES with an 80% win rate. That I consider the "base" system because I feel absolutely no need to consider not taking these "signals within signals" as curve-fitting. If the best I can expect over the long haul is a PF of 1.7 to 2.3, so be it. I will lever that up to as much as those markets will bear.
And, finally, maybe Crude is just a different animal and isn't amenable to this particular type of trade. If it weren't for Crude, I'd be up close to $400/contract over the past week, so it's really the crux of the issue here.
- will score very high win rates with randomized entry points. This is true for both long and short positions. The entries can be randomized in pretty much any conceivable way - random price offsets from baseline, random time offset from open, combinations thereof, etc etc etc.
