Seeking feedback on my rules-based system

HobbyTrading said:
Be careful in only backtesting a nine year history as that has been a bull market in the USA. What works in a bull market may not work when the market moves sideways, or decides to go down.
More...

Thanks, and a good point. Most of the ETFs that I actually trade have only been around for 9-10 years, as you noted. However, during the model training phase, I will actually analyze securities that have been around longer. For instance, I train the model using ES futures (which go back to 1997, I believe), but actually trade UPRO (around since 2010?).

I'd certainly prefer to be backtesting the actual security that I'm trading, but I would rather trade a slightly flawed model with a richer set of statistics than vice versa.
 
Control of past data is not an indicator of future results. If you get results with this, it is not because of what you think you are doing. It will be because of your money handeling strategies with winner and losers.
 
cruisecontrol said:
Was going to post something similar but ph1l beat me to it: Since you are walkforward, the next likely source of overfitting would be in the choice of indicators.

Did you think of a large number of them and then throw out ones that didn't work? Or did you leave them all in and have the system choose? If the choice itself is done in a walkforward way out of a large set, then concern about data snooping would be alleviated.

The profit graphs you posted look quite significant to me, assuming there is no data snooping. Also, the concept of the system in general seems sound.
More...

For the S&P, I analyzed 103 indicators, testing 2615 intervals. From this, I extracted 150 testable strategies and chose the "best" 6 that I'm currently trading. So I'm definitely spanning a large model space, but I'm quite sure that the correlation between many of these strategies is pretty strong. Ideally, I'd like to only trade strategies that are sufficiently "different". Other than looking at the backtest graph for each one, I haven't put any meat on how to actually measure "different".
 
I have to say that I'm humbled that you guys would take the time to read what I wrote and provide this thoughtful feedback. I'm just a middle-aged guy fumbling in the dark through a sea of bad/disinformation. It's good to see that I'm not totally lost, but I'm still waiting for the other shoe to drop. As my wife said, "a lot of smart people are trying to solve this problem, why are you better?"
 
morganpbrown said:
People may laugh at the fortran, but when it comes to this kind of brute force, having fast code and multiple processors on the machine really comes in handy. I can usually run this analysis for one target security in ~30 minutes.
More...
The software language is just one of the many tools you use to get to the ultimate goal (i.e. make some money). It doesn't matter which tool you choose, as long as it helps you to achieve the goal.
 
danielc1 said:
Control of past data is not an indicator of future results. If you get results with this, it is not because of what you think you are doing. It will be because of your money handeling strategies with winner and losers.
More...
Sounds like a variation of that old canard, "Risk management is the only real edge in trading". I'm sorry but but this is just untrue and shows a fundamental misunderstanding of trading. There are only 2 ways to make money in trading: 1) Luck, 2) Having an edge. It is a mathematical fact that no "money handling strategy with winners and losers" in the world can expect to make money trading a zero mean random walk.
 
morganpbrown said:
First post to this board. And probably the last! I have done some data mining and analysis to develop rules-based systems for several "uncorrelated" ETFs. I'm going to describe my methods and seek critical feedback from other posters. (Famous words from a one-time poster?)

I code in Fortran 90 (the guts) and perl (high-level processing), and also use open-source signal processing software.

I'm only trading once per day, and I execute the trades manually (in Robinhood). I'm trading ETFs, not options or futures.

I'm using a past "states" to predict the future behavior of the target security. The past states could be 30-day past S&P return, VIX over or under 200 d MA, etc.

For each predictor, say 30-day past S&P return, I loop over all possible states (e.g., 0-3%, 3-6%, etc.) and backtest the target security. This generates a report:

avg return
security lwr upr #trade profit cash ratio power PPT
XXXXXXXXXX -0.370 -0.340 3 313.00 1.22 255.97 80119.24 104.33
XXXXXXXXXX -0.340 -0.310 8 -144.75 3.13 -46.24 6693.51 -18.09
XXXXXXXXXX -0.310 -0.280 4 -192.25 1.49 -129.02 24803.41 -48.06
XXXXXXXXXX -0.280 -0.250 4 -197.50 1.68 -117.63 23232.66 -49.38
XXXXXXXXXX -0.250 -0.220 9 -159.25 4.48 -35.51 5655.05 -17.69
rm30_ES.F -0.220 -0.190 14 356.50 17.68 20.16 7188.03 25.46
rm30_ES.F -0.190 -0.160 37 962.45 72.32 13.31 12809.16 26.01
XXXXXXXXXX -0.160 -0.130 46 211.00 89.96 2.35 494.92 4.59
XXXXXXXXXX -0.130 -0.100 79 -67.75 215.26 -0.31 21.32 -0.86
XXXXXXXXXX -0.100 -0.070 136 -226.67 364.57 -0.62 140.93 -1.67​

I compute a variety of metrics for each backtested strategy. Most (the ones with the XXXX's) can be discarded immediately, if they lose money, have insufficient trades, insufficient profit/trade, etc.

After aggregating "adjacent" strategies from the report, I produce a master report of all the accepted strategies (usually about 200), which I insert into a spreadsheet and rank using cutoffs. In this case, I settled on the 6 best strategies, which I test daily, and buy if the "signal" hits.

View attachment 200666

Here's what the daily analysis looks like for one strategy. I'm trading UPRO and the signal is when a consumer confidence has had a crossover of its 200-day MA in the past 30 days. The backtest shows me when signals were received. If I have a signal since yesterday, I trade. The green dots show cash in the market (often, in the past, this strategy had NO cash in the market)

View attachment 200667

I've only been trading for a month, far too soon to know if I've got something. The combined strategies seem to perform well in backtesting. Here's the combination of 5 trading strategies for XOP, backtested for ~9 years. I'm able to follow the major upswings. I'm not heavily invested during the big downturns. In sideways markets, I'm not heavily invested, but seem to cherrypick the secular increases.

View attachment 200669

Thus far, I don't optimize position sizing, nor the exit timing. I simply hold the target security for 30 days, unless it stops out.
More...


Consider transitions between ‘states’ as events. The market functions as a sequence of events. Use this sequence of events as the independent variable and time as the dependent variable.
The sequence of events signal both entries, exits, early entries, early exits, waits and reversals.
Every event has what came before, the event, what must come after.

‘Must’ = understanding the price volume relationship. In a nutshell, Volume leads price.

How one goes about defining events makes all the difference.

The challenge is to turn noise into signal, and what is/are the prerequisite/s necessary to accomplish this.
 
You must log in or register to reply here.
Back
Top