Frequency of Re-optimization
- Thread starter sidm
- Start date
I don't really see how the above points are connected to the question I asked, but I will bite anyway. So here are my thoughts.
Data used for analysis/optimization should obviously be limited what is observable in the PAST, since that is all we can see really. That is not the issue being discussed here though. Let us assume for the sake of discussion that none of us make the "duh" mistake of using future data points to judge the present trend while optimizing.
As far as independent/dependent variables go, presence of a trend implies that there is some sort of auto-correlation in the prices. Hence tomorrow's price is the dependent variable and prices in the past become the independent variable.
The question of "appropriate math" is interesting. Here is my take.
(1) The standard mathematical models based on Normal or similar distributions don't work.
(2) Since there is so much randomness and noise in the markets, any strategy must inherently be simple. Complexity makes systems fragile and susceptible to small movements. Hence my aversion to complicated indicators and attraction to simple heuristics.
(3) While probability distributions don't work, there is one concept that has been shown to hold much promise in empirical research: Fractals. Due to fractal nature of the prices, no matter what time scale you zoom into, the basic pattern doesn't change. This tells me that money can be made at all time scales (seconds, minutes, hours, days, weeks, months).
Quite a digression from the original question of re-optimization, but interesting nevertheless.
Quote from kut2k2:
Would this be you showing childish pique over not finding a <s>sucker</s>buyer for your $8.5M <s>boondoggle</s>trading system?
I could only make $500 a month on 2 NQ $12 k accounts forever and unlimited aum fees on Covestor as a top ten outperformer of the s&p in the last 365 days by 2,840 basis points.
Right now it would be third in the World Cup Championship of futures trading and pretty close to making my family worth another comma.
I like this question, I've been running my optimization constantly (even now), but I've been seeing a lot of over-specialization. But at the same time, I feel that optimizing less often wouldn't eliminate the problem, it would just over-specialize for a different time range. kut2k2's suggestion seems reasonable, but when using large data sets (I'm using three years), it would be hard to justify optimizing every 1.5 years.
I've found the initial conditions of the scheme can change the behavior dramatically, so perhaps optimizing for a variety of start dates, initial investments, global stock market environments... might remove the tendency of your optimization to over fit the data.
I've decided that instead of optimizing for a 3 year data set, I split the data up into 6 intervals that are run with the same initial investment. That way if any over-fitting does occur it'll hopefully be averaged out by the other intervals. You could also potentially ignore the best and worst performing individuals.
Another potential workaround is to ignore the tweaking done as the optimizer over fits data. When my optimizer over fits, it tends to simply tweak an algorithm's parameters to buy a stock before an increase, as opposed to creating an entirely different set of parameters. If you run the optimizer constantly, the algorithm parameters you utilize in practice could be the average parameters of every day's optimum parameters for the previous month, for example.
Anyway those are a few ideas I'm working on. Not sure I'm explaining this well though.
I've found the initial conditions of the scheme can change the behavior dramatically, so perhaps optimizing for a variety of start dates, initial investments, global stock market environments... might remove the tendency of your optimization to over fit the data.
I've decided that instead of optimizing for a 3 year data set, I split the data up into 6 intervals that are run with the same initial investment. That way if any over-fitting does occur it'll hopefully be averaged out by the other intervals. You could also potentially ignore the best and worst performing individuals.
Another potential workaround is to ignore the tweaking done as the optimizer over fits data. When my optimizer over fits, it tends to simply tweak an algorithm's parameters to buy a stock before an increase, as opposed to creating an entirely different set of parameters. If you run the optimizer constantly, the algorithm parameters you utilize in practice could be the average parameters of every day's optimum parameters for the previous month, for example.
Anyway those are a few ideas I'm working on. Not sure I'm explaining this well though.