Would You Trade This System?
- Thread starter jayjay121
- Start date
The first idea is true, but unfortunately not the second. When the result is sensitive on small parameter changes, the strategy is certainly overfit, but robustness against small changes is no indicator that it's not overfit. Most overfit strategies are not sensitive to small parameter changes - we got plenty examples for that.
Jayjay,
Nice comments overall - but everyone has missed out the most basic test of edge - it seems.
The first question to ask is if the idea makes sense to you conceptually? Does it pass the smell test ? Can you CLEARLY write out the specific market pattern/behavior that underlines your edge ? Don't post it here on the forum if you don't want to, but you must be clear about it in your head.
I will give an example of what I mean above in the context of Trend Following. Rationale behind trend following is that - "once a lot of buyers become convinced about the bull case in a particular market (because of a fundamental shift, news or anything else) and start buying repeatedly and convincingly, it takes a whole lot of sellers to first stop the train and then reverse it. This is the rationale behind trend following and for sure it looks reasonable basis around which to develop the strategy.
So, if you can clearly articulate above, I think you are good to go. Testing on 10 yrs of data, with 6000 data points, every year profitable, system performance similar from year to year, very limited drawdowns, quick recovery period and most importantly the system works on 20 stocks that you have tested - all the above are good validators about the sustainability of the system.
Just go and trade it - and if within first year, you hit a drawdown > than your worst drawdown, STOP. Otherwise, it looks ok.
I applaud your approach to be very sure that you are not curve-fitting. But with all the testing you have done, it seems you are good to go on this one. I would suggest rather than wasting more time on this topic, it will be much more beneficial to focus on developing more systems and then on MUCH HARDER question of blending together multiple systems and trading them in real time.
That will increase sharpe, returns and reduce DD.
GL
Nice comments overall - but everyone has missed out the most basic test of edge - it seems.
The first question to ask is if the idea makes sense to you conceptually? Does it pass the smell test ? Can you CLEARLY write out the specific market pattern/behavior that underlines your edge ? Don't post it here on the forum if you don't want to, but you must be clear about it in your head.
I will give an example of what I mean above in the context of Trend Following. Rationale behind trend following is that - "once a lot of buyers become convinced about the bull case in a particular market (because of a fundamental shift, news or anything else) and start buying repeatedly and convincingly, it takes a whole lot of sellers to first stop the train and then reverse it. This is the rationale behind trend following and for sure it looks reasonable basis around which to develop the strategy.
So, if you can clearly articulate above, I think you are good to go. Testing on 10 yrs of data, with 6000 data points, every year profitable, system performance similar from year to year, very limited drawdowns, quick recovery period and most importantly the system works on 20 stocks that you have tested - all the above are good validators about the sustainability of the system.
Just go and trade it - and if within first year, you hit a drawdown > than your worst drawdown, STOP. Otherwise, it looks ok.
I applaud your approach to be very sure that you are not curve-fitting. But with all the testing you have done, it seems you are good to go on this one. I would suggest rather than wasting more time on this topic, it will be much more beneficial to focus on developing more systems and then on MUCH HARDER question of blending together multiple systems and trading them in real time.
That will increase sharpe, returns and reduce DD.
GL
PF for 2nd system is higher, but I am more sceptical as most of the performance comes from 2008. Just remove it and then rerun the numbers. that will give a better indicator of what to expect in future.
Also, it seems like skewed performance in 2008 means your results are directly proportional to increased volatility as your target profit might be a function of ATR. If that is the case, do remember that in actual trading, you will be trading a much smaller size in times of huge ATR, to achieve a similar risk level per trade.
Thus if you weigh your share size based on volatility, your 'much better' 2008 results might just become another average year results. Something to think about.
Also, it seems like skewed performance in 2008 means your results are directly proportional to increased volatility as your target profit might be a function of ATR. If that is the case, do remember that in actual trading, you will be trading a much smaller size in times of huge ATR, to achieve a similar risk level per trade.
Thus if you weigh your share size based on volatility, your 'much better' 2008 results might just become another average year results. Something to think about.
OP is not doing hft, as he has said that multiple times, not even micro scalping. He is taking 2-4 trades per day. Doesn't matter if he models on 1 min data or 10 min data. What he is doing is that he is trying to capture major intra day turns in the market that happen 2-4 times a day, no reason for 20,000-50,000 trades.
Curious, if you actually trade, because from your other posts, I have a feeling that you are PhD student at an ivy league university. Not attacking you, lest you misunderstand me, just trying to see if my guess from reading your other posts is correct or not. I am just trying to see how good I am at guessing.
No no no, your deduction assumes that OP is entering using 1 min data and is getting out very quickly like within 1 min or so and is doing this many many times in a day. He is not doing HFT, so small timeframe regular assumptions wont apply here.
OP is just using 1 min data to enter 2-3 times a day. So, your deduction is not correct.
gmst, thanks for taking the time to read all my stats.
You seem to have expressed exactly what im doing.
Some good ideas in there too like 'carry on trading until you get a worst drawdown than in backtest'. This alone is a rule to which i could use, thanks. Funny sometimes you can miss the obvious things.
Also i like the 2008 idea to leave out of all testing as that was a once in a life time period, (hope im wrong of course).
I have 2 options in postion sizing. I can think volatility is attractive and use fixed shares and not adapt to volatility (basically trading volatility), or i can volatility adjust my size using ATR so i get the same amount of profits, losses per trade. It works both ways but if Net profit is my optimisation then using volatilty comes on top and is the system you see.
You seem to have expressed exactly what im doing.
Some good ideas in there too like 'carry on trading until you get a worst drawdown than in backtest'. This alone is a rule to which i could use, thanks. Funny sometimes you can miss the obvious things.
Also i like the 2008 idea to leave out of all testing as that was a once in a life time period, (hope im wrong of course).
I have 2 options in postion sizing. I can think volatility is attractive and use fixed shares and not adapt to volatility (basically trading volatility), or i can volatility adjust my size using ATR so i get the same amount of profits, losses per trade. It works both ways but if Net profit is my optimisation then using volatilty comes on top and is the system you see.