Data point vs. parameter

logic_man · Jun 13, 2012

In the course of documenting my ES trades, I collect a number of data points that don't seem to have any relationship to the outcome of a given trade, but I collect them because I initially thought they might and now I want to stay consistent.

My issue is that as I am bringing that same strategy to other futures contracts, I'm finding that a data point which didn't relate to ES outcomes appears to relate to Euro outcomes. Also, in Crude, something which is a significant parameter in ES, to the point where I filter a trade if the value is at a certain level, doesn't seem to matter at all to Crude trade outcomes.

While I'm hesitant to change my model or ignore some of the things I've learned from the ES, I'm also cognizant of the fact that different assets will behave differently and I shouldn't assume that what matters or doesn't matter for the ES will matter or not matter for other contracts.

For anyone using the same basic strategy across assets, especially futures, have you run into this and do you just operate under the assumption that each asset/contract will have its idiosyncrasies and just go with it?

abattia · Jun 14, 2012

Quote from logic_man:
...For anyone using the same basic strategy across assets, especially futures, have you run into this and do you just operate under the assumption that each asset/contract will have its idiosyncrasies and just go with it?

Yes, different instruments behave differently in my experience. Makes sense if you remember that each instrument has a slightly different crowd of traders/bots trading it.

I asked a similar question here a while back ...
http://www.elitetrader.com/vb/showthread.php?s=&threadid=215986

logic_man · Jun 14, 2012

Quote from abattia:

Yes, different instruments behave differently in my experience. Makes sense if you remember that each instrument has a slightly different crowd of traders/bots trading it.

I asked a similar question here a while back ...
http://www.elitetrader.com/vb/showthread.php?s=&threadid=215986

Thanks for the pointer to your earlier thread. I agree with your premise, as shown in your example of SPY vs. F, that a chart of any of these assets would be difficult to identify by look alone and that a deeper analysis would be required to differentiate a chart of CL vs. ES vs. E6, which is pretty much what I've been undertaking these past couple of months.

In one of your responses, you said that you had enough data to know that these weren't just short-term anomalies. You have an advantage on me there because I have no way to automate a long-term backtest. I may just be in a "state of the world" in which the current trades where the parameters meet certain thresholds appear to behave differently from the ES trades meeting those thresholds, but if I had started to bring the strategy over to Crude and the Euro, say, a year ago or a year from now, these particular parameters would not have raised any potential issues. But, from my own thinking, confirmed by reading your thread, I am pretty well convinced now that I should have come into the adaptation process thinking that there would, of necessity, be some differences in what variables impact outcomes.

dom993 · Jun 14, 2012

Quote from logic_man:

You have an advantage on me there because I have no way to automate a long-term backtest.

It is probably quite detrimental for you not to have the proper tools ... How can you be entertaining a system approach if you are left to manually test & execute them ?

I just converted my two live systems from Ensign to NinjaTrader ... one is very simple, and only took a few days to implement (less than a week, from the time I downloaded NinjaTrader) - and it boils down to about 300 lines of code.

The other one is quite complex, it has taken me about 6 weeks and nearly 6,000 lines - but it now backtests to parity with the original system in Ensign, in a small fraction of the time.

I found a number of Ninja bugs along the way, but fortunately all have more-or-less workable workarounds.

logic_man · Jun 14, 2012

Quote from dom993:

It is probably quite detrimental for you not to have the proper tools ... How can you be entertaining a system approach if you are left to manually test & execute them ?

I just converted my two live systems from Ensign to NinjaTrader ... one is very simple, and only took a few days to implement (less than a week, from the time I downloaded NinjaTrader) - and it boils down to about 300 lines of code.

The other one is quite complex, it has taken me about 6 weeks and nearly 6,000 lines - but it now backtests to parity with the original system in Ensign, in a small fraction of the time.

I found a number of Ninja bugs along the way, but fortunately all have more-or-less workable workarounds.

Well, I actually don't have multiple systems, I have one. The closest analogy to it would be Elliott Wave, where every bit of market action is accounted for in some way within the system, only I have fixed what I consider Elliott's flaws, e.g. I removed the entire portion of the Elliott apparatus related to Fibonacci numbers and substituted my own quantitative framework, which is much more flexible and reflective of actual market dynamics as opposed to attempting to subordinate those market dynamics to an ideal framework. So, the one system is all-encompassing (and completely objective, in the sense that I could teach it to someone and they and I would reach the same conclusions about market action that I would, rather than the situation with Elliott where every Tom, Dick and Harry has his own wave count) and there's no need for any other, nor does any market action sit outside the system.

The issue that I raised to start this thread is more about the subtleties of taking the system to multiple assets, especially as it would relate to filtering trades with negative expectancy.

dom993 · Jun 15, 2012

And my point was that in order to get non-anecdotal results, you need to backtest for long enough periods of time & number of trades that the task of evaluating different options is out of reach in a manual approach.

(I know you already know that, sometimes we simply need 3rd parties to state the obvious for it to register)

logic_man · Jun 16, 2012

Quote from dom993:

And my point was that in order to get non-anecdotal results, you need to backtest for long enough periods of time & number of trades that the task of evaluating different options is out of reach in a manual approach.

(I know you already know that, sometimes we simply need 3rd parties to state the obvious for it to register)

Yes, I agree that would be ideal. To some extent, since my algorithm is based off of an insight derived from another trader with whom I have corresponded for about 6 years now, I look at the positive results he has had using that insight as additional validation of the method and he has been trading for nearly 3 decades.

Part of me also deduces from the fact that it is so unique that it can't be straightforwardly coded in any extant trading platform as validation as well, since it is my firm belief that the more unique an approach is (e.g. do you have to make up your own jargon to describe it, which is true in my case), the more likely that real-time positive results can serve in lieu of long-term backtests. Perhaps that is naive of me, but it seems to make sense.

But, yes, optimizing in real-time can be a bit hairy and I have lost money on trades that, in retrospect, were not optimal, although these losses have not taken me out of the game and, either by luck or by actual identification of non-optimal trades, I have never had to revisit the decision to not take future trades meeting those criteria. In fact, this past week, I avoided 69 pips worth of losses on the Euro by not taking trades and over the past year, I have avoided 112.25 points of losses on the ES. 90% of the time, the decision of to take a trade or not is straightforward and for the remaining 10% I just have to be satisfied with letting them pass by until I collect sufficient data. Typically, I will flag a specific "type" of trade if the first few trades don't work out on paper. Since my model only has a few parameters as it is, this is manageable.