My struggle - is it Data Quality or something bigger?

[sysdevel99]

I searched for data quality, didn't get any hits, so my apologies if this is already covered -> in that case please point me to the links discussing it (that's my guess on the poll for this thread anyway).

Disclaimer:This isn't my first rodeo, although duration doesn't mean competence, just because I've been trading for a while doesn't mean I know what I'm doing.

I like to get ideas from various sources, from blogs to reverse engineering systems from collective2 to books to interviews ...
Then I try to code those systems to see if there's anything there (I do like to write trading system backtests with the hope to find gold). Now here's the challenge, more often than not (19 our of 20) the systems described in these data sources don't work. Not only do they not work they are actually losing systems but not losing enough that you can just reverse them either. Now granted, if I've a winning system I basically don't share it either, but at least I don't "claim" that this is a winning system ...

So what is the reason that these systems don't work (cause many go through a significant effort coding them, showing equity curves, nbr of trades, ....)
1. Are the guys just full of it and they don't know what they're doing
2. Are they purposely misleading to sell more books or ads?
3. am I too stupid to replicate basic rules and mimic similar results (may be the most realistic option although I don't know how anyone could answer that)
4. Are they working based on bad data (that's what kills me with TA all around, if I use thinkorswim for charting and use prophet or native chart for dividend adjusted stocks I'll get very different results in trendlines, retracements ... -> in system trading if they don't use dividend adjusted data maybe that's why their system results look so different)?
5. Is there something else?

 

[Xela]

more often than not (19 our of 20) the systems described in these data sources don't work.

That was my initial experience, too.

The figure did reduce to some extent, as I gradually acquired more experience and judgment at assessing - according to their sources - the probability of "there being anything in them", which is commonly very low.

if I've a winning system I basically don't share it either

I basically don't, either. Partly because I want to be secretive about it, partly because I never have anything to sell/promote/self-promote at all and wouldn't want to, and partly because I have no interest in influencing how others trade. I'll offer more general observations from my more general trading experience though, especially if I think they can perhaps quickly/easily avoid someone being/becoming misguided.

I like to get ideas from various sources, from blogs to reverse engineering systems from collective2 to books to interviews ...

My guess (from what you've said there, though I may be mistaken and/or even maligning you, here!) is that you're being considerably less selective than I am, with what you look at. (I wouldn't dream of looking at most blogs or at Collective2, and I examine ideas only from sources I consider very well-qualified to be sources ... which, admittedly, is perhaps such a subjective and vague thing to say that it contributes almost nothing to the conversation at all ... ops: ).

 

[stepandfetchit]

sysdevel99
I have had similar thoughts and experiences.

 

[NoVoodooHere]

Snake oil malfunction?

 

[sysdevel99]

Snake oil malfunction?

Some of them for sure.
Others however seem to want to do the right research but come up with completely different results. I wonder how much of that is based on the backtesting environment versus the data versus the actual logic [the logic seems pretty basic in many cases].
e.g. ninjatrader versus custom app versus multicharts or tradestation ...

 

[NoVoodooHere]

Some of them for sure.
Others however seem to want to do the right research but come up with completely different results. I wonder how much of that is based on the backtesting environment versus the data versus the actual logic [the logic seems pretty basic in many cases].
e.g. ninjatrader versus custom app versus multicharts or tradestation ...

Geez adunno if I had to consider all that... Makes me sick just thinking about it- - That's why I quit my career as an engineer. The best thing about discretionary trading, is, the simpler, the better. But that's just me.

...Good luck w/ that stuff

 

[dom993]

Reason #1 : a lot of published backtest results come from unreliable backtest engines (Tradestation to be specific) ... when a LMT order is deemed filled on a price touch, the backtest results are simply garbage.

Reason #2 : commissions and slippage are not accounted for.

 

[sysdevel99]

Reason #1 : a lot of published backtest results come from unreliable backtest engines (Tradestation to be specific) ... when a LMT order is deemed filled on a price touch, the backtest results are simply garbage.

Reason #2 : commissions and slippage are not accounted for.

Do you think they eventually figure it out or they've no intention of trading their system anyway?

 

[schizo]

Now here's the challenge, more often than not (19 our of 20) the systems described in these data sources don't work. Not only do they not work they are actually losing systems but not losing enough that you can just reverse them either.

LOL. When I set out to find the holy grail, this was exactly what I had in mind. I wasn't looking for gold. Since there were so many bad ways to trade, I figured it would be better to simply write the most hideous system with the largest drawdown and merely reverse the code. Unfortunately, it didn't work.

 

[tommyho]

Geez adunno if I had to consider all that... Makes me sick just thinking about it- - That's why I quit my career as an engineer. The best thing about discretionary trading, is, the simpler, the better. But that's just me.

...Good luck w/ that stuff

I like your point Voodoo. I'm a noob, trading only 2 yrs BUT what has worked for me is finding something simple, developing a plan that works for me and sticking to it. Too much info gets me way too confused. The less anxiety and questioning the better for me. Thanks Tom