backtesting is stupid

[Trader666]

This is even better... here's Jack on William Eckhardt (who is still running several funds since the 1980s and when the S&P is shown to scale with his equity curve the S&P is so dwarfed it looks more like part of the x-axis):


Did you think Eckhardt was making any money those 18 years??? Some of your posts read like you think he is making a lot of money or that he is a person to use as a standard.

Making a point or two a day and not compounding it beats Eckhardt and his turtles.

Here in ET, people are doing quite well unknown to you apparently.
Comparing the pool extraction and its applications to Eckhardt is really humorous.

http://www.elitetrader.com/vb/showthread.php?s=&postid=2445881&highlight=eckhardt#post2445881

 

[Now is Now]

Quote from jonp:

no i am not, and that is why i don't backtest, i don't know programming, I never took statistics...

i just trade, maybe beginners luck, maybe not.

buy low sell high, sell high cover low. a strategy this simple doesn't need to be backtested it's an axiom of the market. there is no telling from previous situations how the market will react right now in this same situation.

Whether you like it or not....everything you do in assessing your life is a product of "backtesting" (experiences)..


Simple example if you are that way inclined at 19, you should be , ...girls...

Just like parameters and indicators...they are all good until they go wrong...the trick is finding the right one...can only be done after assessing previous experiences and making the appropriate adjustments to succeed...ala "backtesting"

The hard part...ensuring that they (girl, system) stays the right one...it requires work...




NiN

 

[Arthur Deco]

Quote from Trader666:

Yes, Richard Dennis' former partner.

Here's another great quote on backtesting:

"Backtesting is not a viable means for finding anything out."

-- Jack Hershey

Golly! Guess I'll quit doing it, then!

 

[alfobs]

Arthur,
I hope to keep same frequency of posts in the future. I really do not see a reason to join a thread unless I see some absurd or unproven statements based on emotions. Anyway I'd be glad to dialogue but not sure about a subject.

Quote from Arthur Deco:

Alfobs, you are indeed a man of few words, a rarity here, with only sixteen posts in two years, and this only your third post this year. You packed a lot into it. Care to dialogue a bit? This thread is dead anyway.

 

[Dacamic]

Quote from alfobs:

Also in my opinion forward testing does not have any advantage over properly organized backtesting process.

True that.

 

[Arthur Deco]

Quote from alfobs:

Arthur,
I hope to keep same frequency of posts in the future. I really do not see a reason to join a thread unless I see some absurd or unproven statements based on emotions. Anyway I'd be glad to dialogue but not sure about a subject.

Thanks. I am sure ET will provide you with a steady stream of provocations to keep you posting.

Let me quote your pithy post:

"IMHO, backtesting is the only way to get at least some confidence in a system before you start using it. So backtesting cannot be stupid for this simple reason. In fact a deep backtesting (with sufficient amount of data) can cover most of the market moves to prove a system on robustness, and therefore to take care of those cases in future. If somebody failed with trading after backtesting then it happened because of over fitting and lack of variety of market conditions in the historical data. Less indicators, some sound logic behind your algorithm (which must be no more complicated than should), implemented money and risk management will help to make backtesting valid. For instance my one year backtesting of 15 min bars have supported me for last 6 month without any significant deviation from backtest performance results. Also in my opinion forward testing does not have any advantage over properly organized backtesting process."

What I would like to discuss is: "a deep backtesting (with sufficient amount of data) can cover most of the market moves". Let me give you an example from my own practice. I trade NQ intraday from a one-minute chart. I used to backtest on only the last three months of the most recently expired contract. And apply the results to the current contract. As the current contract got about half way through, I might start testing it. But in both cases I became a bit uncomfortable about the small number of sample days, and switched to testing the continuous contract, which EasySignal provides six months of. My weekly reoptimizations are yielding parameter sets which change sufficiently slowly that I am comfortable with the results. But when I read your post I imagined that perhaps you are going for longer history streams. Can you comment? Thanks.

 

[brownegg]

Quote from Arthur Deco:

[via alfobs]Also in my opinion forward testing does not have any advantage over properly organized backtesting process."

I don't think I can agree with this in practice. I understand what you're saying, but I consistently find a specific type of knowledge I can only get from watching a system calculate and operate in real-time; there are things I didn't notice, forgot to add, etc. It's the only way I know of to apply my own overall knowledge above and beyond the specific things I coded.

I suppose someone could be good enough at this they could eventually not need to do it, but if so I don't think I'll get there.

 

[Arthur Deco]

Quote from brownegg:

I don't think I can agree with this in practice. I understand what you're saying, but I consistently find a specific type of knowledge I can only get from watching a system calculate and operate in real-time; there are things I didn't notice, forgot to add, etc. It's the only way I know of to apply my own overall knowledge above and beyond the specific things I coded.

I suppose someone could be good enough at this they could eventually not need to do it, but if so I don't think I'll get there.

Alfobs may or may not speak up, as he says he only responds to abject stupidity, which your post isn't. My guess is that he didn't mean to exclude observation of how the system works in real time. I think what he meant is that so-called forward testing merely leads to more delusion on the order of selective hindsight: "I don't know why my account is down because it looks to me like it works." I had just such an experience that you speak of yesterday. My system said to take a profit, and being a good little robot I did. The trade went on to make twice as much. And of course I knew that when I reoptimized the system it would take some of that unexpected largesse. It did. Watching your system at work is a bit like arguing with yourself: "Who the fuck coded this abortion up?"

 

[brownegg]

Taped across the top of my main monitor on the desk: "Your code sucks."

 

[Arthur Deco]

Quote from brownegg:

Taped across the top of my main monitor on the desk: "Your code sucks."

Yours, too? I thought it was just me. I have had essentially the same codes for years, and yet I still tweak them. Of the twenty-eight running on my screen, only one hasn't been touched in the last six months.