Manual Backtesting

DblArrow:
>The second in not a conclusion of the first.

I didn't imply that the second was a conclusion of the first. I said that the considering the content of the first (only manual testing), the conclusion of the second (things don't change) is niave.

>The second is simply a statement that I have
>come to believe.

And it is that belief that I maintain is niave. You make a rather generic statement...

>But I also believe that if it worked yesterday and the day
>before it should have worked 23 yrs ago, and should also
>work into the future.

But yet without automated testing you can't possible have crunched the millions upon millions of candles, setups and trades to be able to support the belief generically.

>a 20 ema will always be a 20 ema and the
>market will either trade above, below or at.

>Now take a chart 23 yrs back and you will see
>that the market was either above, below or
>at the 20.

Yes, you can draw a straight line at a random number and the market will always be below or above it as well. Not much meat there to sink our teeths into.

>I have taken a chart from many years in the past,
>many various points, and manually checked my system.
>It has given me the confidence that I require in my trading.

That is cool. There is nothing in my statements to imply that you can't trade successfully without automated backtesting -- only that your (things don't change) conclusion becomes toilet paper on the floor after a few hours of computer time.

JB
 
Though not exactly on the topic of manual testing, the discussion of the relevance of old data to current situations brought to mind the issue of volatility. It seems to me that one of the biggest reasons why systems do better at some times than at other times has to do with volatility. This is often taken into account in many trading systems, but it doesn't always smooth out those bumpy patches of system performance. I think one of the reasons for this has to do with how volatility is defined for the given system.

I guess the normal or usual definition has to do with the concept of daily ranges or true ranges if the gap is accounted for. However, systems that use intraday bars rather than daily bars have another type of volatility to take into account. The intraday volatility trend may be significantly different than the daily volume trend, though the two usually are closely correlated. For instance, the intraday volatility of a 15 minute system might need to take into account the average bar height of the intraday bars, as well as the daily volatility in order to come up with a better picture of what the true volatility is for that system at any given time. Other volatility measures may be useful as well.

Why is intraday volatility important for intraday systems? If stops are calculated from bar highs and lows, and the average range of a given timeframe's bars have either significantly increased or decreased, this will have a large impact on the risk / reward profile of the system. Instad of 3 point stops, the stops might need to be altered to 2 points or 5 points, for instance, yet if the reward part of the picture is static then it is easy to see how the ratio gets thrown off. Anyway, food for thought...
 
Quote from MrJohnGalt:

Just another bunch of guys who can't understand and utilize statistics. Period.
More...

I continue to be amazed at the quality and content of your contributions.

Make 'em pretty, Chris
 
Quote from MrJohnGalt:

Just another bunch of guys who can't understand and utilize statistics. Period.
More...

i second that.... but hey, each to his own! :D

there is no right way for everyone!
 
...FWIW (and it ain't much), I have been testing intraday systems on NQ (mostly AM) for nearly two years. I have found that while the entry parameters sometimes are sensitive to intraday volatility, and the expectation definitely is, the stops and takes aren't, and their ratio remains steady around 3.3:1.
 
Quote from MrJohnGalt:

Just another bunch of guys who can't understand and utilize statistics. Period.
More...
Wow! My little thread made it on the radar of the infamous MrJohnGalt, troll extraordinaire? I'm thrilled. :D

Puhleeeze share your wisdom O knowing one!
 
As I mentioned on a different thread, manual backtesting is one of the most important things I have my students do. It isn't enough for a trader to know he has a profitable method. As soon as he has a losing trade or two, he'll start doubting the method, even if others have told him how profitable the method is. He'll start looking for ways to change it, or even drop it altogether. The only way for a trader to succeed is to SEE that it's profitable, to FEEL that it's profitable. The only way to do that is by manual backtesting. That's the only way a trader will get the confidence to trade a method profitably.

All the best,

Phil
 
Quote from splphil:
It isn't enough for a trader to know he has a profitable method. As soon as he has a losing trade or two, he'll start doubting the method, even if others have told him how profitable the method is. He'll start looking for ways to change it, or even drop it altogether. The only way for a trader to succeed is to SEE that it's profitable, to FEEL that it's profitable. The only way to do that is by manual backtesting. That's the only way a trader will get the confidence to trade a method profitably.
More...
I agree on the first part. Doubt of one’s methods can kill profitability, either from lack of understanding, lack of discipline, or inability to handle drawdowns (financially or emotionally).

I don’t understand your second point. Why is there any fundamental difference between manually going through each trade and automating the process via properly tested code, including the use of spot-checks and real-time papertrading trials?

I think you are saying that manual backtesting forces the trader to think harder, building discipline and trust. This may be true for some. Furthermore, there is the risk of over-optimizing or curve fitting automatic methods. In my case, the main disadvantage of manual methods is that they are slow. I do the spot checks and live trials properly. With so many markets, time frames and styles out there to test, why be limited to manual testing?

Perhaps the real answer is to find a balance that suits one’s abilities.
 
Quote from prophet:

I don’t understand your second point. Why is there any fundamental difference between manually going through each trade and automating the process via properly tested code, including the use of spot-checks and real-time papertrading trials?
More...

a lot of us don't understand :confused:

most of the stuff in this thread has me completely confused -- logic has been thrown out the window! but again, to each his own -- if someone thinks that looking at every trade will help them then go for it....would be alot easier to just look at a curve, but who needs productivity! ;)
 
You must log in or register to reply here.
Back
Top