Help With Evaulating Intra-Day Trading Systems

[SimpleMeLike]

Hello,

So I have programmed and back test a few intra-day (5, 15, 30, and 60 minute charts) trading systems on my development platform.

My problem is I do not know how to evaluate the performance of the trading system I am programming and back testing besides looking at the net profit.

A few questions please:

1. Is profitably across multiple instruments prove a strategy robustness? See the attachment please. Strategy shows profits for EMD and ES but not so much for other instruments.

2. Why is Max Drawdown so important? I understand the meaning of it, but why important from a money simplicity understanding? Why would a client be interested in this so much and not annual profit?

3. Can you please recommend a book for evaluating intra-day trading systems?

4. When you evaluating a trading strategy performance what a few performance you look at for a quick review before moving forward to other validation methods?

5. How do you know when you have a system that have high odds of profitability in the future?

Thank you kindly.

 

[nonlinear5]

So I have programmed and back test a few intra-day (5, 15, 30, and 60 minute charts) trading systems on my development platform.

I looks like your backtest is from 2007 to 2010. That's way out of date. See if you can get more recent data.

1. Is profitably across multiple instruments prove a strategy robustness?

Yes, but it's highly unlikely that you'd find something that works well for multiple instruments.

2. Why is Max Drawdown so important? I understand the meaning of it, but why important from a money simplicity understanding? Why would a client be interested in this so much and not annual profit?

Because Max DD is a measure of risk. What the trading system should maximize is not annual profit, but the risk-adjusted annual profit.

3. Can you please recommend a book for evaluating intra-day trading systems?

4. When you evaluating a trading strategy performance what a few performance you look at for a quick review before moving forward to other validation methods?

Sharpe ratio remains the golden standard in performance evaluation. There is a lot of (deserved) criticism of it, and eventually you may want to use your own performance metrics, but Sharpe ratio is a good start.

 

[zdreg]

re:max drawdowns
a 50% move again you means that you now need a 100% move to break even.

it is not going to happen.

 

[They]

1. Is profitably across multiple instruments prove a strategy robustness? See the attachment please. Strategy shows profits for EMD and ES but not so much for other instruments.

System robustness should be measured by drawdown recovery time irrespective of what instrument it is running on. The two instruments that you are highlighting, ES & EMD, are highly correlated. Why would you demand your revert to the mean system to perform well on an instrument that is statistically proven to trend?

2. Why is Max Drawdown so important? I understand the meaning of it, but why important from a money simplicity understanding? Why would a client be interested in this so much and not annual profit?

It is important because clients with money don't want to lose it, clients with money are conservative, clients with money will not give you any if your models are exposed to high risk, no matter what the promised profit. You might find some ignorant relative or gambling friend to give you some funds but the first time they see a real drawdown they too will be demanding their funds back. Notional = emotional.

3. Can you please recommend a book for evaluating intra-day trading systems?

First develop a system that has low drawdowns and recovers quickly. (How to develop that system is another discussion) Some systems traders just put a moving average on their profits and turn the system on & off accordingly. Ultimately systems don't last forever, you need to know when to remove it from your basket of systems.

4. When you evaluating a trading strategy performance what a few performance you look at for a quick review before moving forward to other validation methods?

Is the signal or event trigger worth anything at all? Does the entrance signal give an immediate reaction? If yes, compare the MFE to MAE against time and volume, i.e., after X time or X volume where is price in relation to the original signal price. From this you can determine the strength of the signal/event.

5. How do you know when you have a system that have high odds of profitability in the future?

You don't!

They

 

[Xela]

1. Is profitably across multiple instruments prove a strategy robustness?

It's among the things that collectively prove it; yes. It doesn't necessarily in itself prove it (for example, a strategy could theoretically be profitable across multiple instruments, but for only a limited and not adequately statistically significant time-period).

I would say it's "evidential", rather than "probative".

2. Why is Max Drawdown so important?

Because it speaks to security of funds - about the most significant aspect of investing.

Why would a client be interested in this so much and not annual profit?

Because he's an experienced, perceptive, educated client? The less sophisticated a potential investor is, the more interested s/he will be in profit potential as the primary consideration, I think.

3. Can you please recommend a book for evaluating intra-day trading systems?

Unless you're avoiding automation, I recommend the Kevin Davey book already mentioned above. In a world of questionable vendors and authors, he's unmistakably "one of the good guys". I also recommend Michael Harris (see below), and I'd say that the fact that you're asking specifically about intraday systems isn't necessarily terribly relevant, here: the principles are mostly (not entirely, granted) the same, regardless of the trade durations.

4. When you evaluating a trading strategy performance what a few performance you look at for a quick review before moving forward to other validation methods?

Nowadays I have "our people" to do this, and they know far more about it than I do. In all my independent/retail-trading past, the first three things I ever looked at were - in this order - drawdown, drawdown and drawdown (I was trading my own limited funds, and wouldn't have been able to continue to make a living at all, if I'd risked losing significant chunks of it!).

5. How do you know when you have a system that have high odds of profitability in the future?

Well, "robustness" in its broadest sense ... but that's not much of an answer, is it? More a way of re-wording the question! I recommend Michael Harris's Profitability & Systematic Trading as a not-too-long and highly readable answer to this.

But there's never certainty.

Trying to read between the lines of your post above, I suspect that Ralph Vince's excellent book The Mathematics of Money Management: Risk Analysis Techniques for Traders will perhaps be particularly helpful to you (and he posts here sometimes).

 

[SimpleMeLike]

Thank you They,

1. Is profitably across multiple instruments prove a strategy robustness? See the attachment please. Strategy shows profits for EMD and ES but not so much for other instruments.

System robustness should be measured by drawdown recovery time irrespective of what instrument it is running on. The two instruments that you are highlighting, ES & EMD, are highly correlated. Why would you demand your revert to the mean system to perform well on an instrument that is statistically proven to trend?

Good point on "System robustness should be measured by drawdown recovery time "

First develop a system that has low drawdowns and recovers quickly. Ultimately systems don't last forever, you need to know when to remove it from your basket of systems.

I am developing intraday systems, no more then 2 trades per day.

How many years of back test do you consider? I am seeing when I back test for many years like the performance report I attached above, I see really big drawdowns.

If systems don't last forever, does it makes sense to back test for +5 years or over 1000 trades of back testing data. Maybe it depends on the type of system, swing trading or intraday. I am confused about the statement of systems don't last forever

 

[SimpleMeLike]

I looks like your backtest is from 2007 to 2010. That's way out of date. See if you can get more recent data.



Yes, but it's highly unlikely that you'd find something that works well for multiple instruments.



Because Max DD is a measure of risk. What the trading system should maximize is not annual profit, but the risk-adjusted annual profit.






Sharpe ratio remains the golden standard in performance evaluation. There is a lot of (deserved) criticism of it, and eventually you may want to use your own performance metrics, but Sharpe ratio is a good start.

Thank you nonlinear5

 

[nonlinear5]

How many years of back test do you consider?

The significance of your back test is proportional to the square root of time covered by that back test. The overall shape of the performance evaluation equation typically looks like this:

Performance score = (Reward / Risk) * sqrt(time)

 

[SimpleMeLike]

the first three things I ever looked at were - in this order - drawdown, drawdown and drawdown (I was trading my own limited funds, and wouldn't have been able to continue to make a living at all, if I'd risked losing significant chunks of it!).

Thank you so much Xela and thanks for the resources you provided. I saved them.

Now I am understanding the drawdown concept. If a client needs the money for income purposes then no matter how much profit s/he is made if a really big drawdown occurs then technically they are losing money.

The drawdown will be very bad if soon as start trading the system a big drawdown occurs. So yes, I see drawdown should be low. How low, I am not sure.

 

[schweiz]

The significance of your back test is proportional to the square root of time covered by that back test. The overall shape of the performance evaluation equation typically looks like this:

Performance score = (Reward / Risk) * sqrt(time)

Not sure if that is correct. Daytrading or trading in longer timeframes should, according to your statement, have the same significance as they cover the same time period of the backtest. But daytrading will generate much more trades, and to me number of trades influence significance. This leads me to the conclusion that not time but number of trades are important.
In a 1 month time daytrading 100 times cannot have the same significance as trading 4 trades that are held several days.

What is the error in my logic?