Feedback on System Healthiness

[annabellab]

I'm wondering if I can get some feedback on the back testing results from a few equities which I have put together to trade systemically.

Results are from the last 2 years data. Commissions, slippage, have been accounted for, as well as backtested using look inside bar.

This is a Day Trading System, all positions are closed by the end of the day.

Thanks for your time!

RIMM 5 Min

Profit Factor 1.48
Average Trade Net Profit $35.00
Annual Rate of Return 10%
Average Monthly Return $900.00
Total Trade $ Profit divided
by Intraday Max Draw Down
Peak to Valley 6.4
Total Shares 200

GOOG 10 Min

Profit Factor 1.90
Average Trade Net Profit $93.00
Annual Rate of Return 20%
Average Monthly Return $1717.00
Total Trade $ Profit divided
by Intraday Max Draw Down
Peak to Valley 8.9
Total Shares 105

CME 5 Min

Profit Factor 1.63
Average Trade Net Profit $67.00
Annual Rate of Return 21%
Average Monthly Return $2150.00
Total Trade $ Profit divided
by Intraday Max Draw Down
Peak to Valley 7.6
Total Shares 100

ICE 5 Min

Profit Factor 1.76
Average Trade Net Profit $62.00
Annual Rate of Return 15%
Average Monthly Return $1272.00
Total Trade $ Profit divided
by Intraday Max Draw Down
Peak to Valley 9.5
Total Shares 200

OIH 2 Min

Profit Factor 1.64
Average Trade Net Profit $45.00
Annual Rate of Return 10%
Average Monthly Return $890.00
Total Trade $ Profit divided
by Intraday Max Draw Down
Peak to Valley 12.1
Total Shares 200

 

[annabellab]

I added and excel file with more information. I hope there is somebody out there with feedback, positive or negative.

 

[TrueStory]

First of all, good luck!

The system looks fine and could be very tradeable (sp?), but much more information is required...

1) Is this the same system applied to each of these equities? If so, why and how did you choose those stocks? Does the system work on other stocks also? Etc. etc. etc...

2) You list the systems on more than 1 timeframe. Why? If this was a requirement to make each one profitable, that could be a problem.

3)Did you test the systems on out-of-sample data? If not, you MUST do that to get any idea if the system is good. Must, must, must.

You have avoided the obvious pitfalls such as not including comissions and slippage, not using "look-inside bar backtesting," etc. You may have a very tradeable system, but you just need to provide a little more information to be sure.

PM me if you have any other questions about properly testing your system's robustness. I test systems most of every day...

Good luck!

 

[annabellab]

Thanks True Story!

Ive updated the file with out of sample data. CME 3 min seemed to be the only one which faired ok in out of sample. I also added charts on the second worksheet of the file.

What exactly can we gauge from comparing the out of sample results with the in sample results?

 

[annabellab]

True Story, you were very correct about the importance of out of sample results.. Ive decided to go ahead and reoptimize from June 04 to June 06 as in Sample, and Out of Sample from June 06 to current to compare the results. If anyone has any feedback about sampling please let me know.

 

[TrueStory]

Quote from annabellab:

True Story, you were very correct about the importance of out of sample results.. Ive decided to go ahead and reoptimize from June 04 to June 06 as in Sample, and Out of Sample from June 06 to current to compare the results. If anyone has any feedback about sampling please let me know.

What you can gauge from the out of sample results not matching is that your system is not yet robust enough. It should perform in a similar fashion, although rarely identical.

I would recommend using more data for out of sample. I don't know if your data is limited, but I would try to get the out of sample amount up closer to a 1 to 2 or even 1 to 1 ratio to your in sample. This way you can get a better idea of how your system is doing when it has not been optimized on a particular set of data.

Additionally, I would test more stocks and try to find a range of parameters that works well across a basket of tradeable stocks. At that point, you will have much reason to believe your system is a quality and profitable one. Hope that helps!

 

[annabellab]

This is great advise! Thank you so much True Story.