My Automated Trading Strategy (3Years+ of Historical Data) Please Critique!

Quote from jedwards:

Sounds interesting!

Some questions:
-Have you forwarded tested this or is the results only from backtesting?
-How many trades a day do you get?
-Did you take into account rollover periods in your backtesting? What about spreads between bids and asks, if any?
-average drawdown of 3k and 18k max drawdown sequential? Sounds like the "average" is hiding some important information, what was your max drawdown?

Just a side note, 18k of drawdown is huge. It's one thing to look back at a backtest and see 18k of drawdown and then more winnings, but if you are in the middle of that 18k drawdown and you continue to see pain, it's hard not to think that the system is broken. You'll soon find that it's not finding a winning system that's the hard part, it's sticking with a system through an extended drawdown.

For example, let's say you take 5 losses in a row. You think "There's no way there could be a 6th loss, I've never seen 6 in my backtesting", so you double or triple down, and then you hit another loss. Things like that are what mess you up psychologically.

Anyway good luck, sounds pretty profitable if this actually works and wasn't accidentally over-optimized for your test data.
More...

So far this is all from back testing, I am going to simulate trade starting tomorrow, as I finally critiqued it today.

Spreads in oil is usually around 1-3 ticks, however the strategy will only be using limit orders, and every trade that has been executed in the backtesting has moved through my limit, guaranteeing my fill. However if for some reason I need to hit at market, it won't majorly affect my results as my margin of error is 15 ticks.

About the drawdown, yes I agree 18k is very large, and my next goal is to bring that down. However, my worst overall month was $9000 so the 18k was quickly made back. Now, this is my first time I will be going live with an automated strategy so I agree, I do not know how the drawdowns will affect me psychologically. I guess we will have to sit tight and see.

Thank you so much for taking the time to critique my strategy. I really do appreciate it, and thank you for the kind wishes.

I will be posting the stats shortly...
 
One other question, when backtesting, did you get get bars that hit your limit and then hit your target and/or stop in the same bar? If so, how did you reconcile those?
 
Quote from jedwards:

One other question, when backtesting, did you get get bars that hit your limit and then hit your target and/or stop in the same bar? If so, how did you reconcile those?
More...

The only time it is possible for that to happen is if the trade goes against me by $1500, as the strategy is always long or short, there are no targets. I only have the $1500 stop as emergency.

From what I have looked at, that has never happened yet.
 
Typically, when your backtesting gains go parabolic as yours shows, there is a very good chance there is an error in your coding. I occasionally get returns like that also in my backtesting. Initially, it made me jump out of my chair with excitment. But after further study, I find that a mistake in my coding caused for example my system to access future data. One way this can happen is if you are using an indicator that gives a result at the end of the day, and your system accesses this data at the open of the same day. Since we cannot know at the open what the result is at the end of the day, we are accessing future data.

To test for errors, I check my results manually. That is going the test results bar by bar and comparing it to my rules. Also make sure that the buy price / sell price is correct.

Once you have all the bugs out, and your results are what you expect, don't put real money on the line just yet. Test your system in real time by paper trading for at least a of couple months. It is best to use a simulator such as investopedia.com or one your broker provides you. This way, you cannot be tempted to discount any bad trades. Then compare your results to your backtest and see how it compares. If the results match, then you are ready to apply real money. Start with small amounts first to minimize the result of costly mistakes as you are learning to trade your new system. Then as you become more adept you can trade larger amounts.

All the best in your endeavors.
 
What methods did you use to drastically improve your expectancy per trade? Did you add filters? Simply put your limit order x ticks below closing price (since you said most trades go against you a good bit).

I assume additional filters, however through this trial and error were you applying filters to your entire dataset and keeping what helped? If so be careful as you may have curve fit slightly.
 
Quote from frostengine:

What methods did you use to drastically improve your expectancy per trade? Did you add filters? Simply put your limit order x ticks below closing price (since you said most trades go against you a good bit).

I assume additional filters, however through this trial and error were you applying filters to your entire dataset and keeping what helped? If so be careful as you may have curve fit slightly.
More...

Many things were changed to create the new strategy:

#1. The indicators as a whole were changed. The algorithm is completely changed, but it has the same basis in mind. (A mean reversion strategy.)

#2. The market it trades changed. It now trades oil which has a much higher average daily tick range than the s&p 500 emini.

#3. The timeframe has changed. Instead of trading on a 1 minute chart, it trades on 10+minute charts.

I actually REMOVED some filters, and to my surprise, the simpler it got, the better it became.

The new strategy was backtested with limit orders on the trigger price. Not below, or above. HOWEVER I am looking to, say, buy one tick lower and sell one tick higher, which would bring my avg trade profit up by $20.
 
Quote from DSITrader:

Typically, when your backtesting gains go parabolic as yours shows, there is a very good chance there is an error in your coding. I occasionally get returns like that also in my backtesting. Initially, it made me jump out of my chair with excitment. But after further study, I find that a mistake in my coding caused for example my system to access future data. One way this can happen is if you are using an indicator that gives a result at the end of the day, and your system accesses this data at the open of the same day. Since we cannot know at the open what the result is at the end of the day, we are accessing future data.

To test for errors, I check my results manually. That is going the test results bar by bar and comparing it to my rules. Also make sure that the buy price / sell price is correct.

Once you have all the bugs out, and your results are what you expect, don't put real money on the line just yet. Test your system in real time by paper trading for at least a of couple months. It is best to use a simulator such as investopedia.com or one your broker provides you. This way, you cannot be tempted to discount any bad trades. Then compare your results to your backtest and see how it compares. If the results match, then you are ready to apply real money. Start with small amounts first to minimize the result of costly mistakes as you are learning to trade your new system. Then as you become more adept you can trade larger amounts.

All the best in your endeavors.
More...

Yep I spent about an hour today checking all through the buy and sell signals and all were correct. I had the same reaction...this can't be true. I guess I found an edge, and I might as well ride it until it goes away.

Yes I am going to start to simulate trade it tomorrow. I will continue to update this thread as I go along.

Thanks for the advice!
 
OK! Here are the stats!

Stats Numberwise:

<a href="http://content.screencast.com/users/UATrading/folders/Automated%20Strategies/media/b273cafe-e2f9-4062-b34b-ddf54f1f7831/Stats.png"><img class="embeddedObject" src="http://content.screencast.com/users/UATrading/folders/Automated%20Strategies/media/b273cafe-e2f9-4062-b34b-ddf54f1f7831/Stats.png" width="893" height="706" border="0" /></a>

Drawdown:

<a href="http://content.screencast.com/users/UATrading/folders/Automated%20Strategies/media/74043cec-fe17-4cf8-9559-f5863d92643a/Drawdown.png"><img class="embeddedObject" src="http://content.screencast.com/users/UATrading/folders/Automated%20Strategies/media/74043cec-fe17-4cf8-9559-f5863d92643a/Drawdown.png" width="894" height="654" border="0" /></a>

Monthly P&L:

<a href="http://content.screencast.com/users/UATrading/folders/Automated%20Strategies/media/e1022268-7488-40a3-8298-bee3de335822/Monthly%20PL.png"><img class="embeddedObject" src="http://content.screencast.com/users/UATrading/folders/Automated%20Strategies/media/e1022268-7488-40a3-8298-bee3de335822/Monthly%20PL.png" width="894" height="654" border="0" /></a>

Cumulative P&L:

<a href="http://content.screencast.com/users/UATrading/folders/Automated%20Strategies/media/876c49cd-e83c-4d85-abd8-692de73720f7/Cumulative%20Profit%20Graph.png"><img class="embeddedObject" src="http://content.screencast.com/users/UATrading/folders/Automated%20Strategies/media/876c49cd-e83c-4d85-abd8-692de73720f7/Cumulative%20Profit%20Graph.png" width="894" height="654" border="0" /></a>

This is backtested on about 3 years of historical data in oil.
 
What software did you use to backtest, or did you go through everything manually?

Are you only using 10 min charts to evaluate whether or not your trades get filled, or are you drilling down to smaller time-frame charts to verify that the trades are actually proceeding the way that you think they would? I can honestly see a bunch of situations where 10 min candles may give you false signals depending on the order of how your various points get hit. A move for CL of 150 ticks in 10 mins is a lot, but not unheard of, especially during the Wed crude reports.5

I guess you are always in a trade, either long or short, and you just switch back and forth between the two. You said, however, that you have a stop loss of $1500/trade. After you hit a stop loss, how do you reestablish your trade? You just take the next signal in the same direction as your last one?
 
Quote from jedwards:

What software did you use to backtest, or did you go through everything manually?

Are you only using 10 min charts to evaluate whether or not your trades get filled, or are you drilling down to smaller time-frame charts to verify that the trades are actually proceeding the way that you think they would? I can honestly see a bunch of situations where 10 min candles may give you false signals depending on the order of how your various points get hit. A move for CL of 150 ticks in 10 mins is a lot, but not unheard of, especially during the Wed crude reports.5

I guess you are always in a trade, either long or short, and you just switch back and forth between the two. You said, however, that you have a stop loss of $1500/trade. After you hit a stop loss, how do you reestablish your trade? You just take the next signal in the same direction as your last one?
More...

I am using Ninja trader 7 for all backtesting.

All trades are executed on the open of the bar AFTER the trigger bar, and all trades are made on a 10 minute bar chart, nothing else.

In the event where we have a bar that goes against me 150 ticks, and is the entry bar, nothing will have changed.

Ex. Trigger bar is long, next bar opens at 109.00. I am filled at 109.00 long, and immediatly my stop order is place at 107.50. 5 minutes after the entry, oil drops to 107.50 and I am out. Nothing should change wether I am 10 min or 1 hour after the entry.

Once a stop loss is hit and the system is flat, the strategy just waits for a new signal, long or short, and continues on.
 
You must log in or register to reply here.
Back
Top