Help me be a pessimist about my automated system

B

baggerlord

I've been working on an automated forex system that at this point seems too good to be true. I first programmed it into a 60min EUR/USD chart going back 3 months, and then did some minor tweeking with the indicators. To avoid the pitfall of optimizing I then switched it to a 90 min chart without making any more changes, as this is the timeframe I orignally planned on trading. I've also plugged it into 90 min charts going back 3 months on all 8 major pairs and it returned at least 50 pips and as much as 600 on all of them! It is a low variance method so to trade 1 10k lot in all 8 of them would have only required about $1k-2k USD, making the % returns astronomical.

So, I am at this point assuming I am missing something and this is too good to be true. Help me figure what that is!

I have it set to figure in the commissions I will be paying, and I have it programmed in to calculate 1 tic pip slippage both ways. Think I should raise this? I am using neoticker to test this and I plan to use it to automate if I can't find any problems. Are there any other ways I can have it simulate real life trading more accurately?
 
I forgot to add I am waiting for 3 years of historical data to be sent to me and I won't go live until I test it on that.

Also just for the heck of it i increased slippage to 4 pips each way and it still shows huge profits.

If this thing performs at even half what it is showing it will make me rich, or am i missing something major?
 
If it trades overnight 1 pip slippage won't be enough -- you can get slammed quite a bit overnight. During the day it's probably a valid slippage setting.

If you're trading forex (and not currencies on CME) then also be prepared for some of the crap those brokers pull (I've never traded forex but I've read the horror stories on ET).

SSB
 
It has to trade 24hrs to work. I plan on trading with EFX as it is designed to work with neoticker. Any feedback on them regarding fills and reliablility?

What do you think is a valid slippage setting for night time?

Thxs for your feedback.
 
It can be very exciting when you find something that works. Its the virtual pot of gold at the end of the rainbow. I just hope you dont get there and find this:

<img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=1055521">
 

Attachments

3 months means very little, the dollar has fallen pretty much steadily since then so be a skeptic until you get more data. Is your system catching profitable moves both long and short, or just leaning to one side? etc.
 
it trades both directions, and it was profitable both ways in every pair. The trades in the direction of the overall trend were 3x more profitable that the other direction.

I plan on doing this with $ very soon, just wanted to see if there are any common pitfalls I am overlooking.
 
Quote from baggerlord:

it trades both directions, and it was profitable both ways in every pair. The trades in the direction of the overall trend were 3x more profitable that the other direction.

I plan on doing this with $ very soon, just wanted to see if there are any common pitfalls I am overlooking.
More...

Common pitfalls....

Chances are it won't work.

That is being a pessimist :cool:
 
Quote from baggerlord:

I've been working on an automated forex system that at this point seems too good to be true. I first programmed it into a 60min EUR/USD chart going back 3 months, and then did some minor tweeking with the indicators...
More...

If it looks to good to be true, it IS too good to be true. Have you followed a scientific process? I suspect that you have not, given that you are posting on Elitetrader in an attempt to understand and validate your results.

1. Observe, Classify, and Define. Can you describe theoretically how your system is supposed to generate profits within the constraints of the marketplace (fair pricing, fair value, efficient market)? Is it arbitrage? Is it liquidity provision? Does it depend on the microstructure of the market in question? Can you classify the phenomena on which the system depends and track them discretely? Does the system use a weakness of some other theoretical framework?

2. Form a Hypothesis. Use your observations, theoretical reasoning, and phenomena to develop logic that can be tested empirically. Define the relationships. Does it predict or correlate? Does A cause B? If A changes does B change?

3. Test the Hypothesis Emperically. Define the test criteria, and identify the data that is to be collected. Think out of the box and create "stress tests" for your hypothesis. Intentionally invalidate your assumptions and observe the result. Is your data valid? Are there any outlier trades that are creating unusual results? Test the system with out-of-sample data, then with random data. Does it behave as you would have predicted? Why or why not?

In your post, you spoke only of the testing process, or Step 3 above. There are several problems with your testing thus far:

1. Insufficient Data. Three months of intraday data is simply not enough samples. You have already "curve fit" by having such a small sample.

2. Improper Testing. In your post, you said that you tested the same 3 month interval with different periodicity. This is not sufficient, you need to perform "walk forward" testing. That means taking a random sample of 3 months from 3 or more years ago, then testing each 3 month interval thereafter. You have confirmed only that your rules "work" during the period in question.

3. Invalid Assumptions. Your slippage is far too small. You have to account for computer, software, network, exchange, and human error somewhere. Using 5 or more ticks per trade will give you a better "stress case".

4. Data Validation. Is there an abnormal pattern of returns somewhere in your sample? Huge price swings or trades? Are all of your trades the right size?

5. Stress Testing. Permute the variables that are inputs to your testing. Does what you thought would happen actually happen?

I will say it again, just to make sure you heard me. If it looks to good to be true, it IS too good to be true. Just ask an option trader.


-segv
 
You must log in or register to reply here.
Back
Top