Is there something wrong with this system?

F

frostengine

I recently back tested a system that over the past 47 trading days woudl generated 6k per Ym contract...

Had 31 Winning Days and 16 loosing days, biggest loosing day was $500 which was actually A LOT MORE than every other loosing day, most loosing days only lost around $120;

I am calculating commisions and 1pt slipage each way.

Average trades is 2 per day.....

My question is these results seem a little too good to be true.... are these numbers too good or could be expected from a good system?

I only tested on 47 back testing days, which I know is a small population..... however its all the 15-s YM data i had avaiable.... i am going to test this same method on 10 more trading days basically from the start of this month till now, and see if those 10 days produce similar results as i have already recorded..

Is this without a doubt too small of a sample size?

Another thing to keep in mind, is the same system was run 200 times with the main parameters varied within a certain range everytime, and the system was profitable in every instance.... with profability anywhere from 800 to the most being this 6k... with the average system falling in at 3500.....
 
There is nothing wrong with system but your sample period is simply not long enough to draw any conclusions about performance.

You would want a sample of a least 400 trades to establish the statistical significance of your system. If your doing two trades a day, then you're about 300 days short on your data.

If your system performs well over 400 trades, you may just be onto something.

Runningbear
 
usually when I get something that looks too good to be true, the TS platform has some
quirks about handling certain types of orders in backtesting. The latest being a strategy trading in the 1 tick data level. It tested something like 4-5grand profit per month on one contract. It ultimately turned out to be the testing platform's ability to properly handle sub 1 min data.

I run into pipe dreams like these on average once every 3 month.
 
You also may want to start out testing swing/intermediate trading systems and then once done successfully you can move on to intraday.

The best mechanical intraday mechanical systems IMO are the ones that identify direction and hold the position through the close. These also tend to mimmick their hypothetical performance much better. Testing scalping/frequent trading systems in TS can turn out to be a slippage /order fill nightmare. I dont think TS is up to the task.
 
Just to clear up a few things regarding this system... a few of you have assumed that i used TS to do my backtesting... actually the software i'm using to do my back testing is one I programed myself, and the good results did not originate because of an error running the specific system....

However after getting my hands on aprox another 30 days of data... i notice the system did not keep up with the numbers..... i'm now testing on 75 days worth of intraday data and although the system is still profitable in every instance when the parameters are adjusted... the profit is actually a little lower than the total profit when run for only 47 days......

Instead of 6k profit per contract over the 47 day period, its now a 5100 profit over a 75 day period...... Not sure what sort of conclusions to draw from this....

The origional days may have just been perfect/lucky days for the system and it went on a good run..... or the new days could have just been a bad streak for the system.... I guess only way to tell is to get more data... will try and get another month of data and see what happens... although I think i'm almost as far back as IB keeps intraday data.....
 
A follow up....

After further testing, now with 92 trading days, the profit is now only a $2200 per contract........ All new data I have been adding has been dropping the profitability of the system.. and now the loosers are actually outpacing the winners when with only 45 days the system almost had a 2 to 1 win vs loosers advantage....


So i guess when numbers seem too good to be true, they normally are :)
 
You actually, really thought you could come up with a profitable system on the Dow? You're wasting your time, bud! Not that the others are any better but the Dow is a joke.


Quote from frostengine:

A follow up....

After further testing, now with 92 trading days, the profit is now only a $2200 per contract........ All new data I have been adding has been dropping the profitability of the system.. and now the loosers are actually outpacing the winners when with only 45 days the system almost had a 2 to 1 win vs loosers advantage....


So i guess when numbers seem too good to be true, they normally are :)
More...
 
About backtesting- its better to test on about half of the days of data you have available- then test out the system over the full range of days. This will help avoid curve fitting - and will let you see how you do on out of sample data. Each time you make a change only test on half the days. Switch up the days if possible - so you aren't always testing on the same in the same order. Developing a system that works well over the past X amount of days doesn't prove much - you would have to let it run for a while on new data before really seeing if the system works.
 
i backtest for a minimum of 1 year...in certain case they DO seem to good to be true!

Look for a profit pattern that starts out profitable and then slowly grows over time rather than a pattern that has you negative half the time or whips you up and down. I have discovered that the simpler i make the system, the more profitable it becomes (but that is probably due to my walnut sized brain)...hope that helps!

..dont want to be a dick..but i may as well point out the difference in "loosing" and "losing" form your original post (since it is a word we use a lot on this board. I once had the problem of spelling "spanish" as "spainish"... I once had a friend with the last name of Loser..pronounced "LOW-seerrr). poor bastard. And we were in a military academy were everyone used last names...the guy had a name tag that said "loser"...hahaha!

good luck
 
You must log in or register to reply here.
Back
Top