Milk it or scrap it?

A

Agyar

So... I've developed a system with the following characteristics. This is only one year of tick data on ES because that is all I have.

Number of trades: 150
Profit Factor: 2.41
% profitable: 60.5%
Avg. win/Avg. loss: 1.57
Sharpe: 1.2

There is a max of one trader per day. This system seems to be pretty successful at catching some big moves. I have a 5 point stop on it, but I can vary that from 3 to 7 and not get much of a change in results. There is no profit target, it just closes position at the close. It works fairly well on the YM for the 6 months of tick data I have there, but doesn't work at all on the ER2. It is all limit orders except the profitable close and I only count the fill if my price gets completely traded through, so I'm not too worried about slippage. There is really only one other parameter and I can vary that in a 2 point range and not get much of a change in results, so I don't think I am overoptimized either.

So my questions are

1. What else should I be looking at to make sure this is "robust"? Besides trying to get my hands on more tick data.

2. Would you trade this system? Why or why not? My inclination is to forward test it for a few months and if it looks in line with my past testing to give it a go.

Ultimately, my high level question is, do you think a system needs to be robust over many years/instruments/etc to be usable, or would you use a system that seems to be doing very well in current conditions and just monitor it for failure?
 
Quote from Agyar:

So... I've developed a system with the following characteristics. This is only one year of tick data on ES because that is all I have.

Number of trades: 150
Profit Factor: 2.41
% profitable: 60.5%
Avg. win/Avg. loss: 1.57
Sharpe: 1.2

There is a max of one trader per day. This system seems to be pretty successful at catching some big moves. I have a 5 point stop on it, but I can vary that from 3 to 7 and not get much of a change in results. There is no profit target, it just closes position at the close. It works fairly well on the YM for the 6 months of tick data I have there, but doesn't work at all on the ER2. It is all limit orders except the profitable close and I only count the fill if my price gets completely traded through, so I'm not too worried about slippage. There is really only one other parameter and I can vary that in a 2 point range and not get much of a change in results, so I don't think I am overoptimized either.

So my questions are

1. What else should I be looking at to make sure this is "robust"? Besides trying to get my hands on more tick data.

2. Would you trade this system? Why or why not? My inclination is to forward test it for a few months and if it looks in line with my past testing to give it a go.

Ultimately, my high level question is, do you think a system needs to be robust over many years/instruments/etc to be usable, or would you use a system that seems to be doing very well in current conditions and just monitor it for failure?
More...

Agyar,

Your results look very good to me. In fact, they may be too good; but you may have something here.

For a system that uses llimit orders to get in, I'd be very careful in a backtest with any assumption that you got in at the price you think. However, you did say that you only entered if the price traded through your price. From my testing, this is very important; otherwise you can't ssume that you've entered the trade at all.

"1. What else should I be looking at to make sure this is "robust"? Besides trying to get my hands on more tick data."

I think robustness has primarily to do with the number of parameters used in your system in relation to the number of trades in your backtest.

Do you really need tick data? You can get good quality 1 minute ES data very cheaply.

"2. Would you trade this system? Why or why not? My inclination is to forward test it for a few months and if it looks in line with my past testing to give it a go."

Without knowing more about the system, it is hard to say whether it is worth trading. However, it does look good. Certainly forward testing is warranted.

Any chance you could post an equity curve?

Richard
 
This is only one year of tick data on ES because that is all I have.
More...

IMO, designing and testing a system on the same data is almost always worthless, or at least, very poor evidence of the viability of the system.

It is surprising, but it is very easy to design a system on a year's data that trades over 300 times with a very consistent equity curve, that when you test it on the next year's data, it is a total disaster.

You'll need at the very least to test your system on data that was not used in its construction.

Fletch
 
Quote from fletch2:

IMO, designing and testing a system on the same data is almost always worthless, or at least, very poor evidence of the viability of the system.

It is surprising, but it is very easy to design a system on a year's data that trades over 300 times with a very consistent equity curve, that when you test it on the next year's data, it is a total disaster.

You'll need at the very least to test your system on data that was not used in its construction.

Fletch
More...

Yes I agree with what you are saying to some extent... but then I come back to my high level question. If you had a system that was making a ton of money, but you KNEW it was based on "market character" as acrary would say, would you still trade it and just monitor it for failure? It seems a shame to just scrap it.

The max drawdown on my testing is only 9%. If I got much past that in any kind of real trading I would be pulling the plug for sure.
 
Quote from Raystonn:

Did you include commissions in this? What is your average trade net profit?

-Raystonn
More...

I did not include commissions, but they aren't too much of an impact as this system only trades about 3 times per week.

The net profit is about 2 points per contract per trade, after commissions.
 
Quote from rickty:

Agyar,

Your results look very good to me. In fact, they may be too good;

That's what I'm afraid of.


Do you really need tick data? You can get good quality 1 minute ES data very cheaply.

I think I could get away with 1 minute data actually now that I think about it. For some of the other systems I'm trying to develop it wouldn't work, but for this one I think it would.


Any chance you could post an equity curve?

Richard

I would but my Multicharts beta just coughed up a lung and I can't even get it to load any more. Tried a full uninstall/reinstall and no dice. Life is conspiring against me getting to work on this for the next few days, but I'll try to get something up here eventually.

Thanks for the comments.
 
You must log in or register to reply here.
Back
Top