Damn you, out of sample trades!

L

logic_man

It always seems to happen right after you either optimize or find something new, that the first batch of trades sucks.

I rolled out a "scalping" system this week which during backtest had over 90% wins and an average trade expectancy of ~$200/contract.

So, what happens? I have an 83% win rate through the first 18 trades and an average expectancy of -$32/contract per trade. The reason for the negative expectancy is that it's deliberately designed to have very few, but big, losing trades. I can probably work on optimizing the exits, but since I had such good results in the backtest, it wasn't of major concern.

To make matters even more annoying, I got stopped out by a single tick on a CL trade right before the Fed minutes, when the move right after the Fed minutes would have made that a winner. In my backtesting, CL hadn't had a losing trade in a month and now it's had two in week of live trading.

Whoever says this crap is easy is kidding themselves.
 
Quote from Jack_Larkin:

sounds more like curve fitting...

how many years back did you test?
More...

I'd say it was curve fitting if it weren't for the fact that the risk-reward ratio is so skewed toward large risks (and hence large losses), so it's not as if it is some kind of perfect system. Otherwise, I would never expect to see anything with a 90% win rate. There is some optimization as regards to trade selection, but these are very objective. For example, if I get a signal in the opposite direction of a signal that's already in effect, I ignore that second signal because I know from my data that those second signals are more likely to fail to reach the profit target and get stopped out. I don't consider that curve fitting to use that as a filter. So, completely unoptimized, the expectancy is $43/contract per trade and the win rate is in the mid-70% range. I've got about 700 trades in my sample from 3 markets for the past 6 months to a year. I have also worked with this entry method for nearly 3 years now and know from that that there has always been a high win rate for this particular type of trade.

Also, if it were curve-fitting, I wouldn't expect to have an 83% win rate, even on a sample of just 18 trades. If it were truly a coin-flip entry method at its base, the odds of 15 wins in 18 trades would be on the order of less than 0.5%.

It's just a really good entry and profit target selection method. The only problem is that it does rely on these huge win rates because the losses are 3-4X the size of the average winner.
 
Quote from logic_man:

In my backtesting, CL hadn't had a losing trade in a month and now it's had two in week of live trading.
More...

I learned many lessons about CL during my first year of trading it and posted a critical conclusion that you're probably not accounting for:

Quote from NoDoji:

CL lives in a cave far from civilization.
More...

:eek:
 
Quote from Random.Capital:

Yes, to me, too.

Is this "scalping" happening from a colo, or from a consumer-class connection?
More...

Well, I call it "scalping" but it's not really that. The profit target can be pretty far away, e.g. $1 or more in Crude. It's "scalping" relative to my typical trade, though, and the target can also be .01 away from the entry. It all depends on the context.

See my post above in reply to the curve-fitting possibility. I really don't think it is since with curve-fitting, you would be more likely to see both the win rate and the average win size completely diverge from the backtested data. Here, it is really only the win size which is diverging.

But, it is an example of how new ideas always seem to stumble out of the gate, for me at least.
 
Quote from NoDoji:

I learned many lessons about CL during my first year of trading it and posted a critical conclusion that you're probably not accounting for:



:eek:
More...

Yeah, the stop-out by one tick was pretty "uncivilized" :)
 
Quote from logic_man:

I've got about 700 trades in my sample
More...

But with a 90% win rate you have a sample of only 70 losses. That's not a big sample given that it's the losses that are more significant.
 
Quote from logic_man:

I really don't think it is since with curve-fitting, you would be more likely to see both the win rate and the average win size completely diverge from the backtested data.
More...

You can curve-fit either way.
 
You must log in or register to reply here.
Back
Top