Not exactly, because in the 700+ sample, the win rate is closer to 80%. But, there's nothing I can really do about that. Getting a really big sample of losses would require me having more data and even if I bought the data, I can't really do any automated backtesting because the algorithm is so custom that it would require me to build a backtesting engine from scratch and I just don't have the desire to do that. I believe in backtesting, but I also trust that my nearly 3 years of experience with this general approach have taught me what does and doesn't work. At least, I think that this will eventually get up and running and generating profits for me, but it has gotten off to a less than stellar start.
Damn you, out of sample trades!
- Thread starter logic_man
- Start date
Not exactly, because in the 700+ sample, the win rate is closer to 80%. But, there's nothing I can really do about that. Getting a really big sample of losses would require me having more data and even if I bought the data, I can't really do any automated backtesting because the algorithm is so custom that it would require me to build a backtesting engine from scratch and I just don't have the desire to do that. I believe in backtesting, but I also trust that my nearly 3 years of experience with this general approach have taught me what does and doesn't work. At least, I think that this will eventually get up and running and generating profits for me, but it has gotten off to a less than stellar start.
Yes, in-sample you can because that's where you have the control to either select or reject trade outcomes based on parameter values.
I had no control over how the market played out when I started taking trades, yet the win-rate was at least within spitting distance of the sample win-rate. In addition, the win rate in the sample was phenomenally high, yet nearly everyone agrees that "random entry" win rates will tend toward the 50% mark. As I said, using that 50% as the "coin flip" version of trade entry, there is a less than 0.5% chance of getting 15 winners in 18 trades. I don't know of anyone who can "curve-fit" their way to an 83% win rate in actual trades. In fact, if you take away that one-tick Crude stop-out today and put the "good outcome" for that trade in it's place, I'd be in the black overall, although not by as much per average trade as I would have expected from the backtest. So, what I'm complaining about is really a marginal difference between the backtested results and the actuals hinging on the results of 1 trade.
So, if you want to engage in a discussion about curve-fitting, fine, but you should at least acknowledge that the probabilities are not on your side.
when naive , newbie developers use only 6 months of data to backtest...
it will ALWAYS happen.
and these folks are the first to nag and cry on forums how backtesting and optimization are bad
sometimes they use 10 years of data, but build systems which make <150 trades -- the same crappy shit.
whats worse- hedge fund managers are not smarter than that.
so after you worked your ass off developing robust models for 5 years...
traded them, etc.
the clown (with IL education, of course) can blow you off saying "but i'd rather see a track record on at least a $1M...."
f*** you Sir. this is precisely why most firms lose. they think that out of sample results generated on >1m account have some value.
They do not. But let them live in their fairy tales
it will ALWAYS happen.
and these folks are the first to nag and cry on forums how backtesting and optimization are bad
sometimes they use 10 years of data, but build systems which make <150 trades -- the same crappy shit.
whats worse- hedge fund managers are not smarter than that.
so after you worked your ass off developing robust models for 5 years...
traded them, etc.
the clown (with IL education, of course) can blow you off saying "but i'd rather see a track record on at least a $1M...."
f*** you Sir. this is precisely why most firms lose. they think that out of sample results generated on >1m account have some value.
They do not. But let them live in their fairy tales
I'm betting this gets one star
No, I don't accept that as correct. "Nearly everyone" does not agree with that statement, and in fact the statement is incorrect, as it is trivially easy to create systems with very high win rate using completely random entries.
ya I know......I'm an et new be and I think all of my threads so far have one star too
So, you have a net loss of about -600 / contract after 18 trades ... this means absolutely nothing.
1. What peak P&L did you reach, and is that in-line with your backtesting ?
2. What was the worst drawdown in 18 trades in backtesting? How far is that from your current drawdown?
3. What max drawdown did you decide on for that system, before taking it live? What does the current drawdown represents, in % of that?
4. How reliable is your backtesting? Quality of data, ability of backtesting to accurately represent real-time behavior of your system?
BTW, 1pt in CL is currently about 44% of the 20-days ATR (which has been declining steadily from ~2.95 beginning of July to ~2.25 as of today ... calling your system "scalping" is pushing the envelope IMO, even if average risk >> average reward ... what's your average time in trade in backtesting ?
Hmmm, well, then I suppose you'll have no problem providing a link to such trivially easy to create systems, right?
Not sure why you seem so insistent that I'm curve-fitting, but whatever floats your boat.
The basis for this system, which I don't always use for this particular purpose, has always been a 60%+ win-rate system in live trading, but it is more of a intraday/overnight swing trading system. When it really gets on a roll, it can be an 80% winner for a few months at a time. All I have done is change the way in which the trade is managed by putting a profit target in there at the front-end of the trade, rather than letting the entire trade play out. I suppose now that you are going to say that other system is itself curve-fit, right? Then I should go to Vegas and just put it all down on whatever number I want because I'm luckier than I thought I was.
Quote from DT-waw:
when naive , newbie developers use only 6 months of data to backtest...
it will ALWAYS happen.
and these folks are the first to nag and cry on forums how backtesting and optimization are bad
sometimes they use 10 years of data, but build systems which make <150 trades -- the same crappy shit.
whats worse- hedge fund managers are not smarter than that.
so after you worked your ass off developing robust models for 5 years...
traded them, etc.
the clown (with IL education, of course) can blow you off saying "but i'd rather see a track record on at least a $1M...."
f*** you Sir. this is precisely why most firms lose. they think that out of sample results generated on >1m account have some value.
They do not. But let them live in their fairy tales
I'm not against backtesting or optimization, at all. What I'm slightly venting about is that I would have thought after 18 trades I'd be in the black, but I'm down slightly and that this has happened to me a few times before when I roll out a new idea or optimization.
That said, I'm net profitable so it's not as if these ideas don't pan out, they just get off to slower than expected starts.