ES method - How to duplicate backtest?

gwb-trading · Apr 16, 2004

Does the the back-testing of the system take into account slippage (0.2% in & out min), commissions (worst case), and the probability that you will get filled.

What if 20% of the trades did not get filled. What if these were the 20% best trades. Did you perform a monte-carlo analysis to determine the robustness of the system, or look at it from a "zero test" perspective.

There are a number of good tools available for automating, and providing this information. One is Wealth-Lab http://www.wealth-lab.com

Hang in there, 98%+ of the systems that I have programmed don't cut it..... any quant hedge fund has the same issue.

- Greg

mrpace · Apr 17, 2004

Is it then next to impossible to come up with an effective real world system that takes advantage of high win % and low win/loss ratios?

Typically, high winning percentages and low win/loss ratios are indicators of a system that holds for only a very short while, and a system that will be quite vulnerable to slippage.

So, to ask a stupid question, how do scalpers do it? Are there no ultra-short term ES traders out there? (another stupid question)

harrytrader · Apr 17, 2004

Quote from mrpace:

Not to be confrontational, but how does it not work on paper? It has a positive mathematical expectation. It shows a decent profit over a five year test period over thousands of trades. How does it not work on paper?

I understand why it probably won't work in the real world, but what about what I see on my backtest charts?

No it doesn't show a positive mathematical expectation, it shows an GLOBAL profit that can be due to PURE CHANCE
http://www.elitetrader.com/vb/showthread.php?s=&threadid=30991&perpage=6&pagenumber=9
answer I made to a silly article pretending that you could make +54000% !

Re: Yang on Moving Averages
SQRT((261*(2002-1970))-1)*1*100*3 = +/- 27415 %

this is just a very rough order estimation of performance for buying or selling randomly between 1970 and 2002 that is to say a person could make a POSITIVE performance of an order of 27415% and an other a NEGATIVE performance of - 27415% and not be more or less competant than the first one whose people could think he has a financial genious touch whereas it could be just by chance . And since I used an understimated law by taking a random normal law this order should be much greater in reality and reach 50000 or 100000% so that the numbers below are not significant statistically and that's what economists have already said .

This is a classical flaw in Stock Market similar to the gambler's flaw in casino gambling. The more frequent the trading and the more expanded the total time period the more big the cumulative percentage. As an other consequence and for exactly the same reason it is also a flaw comparing 1 or 2 days moving average results with 200 days moving average because the law of variation is not the same and is more volatile by definition for short than long MA.

This is kind of myth like the martingale for gamblers keep people abreast of finding the holy grail for making eally rich without any effort ... and real edge . Of course counting on chance it's always possible but it is also possible to get ruined as rapidly : -50000% could be as probable as -50000% cumulating all the years.

The vendors of trading systems use the same kind of trick for showing superformance. In fact many if not all trading system testers are flawed with that kind of presentation tool when it is the main if not single possibility of visualising performance. This is only "apparent" performance not true that is to say comparable performance.

mrpace · Apr 17, 2004

yes it does show a positive mathematical expectation.

Probability * W/L - (1 - Prob)

.64 * .65 - (1-.64)=.056 which is a positive expectation over 7000+ trades

Pure chance? Here's the performance breakdown by contract:

Code:

Contract	P/L 	Winners	Losers	 Total$Win 	 Total$Loss 		
@ESH4	$1,134.00 	112	58	 $3,674.90	$(2,540.90)		66%
@ESH2	$408.90 	176	106	$7,180.20	$(6,771.30)		62%
@ESM2	 $(523.20)	216	143	$7,975.70	$(8,498.90)		60%
@ESU2	$3,844.10	319	164	$17,956.30	$(14,112.20)		66%
@ESZ2	$1,469.20	223	148	$10,629.60	$(9,160.40)		60%
@ESH3	 $361.80 	178	106	$7,208.10	$(6,846.30)		63%
@ESM3	 $(172.80)	173	113	$6,294.60	$(6,467.40)	TRADES	60%
@ESU3	$1,820.10	158	80	$6,266.60	$(4,446.50)	7813	66%
@ESZ3							
@ESH1	$3,936.50	327	168	$16,842.90	$(12,906.40)	Win%	66%
@ESM1	$4,136.50	324	171	$17,094.80	$(12,958.30)	64%	65%
@ESU1	$1,263.30	245	148	$11,161.50	$(9,897.90)		62%
@ESZ1	$(1,111.60)	228	139	$11,018.10	$(12,129.70)	P/L	62%
@ESH0	$(3,684.80)	285	191	$15,682.00	$(19,366.80)	$36,509.80	60%
@ESM0	$6,761.00	371	184	$29,281.70	$(22,520.70)		67%
@ESU0	 $(160.60)	228	144	$10,005.60	$(10,166.20)	AvgWin	61%
@ESZ0	$3,277.70	315	161	$18,725.50	$(15,447.80)	 $51.44 	66%
@ESH9	$6,828.70	324	157	$19,582.30	$(12,753.60)		67%
@ESM9	$4,873.60	299	144	$17,027.30	$(12,153.70)	Avg Loss	67%
@ESU9	$2,479.00	261	134	$11,484.70	$(9,005.70)	 $(78.73)	66%
@ESZ9	 $(431.60)	244	148	$12,403.80	$(12,835.40)		62%
						W/L Ratio	
						0.65

The percentages listed to the far right are the Win% per quarter (a few get blown out to the right). The totals column is the second to last column listed, and it sums up the whole mess.

Of course I'm not still trying to claim this system is tradeable in real life, but on paper, how can you say it doesn't have a positive mathematical expectation?

BKuerbs · Apr 17, 2004

Quote from mrpace:

....
Of course I'm not still trying to claim this system is tradeable in real life, but on paper, how can you say it doesn't have a positive mathematical expectation?

What is the worth of a positive mathematical expectation on paper? You need it in real trading. It is easy to backtest any system to get a positive expectation. What is the use of this?

Some remarks:

IMO, you use the term slippage the wrong way. This term applies to a market order, where you enter an order at a certain price level and cannot be sure to get your order filled at this price. You want to use limit orders, at least for entries. Then you do not have slippage, but have the risk of getting no fill at all.

You may get positive slippage, a fill at a price better than your limit; either you have set your limit very generously, or it is (mostly) bad news.

This leads the a question: how robust is your system as to fills? Let's assume you have a series of trades numbered 1...10. Now assume you get no fill for trade #4. Will the entry signal for trade #5 be the same now, at the same price and time as in the original series or will it be changed (most likely)? Then test your system again assuming every 4th, 5th and so on .. trade is not filled. What are your results now? At what number of non-fills does your system start to fall apart (if at all)?

Next question: what data did you use to test your system? Looks like you are using data from the IB data feed?

IMO, you have to use to different datastreams for these short term systems: one for signal generation, the one you will use in real trading to generate your entry/exit signals. And the original T&S data from the exchange for order execution.

Most data feeds will not show you every trade, like IB's they will show you a snapshot only. Your signal generation will depend on it, so you have to use it in your tests for this purpose. But your order at the exchange will be subjected to the full order stream, you have to use this when evaluating your order execution.

There is a lot of noise in the data, especially when the going gets fast, your data feed will not see all of this noise, but your limit order and your stop loss order will. This may work in both directions, to your favor and to your disadvantage: you will get stopped out at prices, you do not see in your data feed, especially when the price seems to hover just above your stop loss. Your test will show you a winner, while in real trading you have suffered a loss.
This may work to your advantage, when you use this noise to get out of a bad trade at a better price than your stop loss would.

Do not forget to enter a time delay when you test with two data streams, your order takes time to arrive at the exchange. The ES can move very fast.

It is safest to assume no fill at all, when the price does not trade through your limit. This is a bit conservative, but you would have to build complicated models when you want to evaluate your position in the waiting queue.

Regards

Bernd Kuerbs

DT-waw · Apr 17, 2004

Quote from gwb-trading:

Does the the back-testing of the system take into account slippage (0.2% in & out min), commissions (worst case), and the probability that you will get filled.

As BKuerbs pointed out, you either:
- use market orders and include slippage
OR
- use limit orders and simulate probability of getting filled without incl. slippage.

0.2% in&out is too high for ES. It's 2.2 pts. Taking ES high liquidity into an account I would estimate slippage at 0.5 pts in&out.

I wonder if anyone has a good performing ES system with over 300 trades/year and 0.6 pts/RT slippage&commissions. Your system generates only 700 ES pts/5yrs without slippage.
With lower trade frequency you could get 1500 pts/5yrs incl. slippage. If your system has low DDs ~30-50 pts, then it is worth to test it again with realistic order fill simulation.

harrytrader · Apr 17, 2004

On paper for me doesn't exclude that you must take into account some noises that are well known in advance slippage, fees,... If it was a swing system (W/L >> 3) you don't need to estimate them but when it is scalp (W/L < 1) realisation really makes all the difference between winning and losing. There are inherent realisation conditions that could be estimated in advance so that could be done on paper and there are realisation conditions that cannot be estimated very precisely in advance like your own ability to execute perfectly your plan but at least you should take into account the first ones. Mostly you risk to work for your broker ... with your own money and taking all the risk.

Code:
Quote from mrpace:

yes it does show a positive mathematical expectation.

Probability * W/L - (1 - Prob)

.64 * .65 - (1-.64)=.056 which is a positive expectation over 7000+ trades

Pure chance? Here's the performance breakdown by contract:

Code:

Contract P/L Winners Losers Total$Win Total$Loss @ESH4 $1,134.00 112 58 $3,674.90 $(2,540.90) 66% @ESH2 $408.90 176 106 $7,180.20 $(6,771.30) 62% @ESM2 $(523.20) 216 143 $7,975.70 $(8,498.90) 60% @ESU2 $3,844.10 319 164 $17,956.30 $(14,112.20) 66% @ESZ2 $1,469.20 223 148 $10,629.60 $(9,160.40) 60% @ESH3 $361.80 178 106 $7,208.10 $(6,846.30) 63% @ESM3 $(172.80) 173 113 $6,294.60 $(6,467.40) TRADES 60% @ESU3 $1,820.10 158 80 $6,266.60 $(4,446.50) 7813 66% @ESZ3 @ESH1 $3,936.50 327 168 $16,842.90 $(12,906.40) Win% 66% @ESM1 $4,136.50 324 171 $17,094.80 $(12,958.30) 64% 65% @ESU1 $1,263.30 245 148 $11,161.50 $(9,897.90) 62% @ESZ1 $(1,111.60) 228 139 $11,018.10 $(12,129.70) P/L 62% @ESH0 $(3,684.80) 285 191 $15,682.00 $(19,366.80) $36,509.80 60% @ESM0 $6,761.00 371 184 $29,281.70 $(22,520.70) 67% @ESU0 $(160.60) 228 144 $10,005.60 $(10,166.20) AvgWin 61% @ESZ0 $3,277.70 315 161 $18,725.50 $(15,447.80) $51.44 66% @ESH9 $6,828.70 324 157 $19,582.30 $(12,753.60) 67% @ESM9 $4,873.60 299 144 $17,027.30 $(12,153.70) Avg Loss 67% @ESU9 $2,479.00 261 134 $11,484.70 $(9,005.70) $(78.73) 66% @ESZ9 $(431.60) 244 148 $12,403.80 $(12,835.40) 62% W/L Ratio 0.65

The percentages listed to the far right are the Win% per quarter (a few get blown out to the right). The totals column is the second to last column listed, and it sums up the whole mess.

Of course I'm not still trying to claim this system is tradeable in real life, but on paper, how can you say it doesn't have a positive mathematical expectation?

harrytrader · Apr 17, 2004

If you can enter in short term countertrend then you can consider to avoid slippage and even that slippage works for you.

Quote from harrytrader:

On paper for me doesn't exclude that you must take into account some noises that are well known in advance slippage, fees,... If it was a swing system (W/L >> 3) you don't need to estimate them but when it is scalp (W/L < 1) realisation really makes all the difference between winning and losing. There are inherent realisation conditions that could be estimated in advance so that could be done on paper and there are realisation conditions that cannot be estimated very precisely in advance like your own ability to execute perfectly your plan but at least you should take into account the first ones. Mostly you risk to work for your broker ... with your own money and taking all the risk.

Albert · Apr 17, 2004

I put in $3200 into a Global Futures account on March 24 and am now up to $12,136 as of yesterday. I did it with an average profit per trade of around $30.
I have low commissions and very low margin requirements which allows me to leverage the hell out of the account. Average time in trade, though, is only around 8 minutes, so even if it's not super fast trading it still is short term.
All trades are taken off the one minute chart and in synch with the 30 minute chart.
Don't get me wrong, I am still a loser overall. If you look at all the trading I've done, this patch here just gets me to breakeven from last year. But I tried trading on longer time frames and was too weak to follow my plans. Now I just sort of go with the flow, get in on stops and get out on limits, just the opposite of what the "best" traders recommend. Oh yeah, since I am weak, I have had NO LOSERS over the past two weeks. In reality that means I have averaged up, down and sideways to get back to even or a small profit.
You can scalp this thing. I don't think you need backtesting back to 1940 to prove anything.
Best of luck,
Albert

harrytrader · Apr 17, 2004

Also 36,509.80 / 7813 = 4.67$ is a more simple and straight way I used to answer what I've answered at first

.

Quote from harrytrader:

On paper for me doesn't exclude that you must take into account some noises that are well known in advance slippage, fees,... If it was a swing system (W/L >> 3) you don't need to estimate them but when it is scalp (W/L < 1) realisation really makes all the difference between winning and losing. There are inherent realisation conditions that could be estimated in advance so that could be done on paper and there are realisation conditions that cannot be estimated very precisely in advance like your own ability to execute perfectly your plan but at least you should take into account the first ones. Mostly you risk to work for your broker ... with your own money and taking all the risk.