Directional edge testing

[abattia]

Another thing to bear in mind is that Last Traded Price (LTP) backtesting (despite steps that can be taken to improve the general realism of fill/slippage assumptions) suffers from the drawback that it takes no account of whether the LTP was at the bid or the ask, and therefore the “spread” (or even a multiple of 2 x the spread) gets introduced as an unknown error in the results.

I have tried to think how that might be related to your case here, but haven’t succeeded in figuring it out!

Perhaps others will have a clearer idea?

 

[dom993]

Quote from HurricaneUS:

what do you mean..."why is it flatlining?"

the flat line is the law of large numbers kicking in...google it

Using random entries, the flat line would be flat at 50% ... and as the stop/target size increase, I do expect any directional signal to end-up reverting to "random", ie. 50%.

It took me a while yesterday to start doing proper measurement, because when I removed the exit signals, I was left with Long & Short trades opening simultaneously (a Long opening while a Short is still open, etc.), and at that point stupid Ninja pairs the first Buy after a SellShort in the trade log and that messes all up. I had to run Longs & Shorts on separate instruments (prior expiry for one of them) and calculate all entry/exit prices in the correct TF before I could get to usesul stats.

My understanding of those stats I initially shared, as the target/stop distance increase, there is more and more trades taken out on the exit signals, less and less on the target/stop, and that flat line at 56.5% is the system win% w/o any target/stop.

After clearing the mess with Ninja handling of concurrent Long / Short trades, I could analyze things better - using the trade log & detailed exit names.

My analysis: the signals I use for entry & exit have a lot of noise. As can be seen in the Stop/Target columns, increasing the size of the stop/target does improve the win% very significantly - but on a much decreasing fraction of the total trades. The other trades are taken out by the exit signals, and apparently those exit signals have a better win% when they happen "close" to the entry signal, before the trade reach either stop or target.

 

[dom993]

Quote from abattia:

I'm guessing you're
1. testing with NT
2. with a 1-tick/1-second secondary time period
3. with BetterThanDefault fill type
4. with some slippage assumptions

1) 2) 3) 4) : correct. I am using 1-sec. TF for fills in backtest (there is no point using the 1-tick TF, it is synchronised to the second with the other TFs).

I'm guessing you’re doing something like the following calculation... take, as an example, case of a long trade, with assumptions that target order is a limit order, stop order is a stop market order.

Say targetPrice = entryPrice + targetPrice
So, stopPrice = entryPrice - (targetDistance + 1-tick) = entryPrice - targetPrice - 1-tick

Target order will fill at target price (i.e. entry + targetDistance) once price moves at least targetDistance + 1-tick above entryPrice (but only netting targetPrice per unit traded).

Stop order will fill at stop price - slippage once price has moved at least targetDistance + 1-tick (losing net targetDistance + 1-tick + slippage per unit traded).

So the basis of what you are saying is that in either case (target or stop) the order triggers once price has moved at least targetDistance + 1-tick.

Is that correct up to there? If not, then ignore the rest! LOL!

Correct

= = = = = = = = =
... otherwise ... read on ...

The fact that target and stop moves are of equal size would suggest win rate should be approx 50% (in deference to HurricaneUS, I won’t provide another link here to that paper I brought to your attention in a previous post …).

I would suspect therefore that the result you are seeing is coming from the trade management activities rather than the initial target/stop.

Perhaps these trade management activities may have the net effect of shortening your average winner/lengthening your average loser?

What does NT report about average winner and average loser? If the main effect is the initial target/stop, then avg winner should be abs (entryPrice-targetPrice), and avg loser should be abs(entryPrice - (stopPrice + slippage effect)), and you could calculate the expected ratio.

If on the otherhand the trade management effects are the cause of the >50% win rate, then this ratio (avg winner / avg loser) would be lower than the above calc would suggest.

I think you are correct, it is the exit signals that create that flat line.

Thanks a lot for your analysis. BTW, this is not a mean reversion system, rather a "follow-the-trend" system. Or should I say a follow the "trend-signals" system, as currently the system doesn't care if the signal was wrong - as evidenced by price moving against that trade.

I have an optional "mean-reversion" component (not discussed yet), whereby when the actual trend is evidently against the last signal, the system will add-on at some point ... I really don't like it, but it improves every performance measurement - P/F, DD, avg/trade.

 

[dom993]

I am presenting here the Summary report for this prototype, taking all entry signals (3 types), and using add-ons on 1 of those types. (max number of contracts 58, avg. number of contracts per exit point: 6).

This gives me about 55.4% win-rate (for entries), with an avgW/avgL = 0.99 (1-t slippage per entry / same for exit / commissions included), on a total number of entries of 31,338 (5169 exits).

Is there enough here to "prove" there is an edge in there, albeit not big?

I certainly like the symmetry of the results for Longs & Shorts (aside from the DD, which is way bigger on the Long side).


Edit: just computed the win% as a function of exit points, it is 56.58%

 

[dom993]

P&L graph:

 

[dom993]

Per Year:

 

[dom993]

P&L per exit:

 

[dom993]

This one is a (partial) distribution of exit P&L ... the label under each bar is the average per trade in that bar (each bar is $2k wide).

I restricted that view to about 60% of the P&L, the tails are pretty long on each side, and (of course) longer & heavier on the positive side.

 

[abattia]

Quote from dom993:
...This gives me about 55.4% win-rate (for entries) ...
... just computed the win% as a function of exit points, it is 56.58% ...

Sorry, not clear to me what the difference is between how you take these two measurements...

 

[Sergio77]

Quote from dom993:

This gives me about 55.4% win-rate (for entries), with an avgW/avgL = 0.99 (1-t slippage per entry / same for exit / commissions included), on a total number of entries of 31,338 (5169 exits).

Edit: just computed the win% as a function of exit points, it is 56.58%

Too low of a win rate. Add slippage and other factors and all the gain is gone.