How to compare a system against random entries

S

Spec_pt

I have seen this point mentioned in several post about developing mechanical systems, but I never saw it explained. So, when looking at a mechanical system results, how can we compare that with pure random entries? How to do that ?

If you remember this being explained here in ET, it would be appreciated if you could report the link because I couldn't find anything, both here or Google.

Many thanks
 
You could do exactly that. Run your system with random entry rules instead of whatever rules you got. Run that system a few thousand times (obviously, the results will be different because your entries are random). The distribution of those results is the exact answer you are looking for.

Quote from Spec_pt:

I have seen this point mentioned in several post about developing mechanical systems, but I never saw it explained. So, when looking at a mechanical system results, how can we compare that with pure random entries? How to do that ?
More...
 
Van Tharp's book "Trade Your Way to Financial Freedom" discusses this on pages 253 to 255. Chuck LeBeau's book "Computer Analysis of the Futures Markets" discusses it on pages pages 170 to 174.

And here are a couple of chat-board discussion threads that touch upon the same theme: (Link-1) , (Link-2)
 
Search on here for posts by a guy named acrary with the keyword edge test. He lays it out clearly with examples...basically a bastardized version of White's reality check.
 
I think it is fairly easy and can be done with any backtesting platform. You initiate random entries using a random number generation function and then apply your exits. Do that for a number of times, like 100, find the mean return of those runs and compare it to the mean return of your system. If the mean return of the system is at least 3 sigma away from the mean return of the random runs then you may have an edge.

I invite other members to comment and maybe propose improvements and/or modifications to this approach.
 
Quote from goodgoing:

I think it is fairly easy and can be done with any backtesting platform. You initiate random entries using a random number generation function and then apply your exits. Do that for a number of times, like 100, find the mean return of those runs and compare it to the mean return of your system. If the mean return of the system is at least 3 sigma away from the mean return of the random runs then you may have an edge.

I invite other members to comment and maybe propose improvements and/or modifications to this approach.
More...

I made a tool to do something similar. Usually I do 10,000 runs, then it gives a nice rounded distribution curve. One day I hope to get the curve values out and check what kind of distribution it is, I'm not sure they come out as normal curves or not. Usually, it looks wider and flatter than a normal curve.
 
Quote from zedDoubleNaught:

I made a tool to do something similar. Usually I do 10,000 runs, then it gives a nice rounded distribution curve. One day I hope to get the curve values out and check what kind of distribution it is, I'm not sure they come out as normal curves or not. Usually, it looks wider and flatter than a normal curve.
More...

Does random positions perform better than calculated positions?
 
You must log in or register to reply here.
Back
Top