Usually a system's results will be in the top 5% of the random entries, but never past it. I'm not sure how to interpret it -- does that mean it's not a viable system, because it's less than 3 sigmas out? Or would a system exceeding the highest random one be a sign of curvefitting?
How to compare a system against random entries
- Thread starter Spec_pt
- Start date
Usually a system's results will be in the top 5% of the random entries, but never past it. I'm not sure how to interpret it -- does that mean it's not a viable system, because it's less than 3 sigmas out? Or would a system exceeding the highest random one be a sign of curvefitting?
Just a thought. If you take completely random entries and exits and dont have barriers on either side (say your time frame allows this assumption to be true; very valid if you trade intraday and say trade something like ES or SPY). Then you can do a lot of testing or assume that with zero edge your random entry, random exit system should over the long run return you with ZERO dollars. Expectancy should be zero (your deviations, etc will all change based on your entry and exit rules; but without an edge, your long term profitability should be zero. If you dont get that; run it across multiple streams of random inputs and do some monte carlo stuff - eventually my hypothesis is your expectancy will be zero.
Now if you agree with that; you can save yourself all that trouble and just analyse your system for expectancy better than zero and be done with it. Ofcourse, just theorizing. Someone can try to prove or disprove it - will take some effort but doable. i think its a cool topic for an undergrad student doing data analysis.
PS: That said, one way to compare the goodness of system exits or entries that i have used in the past is first test one at a time; say you are testing your exits: compare your exits vs exits taken at a default say 5,10,20,30,40,50 bars (whatever your timeframe is) and see if your system generated exit is any better than a dart throw. Entries is a bit tricky but you can extend the same idea.
-gariki
Are you talking about the results of a specific system or in general? If you have a specific system and you can provide monthly returns for at least 3 years I can run a bootstrap and tell you if there is evidence against randomness.