How to determine if strategy entry criteria beats random entry

fan27

fan27

Let's say for the sake of an example, I have a strategy which yields 500 trades over a thousand stocks. The entry criteria is some price related pattern and stop and limit orders are placed 2 x the Average True Range away from the entry price. Assuming the strategy result metrics look good, how do I know the entry criteria is better than random entry?

One idea I had was I could randomly pick entries in the stocks that had signals and use the same exit criteria. For example, if stock ABC had 5 signals, I would randomly pick 5 entries in ABC and continue for the rest of the stocks. I could run this simulation 100 times and see how many of those runs my original strategy outperformed. If it beats more than 90 or 95 of those random entry runs, then I likely have an edge.

Are there other approaches that are more robust?

Thanks
fan27
 
Your question boils down to, "Is a tested, criteria-based trading system better than random entry?"

The answer depends on what's going on.

To give but one example, in a bullish market, a bullish system and Random Entry might give similar results. But let that market turn, and the signals from the bullish system would turn off, while Random Entry would continue on -- and results would (joyously) diverge.

So,

ITDepends.jpg
 
fan27 said:
Let's say for the sake of an example, I have a strategy which yields 500 trades over a thousand stocks. The entry criteria is some price related pattern and stop and limit orders are placed 2 x the Average True Range away from the entry price. Assuming the strategy result metrics look good, how do I know the entry criteria is better than random entry?

One idea I had was I could randomly pick entries in the stocks that had signals and use the same exit criteria. For example, if stock ABC had 5 signals, I would randomly pick 5 entries in ABC and continue for the rest of the stocks. I could run this simulation 100 times and see how many of those runs my original strategy outperformed. If it beats more than 90 or 95 of those random entry runs, then I likely have an edge.

Are there other approaches that are more robust?

Thanks
fan27
More...
Trade it with real money and see. That's how you find out.
 
There is a published paper that may provide guidance.

Biondo et. al. published a paper entitled “Are Random Trading Strategies More Successful than Technical Ones?” - 11 July, 2013

http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0068344


The authors are affiliated with the University of Catania, Italy and ETH University in Zurich, Switzerland. So this was not a marketing exercise by a broker in Cypress.

They looked at not only absolute results but volatility in a portfolio.

Also worth mentioning that Van Tharp has touched on this topic and talked about Tom Basso doing testing on random trading.

I found many of these approaches to be a bit thin regarding their rigour. But testing random, or better yet, a pure mathematical approach to random, requires a fair bit of work or experience in maths.

Short answer, with excellent money management a random entry returns a positive result, and in many cases superior to the common indicators used by retail traders. Note that the mathematics is quite a bit more complicated than that simple one liner, and should be reviewed in detail.

I’ve toyed with the concepts quite a bit over the years as a learning tool, but never for substantive trading. I found it a good exercise that assists in seeing the necessity of a complete trading strategy, or what I call a package.

Ultimately I trade my substantive account using non-random entries, since they obtain superior result. But an exploration of random entries assisted in developing my concepts of risk to reward, size of trade, exit criteria, selection of financial instrument, risk of ruin, risk of draw down. All of which have nothing to do with, “do I buy or sell”.

Your parameters of selecting from 1,000 stocks would make a thorough analysis a daunting task. For simplicity sake, if I am considering a new strategy involving equities, my first question is, on a sample of 300 trades, does it beat the S&P500? If it does, I then dig into the more detailed criteria of expectancy, risk of ruin, volatility, Sharp and Sortino ratios etc. I’ve never used a random approach as a baseline in equities, but I have in spot forex (but that is an entirely different ocean to trade, with many more sharks).

You also mentioned running your simulation 100 times. I believe that is too small a sample size, but I am not sure how many points of data each simulation generates, so that may not be a fair comment on my part.

I think that your idea of seeing if your strategy beats random 90 or 95 times out of 100 is not complete. Since I am obsessive compulsive about risk management I would want to see a more robust review. For example, if your strategy beats random 95 out of 100 times, but your risk of ruin is 98% over 10,000 trades, it is not something I would trade live. How do you define “beat” is a question to consider. Exceeding random by 65% while having an excellent expectancy and a low risk of ruin could be superior.

Exited my NFP Friday trades so had time to ramble a bit, hope it helped and did not confuse. I would definitely recommend you play with these concepts, not for the final result, but to see what else you learn during the process.

So often, it is as we try to climb one mountain, and pause to lift our heads, we see so many other opportunities in the now broader vista that's around us.
 
Couldn't you run your backtest, run your "random" system, capture the distribution of returns and then run some sort of t-test on them to see if the distributions are statistically different? You would probably need to think through how many parameters your system has and what p-value you need to use. I wouldn't just default to the 5% you see a lot of the time especially if your system has a lot of parameters.
 
jjapp said:
Couldn't you run your backtest, run your "random" system, capture the distribution of returns and then run some sort of t-test on them to see if the distributions are statistically different? You would probably need to think through how many parameters your system has and what p-value you need to use. I wouldn't just default to the 5% you see a lot of the time especially if your system has a lot of parameters.
More...



You have to be cautious with such an approach. A common misunderstanding is the probability of price distribution. It is not a normal distribution.

see: Bollinger on Bollinger Bands, by John Bollinger,

There is a Chapter (I can't recall which off the top of my head) where he reviews extensive data that demonstrates that price is not a normal distribution. He cautions traders when using Bollinger bands not to use the percentile functions normally associated with standard deviations. If that were the case trading would be simple, trade reversion to mean when price reaches a 3rd standard deviation and you would win 99.8% of the time. That does not happen, that is not the way price action works, and if you simply trade on that you will lose over the long term.

You often here people say it is 50:50 for the price to go up or down $1, and it is the cost of commissions that cause the loss. That is not accurate. There is a bias, but that bias fluctuates over time. This is why a properly designed trading strategy will exceed a random result.

The best work on the that topic was done by James Simons at Renaissance for Medallion. They were able to use the mathematics of topology to create a multi-dimensional model of price fluctuation over time with relative probabilities of movement. A series of integrated functions within specific limits creates a predictive model, but a large amount of computer power is needed for the computation. Simons did ground breaking research in pure maths prior to becoming involved in finance.

An OG of quant funds (from 1994 to 2014 70%+ annual returns). The details of their methods are a closely held secret.

 
  • Like
Reactions: 777
I am very aware that price distributions aren't log normal. The question is is it close enough for a t-test to give you a reasonable answer over a large enough data set. I don't think a fat tailed distribution will significantly impact your answer if you're just trying to determine whether you can safely assume two distributions over a short timeframe are different. That being said, if you're concerned about it run Mann-Whitney U-test or something similar that doesn't assume a normal dist.

The risk to assuming a normal distribution in this case is pretty small compared to assuming a normal distribution for some relative value, mean reversion strategy.
 
jjapp said:
I am very aware that price distributions aren't log normal. ......
More...

You appear to have been offended by my post. Please accept my immediate, and genuine apology without any reservation.

I was mentioning things of a general nature for the benefit of anyone who happened to read the thread. I did not mean to infer that your knowledge was deficient, or to demean you, in any way.

I have no idea of your level of knowledge, on any subject at all, and know nothing about you at all. It would be irrational of me to say something for, or against you, when I have no information.

So again, please do not feel offended by my post, and I apologize for anything that you found offensive. In the future I shall try to be clearer in my word choices, especially where I mention something of general information and not directed towards an individual.

Writing longer posts, without my taking adequate time to review, and consider how different people might interpret them emotionally, is also an error. Another item on my list of things I should work towards remedying.

I hope you have an enjoyable weekend and a successful upcoming week of trading.
 
You must log in or register to reply here.
Back
Top