Quote from talontrading:
Wow.... I didn't realize he had posted his complete trade set already. FYI I am going to cross post this in my thread because I hope to give some valuable information on how we look at distribution of system returns... and I know the little turd deletes my posts from his thread so I can't count on it staying here. C'est la vie, ne c'est pas?
Anyway... first things first. There aren't enough trades here to analyze properly. If we have a system that trades this infrequently, we must be very convinced of two things to put money behind it. 1) it's fundamental validity and 2) the development process. Let's accept those at face value here even though this particular approach is profoundly flawed and I have reason to believe the development process involved a fair degree of optimization. We will proceed as if this is were not true though.
So... One at a time... BoWo's quotes in bold and my answers below. Sorry to do this to him, but I honestly think there is a valuable lesson for everyone here... a reminder of basic statistics and approach... and hopefully for him a lesson on hubris:
Quote from bwolinsky:
I have done some research on the profit percent distribution of the trades and find the average profit accross all trades is 2.575%, with a standard deviation of 5.7665%.
This implies that 95% of the trades will be 2.575+/-5.7665*1.96=13.87734% or -5.53%. I believe it's good that 95% of the trades approximately already fall into this distribution, and especially the low value is a little beyond my stop of 5 and an 8th percent.
Well... um... no. Your simple rule of thumb is correct if and only if the data are distributed normally. In market data and trade returns, pretty much nothing is normally distributed. This is a rookie mistake. However, let's take a look at the data. See attached file. The bars are the returns of your actual trades, the red line is a "what if" the returns were normally distributed, and the dark green line is the empirical distribution of your sample. As you can see from the shape of the graph, these returns arent even close to being normally distributed. (This in itself is fine... exactly what we would expect from real market generated information.) However, the "eyeball" test isn't a good one.. there are a number of other tests we can run:
Without boring you with the details, running Shapiro Wilk test on this data set gives a probability of <.00000 for the data being normally distributed. A formality, but one we should be in the habit of conducting.
Going back to the old eyeball test though, I see a problem. The "left tail" of your distribution is severely truncated. This is a problem... This is a characteristic I have seen time and time again in system development that is either done on an incomplete data set or without enough trades. In simple English, a distribution like this assumes you'll always be able to get out at your stop... and you won't. So in actual practice you can expect worse losses than your system results predict. How big of a problem this is depends on a number of factors... I can't give a good rule of thumb without knowing the system intimately.
So... be careful of your standard deviation assumption... it is not correct... your returns are likely to be much wilder than your system development leads you to believe. This is a problem.
I also examined the kurtosis of the distribution and found that it is a less peaked distribution as evidenced by it's value being less than 1 at exactly .749182809. Many people would then conclude that that would imply there are fat tails possible in my system, but, you would have to look at the "skewness" to determine where those tails are, and, based on the skew of 0.927014752, I can conclude that negative values are quite limited compared to the huge profits on the right, positive side of the distribution. This is a good thing, and one day I hope somebody will realize just how good of a distribution that is.
I also, at some point, hope others would share their distributions with me, as I have, to compare. I believe there are tons of systems who may be exhibit lagging kurtosis with negatively skewed distributions that imply the system has "hidden risks" inherent in the system. A kurtosis below one, as I have said, means the distribution is "less peaked", and the step most forget then is to examine the skewness to determine "where the fat tails are at." In this case, if you had found a system with kurtosis below 1, and resoundingly positive skewness, you may conclude that the so-called "fat tails" are actually benefiting you in that they are "positively skewed, fat tails."
No. No. No. No. and NO! I am sorry, I cannot be polite here... this guy claims to be the "best system developer" and then makes such a fundamental error... it has to be pointed out.
First of all, let's deal with your math. You can't use Excel for analysis because the statistics in Excel are wrong. Here are the actual results from Excel's Data Analysis module:
Mean 2.575227273
Median 1.71
Standard Deviation 5.766540423
Sample Variance 33.25298845
Skewness 0.927014752
Kurtosis 0.749182809
And here are the correct results from another piece of software:
Mean 2.575227
Median 1.71
Std. Dev. 5.76654
Variance 33.25299
Skewness 0.9111379
Kurtosis 3.639872
Note that Excel gives incorrect values for Kurtosis and Skewness. In this case, not fatal, but there is an important lesson here. DO NOT USE EXCEL FOR DATA ANALYSIS.
Now, on to your analysis of kurtosis. "Positive excess" kurtosis (Excel gives .75 vs Stata's .64... both are positive at least) may be generalized to mean that the tails are heavier, shoulders lighter, and more values cluster around the mean. So you are exactly wrong when you say "I also examined the kurtosis of the distribution and found that it is a less peaked distribution as evidenced by it's value being less than 1" Leptokurtic (excess kurtosis > 0 (not 1)) distributions are MORE PEAKED and have FATTER TAILS. What you mean to say is that this distribution has fat tails.
Next you say "you would have to look at the "skewness" to determine where those tails are, and, based on the skew of 0.927014752, I can conclude that negative values are quite limited compared to the huge profits on the right, positive side of the distribution." This is not what the skew tells you... and you're still trying to incorrectly apply rules that apply only to the normal distribution here. I won't go on with the math lesson, but this is simply incorrect... there are number of books on descriptive statistics that can help.
I encourage anyone to examine this distribution, and provide their current system for analysis.
Done BoWo. Attached find a txt file of actual percentage returns from an intraday system one of our trainee traders developed. I'll leave you to do the analysis (and, ahem, encourage you to not use Excel.) Point being, these are actual returns not theoretical backtest... and 1 year ago this person couldn't even tell you what a bid/ask spread was. This is a testament to the power of doing the right thing, learning the right things, and focus focus focus.
The point I'm making about "fat tails" is the proverbial "Black Swan" argument that really denies basic statistics. Certainly there are always outliers, but they don't happen very often. Once in a 1000 years even for some calculations of financial events, so the probability you hold it on that day is not even something to consider in your approach, and the "Black Swan" theory really has no basis in my opinion, because all it says is that <b>if there's always the possibility of a large move, you must not ever take risk</b>, which is a false assumption. Given that the probability of such events is so unlikely then if it does happen to you, you shouldn't change anything with what you were previously doing, as it can be considered economically and essentially a "sunk cost" so that it does not enter your strategic investment decisions.
You're obviously referring to Taleb's book and the joke my friend is on you. The whole point of Taleb's writing is that people who have an elementary understanding of statistics don't understand Black Swans. You have proven that here. I sure wouldn't want you managing my money since your answer seems to basically be dont worry about the big tail risks... I mean... yeah sure... these black swans only happen about once every 10 years and they almost bring down the whole financial system. Why try to understand these risks when they are so trivial, right?
Obviously that was an unkind joke, but the real point is that we never understand our risks. They are always FAR worse than we expect them to be... In your case you might consider the risks involved in these leveraged products in terms of counterparty risks, etc... So called Black Swans, and respect for the risks and possibilities inherent in these events, are one of the central problems in trading. I'm sorry you have chosen to ignore the possible lessons here so completely.
I hope I wasn't too big of a d*ck to BoWo here... I was partially responding to him but also hoping to correct some misinformation.
