You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
Activate/Deactivate System?
- Thread starter EricP
- Start date
Quote from EricP:
Is my system profitable? At what level of confidence?
Assuming we are using 'future' data (whether this is backtested future data or actual forward tested data doesn't matter), then we can use statistical analysis to our data to determine the likelihood that our system is a profitable system over the long run as follows:
x = abs(Avg-Profit) * (Number of Trades)^0.5 / (Std Dev of Profits)
Once "x" is calculated, you can look up the confidence level on the chart below to see the likelihood that this system will be profitable over the long haul:
x ------------------ Confidence Level
0 ------------------ 50%
0.075 ------------ 53%
0.126 ------------ 55%
0.25 -------------- 60%
0.36 -------------- 64%
0.52 -------------- 70%
0.67 -------------- 75%
0.84 -------------- 80%
1.04 -------------- 85%
1.28 -------------- 90%
1.4 ---------------- 92%
1.645 ------------- 95%
2.05 --------------- 98%
3.0 ----------------- 99%
4.0 ----------------- 99.5%
5.0 ----------------- 99.7%
Note that if the average profit is negative, then this will give you the confidence level that this is a losing system, over the long run.
One quick example to illustrate this equation. Assume that we have 50 trades, yielding an average profit of $40 per trade, with a standard deviation of $150 per trade. Using the equation and table:
x = abs(Avg-Profit)*(Number of Trades)^0.5 / (Std Dev of Profits)
x = abs(40)*(50)^0.5/150
x = 1.88, from table, Confidence Level ~97%
-Eric
Just a quick question. For example, if you were trading stocks and pyramid your profits. Would this test be valid or would you have to test it on the same # of shares or same dollar amounts on each trade?
Quote from shorty_mcshort:
Just a quick question. For example, if you were trading stocks and pyramid your profits. Would this test be valid or would you have to test it on the same # of shares or same dollar amounts on each trade?
I consider each trade to be an independant event, regardless of whether it is 200 shares or 1500 shares. I think it would still be worthwhile and useful for you, although I would avoid the word 'valid,' as this might imply a sense of statistical accuracy that we do not have or require for our need (in light of distribution not being bell curve). Being less than perfect can be very profitable, while taking the effort to have a statistically 100% supported analysis might be academically interesting (not!) but is a major waste of time for a trader, IMO. We are seeking to get 90% of the results with 10% of the effort, which I think this method achieves nicely for me.
-Eric
Quote from EricP:
I consider each trade to be an independant event, regardless of whether it is 200 shares or 1500 shares. I think it would still be worthwhile and useful for you, although I would avoid the word 'valid,' as this might imply a sense of statistical accuracy that we do not have or require for our need (in light of distribution not being bell curve). Being less than perfect can be very profitable, while taking the effort to have a statistically 100% supported analysis might be academically interesting (not!) but is a major waste of time for a trader, IMO. We are seeking to get 90% of the results with 10% of the effort, which I think this method achieves nicely for me.
-Eric
Very interesting. Thanks for sharing. If someone is interested in learning more statistics that you could apply to trading where could I find that information?
Quote from Hofferino:
How does one come up with a value for standard deviation per trade (such as your $150 in the example)?
Microsoft Excel has a function for the Standard Deviation.
STDEV(a1:a51), for example to calculate the standard deviation for the numbers in cells a1 to a51.
Many calculators also have a function for this, and I'm sure you can search the internet for the actual equation for calculating the standard deviation if needed for some custom coding.
-Eric
Quote from shorty_mcshort:
Very interesting. Thanks for sharing. If someone is interested in learning more statistics that you could apply to trading where could I find that information?
Hmmmm. Good question. Hopefully, someone will have a link or book recommendation for us. Acrary's recent thread has some very good information, in case you missed it.
-Eric
Quote from Hofferino:
How does one come up with a value for standard deviation per trade (such as your $150 in the example)?
Hoff,
If you have Excel then you can go to Insert/Function type in Standard Deviation in the search box and click on STDEVP.
Also note that the original post talks about standard deviation of profits not per trade.
Quote from shorty_mcshort:
Also note that the original post talks about standard deviation of profits not per trade.
I think you both have the right idea. I am talking about the standard deviation of the profits from the trades. Using the first example posted:
+$2000
+$350
($300)
+$1400
($3000)
+$750
And plugging into Excel for STDEV, I get a standard deviation of $2652. Also, you could use the STDEVP function, which estimates the Standard Deviation over an entire population of data, and I get $1875 as the Standard Deviation. Either way should be fine. Recognize again that we are not seeking 100% perfection in accuracy, but a rigid and structural way to activate and deactivate trading systems.
-Eric
P.S. For what it's worth, the values for STDEV and STDEVP will much more closely approximate one another when you are using a larger number of datapoints in your analyis.
EricP,
Thanks for your contributions - this is very nicely done.
Question :
you mentioned testing for profit confidence as follows
[ p * n ^0.5 ] / sigma
where
p = net profit per trade
n = # of trades
sigma = std dev per trade
we can factor out n so, it's basically
[ n^0.5] * [p / sigma]
It is clear that assuming p > 0, as n increases, your score is going to increase, and that you can theoretically seek for any score you want by increasing n.
I guess to bring this a step further, would you say that for any given n, the corresponding score will tell you the confidence of being profitable for a period of length n?
Thus : if n = 100 and the score corresponds to 98%, then we can say that the system has a 98% confidence in being profitable every 100 trades. If the same system has a score of 80% at n = 50, then we can say that at 50 trades, the confidence of being profitable drops to 80%.
Thus, the question becomes - say if we are tracking the last 100 trades of a system and working out the corresponding score based on n = 100, and then dropping the system if the score falls below 80% or whatever benchmark we set.
How do we know 100 is the appropriate length to use? How do we know 80% is the appropriate cut-off to use?
Should we just be arbitrary based on our "judgement" or is there a rigourous way of approaching this problem?
Thanks for your contributions - this is very nicely done.
Question :
you mentioned testing for profit confidence as follows
[ p * n ^0.5 ] / sigma
where
p = net profit per trade
n = # of trades
sigma = std dev per trade
we can factor out n so, it's basically
[ n^0.5] * [p / sigma]
It is clear that assuming p > 0, as n increases, your score is going to increase, and that you can theoretically seek for any score you want by increasing n.
I guess to bring this a step further, would you say that for any given n, the corresponding score will tell you the confidence of being profitable for a period of length n?
Thus : if n = 100 and the score corresponds to 98%, then we can say that the system has a 98% confidence in being profitable every 100 trades. If the same system has a score of 80% at n = 50, then we can say that at 50 trades, the confidence of being profitable drops to 80%.
Thus, the question becomes - say if we are tracking the last 100 trades of a system and working out the corresponding score based on n = 100, and then dropping the system if the score falls below 80% or whatever benchmark we set.
How do we know 100 is the appropriate length to use? How do we know 80% is the appropriate cut-off to use?
Should we just be arbitrary based on our "judgement" or is there a rigourous way of approaching this problem?