If you do not think in statistical terms, I am afraid you would be just wasting your time and money. The least you should do is to think like a casino, but in the market you can be smarter than a casino because the people in the market are dumber than the people who go to a casino.
Profitability Question and Example
- Thread starter zerfetzen
- Start date
If you do not think in statistical terms, I am afraid you would be just wasting your time and money. The least you should do is to think like a casino, but in the market you can be smarter than a casino because the people in the market are dumber than the people who go to a casino.
It's all good, I'm just here to learn about trading 
You're right, we were talking about different things. Your notation suggests to me that you are differencing price, in which case I can easily see it correcting for autocorrelation in a series, good point.
When you say the variance of X(t) grows linearly with the square root of time, is that the variance of the error term in the prediction, such that it is much harder to predict X(390) than X(1)?
I still think I can demonstrate that, even in that condition, iid should not hold, but we can assume it does, probably without radically changing anything. But funny you mention it, my assumption that iid does not hold has allowed me to develop a method that is good so far, and if it works in large numbers, then iid is not possible in that case.
But I have no vested interest in whether or not iid holds, so I'm more than interested in pursing a method that assumes iid as well.
It sounds like both you and I need to increase the size of our predicted sample. For my current method, I have racked up 100 predictions over 4 days, logged so I can study them. Unfortunately, once I apply a trading criteria to the probability, such as I would only trade when prob>=0.66, I only have 25 out of 100 predictions. But 68% of those are correct. So far you have predicted 3 out of 4 trials on the button, is that correct? If so, we both need many more trials.
By the way, quick side question, you suggested before that indexes are a great way to start. I'm up for anything, but what makes you suggest that? I don't learn if I don't ask questions, so I ask a lot. Thanks.
You're right, we were talking about different things. Your notation suggests to me that you are differencing price, in which case I can easily see it correcting for autocorrelation in a series, good point.
When you say the variance of X(t) grows linearly with the square root of time, is that the variance of the error term in the prediction, such that it is much harder to predict X(390) than X(1)?
I still think I can demonstrate that, even in that condition, iid should not hold, but we can assume it does, probably without radically changing anything. But funny you mention it, my assumption that iid does not hold has allowed me to develop a method that is good so far, and if it works in large numbers, then iid is not possible in that case.
But I have no vested interest in whether or not iid holds, so I'm more than interested in pursing a method that assumes iid as well.
It sounds like both you and I need to increase the size of our predicted sample. For my current method, I have racked up 100 predictions over 4 days, logged so I can study them. Unfortunately, once I apply a trading criteria to the probability, such as I would only trade when prob>=0.66, I only have 25 out of 100 predictions. But 68% of those are correct. So far you have predicted 3 out of 4 trials on the button, is that correct? If so, we both need many more trials.
By the way, quick side question, you suggested before that indexes are a great way to start. I'm up for anything, but what makes you suggest that? I don't learn if I don't ask questions, so I ask a lot. Thanks.
You think very well, and I like it. Let answer some of your questions, and I also want to sort of split the framework in your mind in multiple compartment so to speak.
Why and index and not a stock? Think of it this way. Assume you work for an insurance company or a bank. How do you reduce risk of getting a bad client. You admit that you can make the mistake and/or you have no proof you cannot make the mistake. So what do you do, you essentially spread your eggs, which means you decide that a set of clients is better than few client.
The law of large numbers takes care of the unknown, but conceivably possible, risk.
When you take a stock, you do not know everything about it (even if you think you do). If you take a 100 stocks, you also do not know, but you do know that if there is trouble somewhere it would not happen at the same time. Therefore trading an ETF is a free hedge against being wrong on a single stock.
In addition, stastical analysis would apply to such stocks because while they move in a given day, they are heavy to move down too much or up too much.
In my next post, I will explain the problem of division of framework and questions to be posed/answered.
Why and index and not a stock? Think of it this way. Assume you work for an insurance company or a bank. How do you reduce risk of getting a bad client. You admit that you can make the mistake and/or you have no proof you cannot make the mistake. So what do you do, you essentially spread your eggs, which means you decide that a set of clients is better than few client.
The law of large numbers takes care of the unknown, but conceivably possible, risk.
When you take a stock, you do not know everything about it (even if you think you do). If you take a 100 stocks, you also do not know, but you do know that if there is trouble somewhere it would not happen at the same time. Therefore trading an ETF is a free hedge against being wrong on a single stock.
In addition, stastical analysis would apply to such stocks because while they move in a given day, they are heavy to move down too much or up too much.
In my next post, I will explain the problem of division of framework and questions to be posed/answered.
The other point was the question of IID.
First there is the question of profitable system. You can do it in two ways:
1. The QQQQ are showing auto-correlation, and you exploit it. My guess ia that a lot of people do that. So there is competition.
2. You have a rule to select a subset of data from the overall data, which shows some advantage.
You should note now that method 2 may work even if the original data is IID, but method 1 would work only if the IID is violated.
Which one do you think you should prefer? I would take 2. anytime, for many reasons. One was the competition, and two I am not concerned about an assumption (IID) on underlying data that might be wrong. Do you understand my point here?
Let me touch on the other point. You seem to be concerned with IID within the data. What I am trying to do is to put in your mind two set of questions:
1. Question 1: assuming my model (not the data), what is the probability to pick a top of a day.
2. Question 2: is IID valid or not valid in the data. That is the question you seem to be working on.
Question 1 should be very useful. Why? Because if I can answer it, then I will know the probability to pick the top of a day using a given time increment.
If I then find that I am picking tops of a day way better than the slim theoretical probability, then what do you think we should conclude?
For instance suppose you have a guy who wins lottery every week. Do not you think that the lottery and others will pose the question about whether it is luck, skill or something else?
My job is to be that sort of guy in the area of trading.
That is what methods of type 2 should be doing: generating a subset of data in a way that is significantly better than the theoretical probability, or why not close to "certainty?"
Think of the (t-1) to (t) transition as a white noise. Let us assume the variance is known (mean is zero). The question then becomes, what is the probability of picking the top or the bottom of a day?
The other point you raise is related to X(390) (X(390) is the price at the close not the change of price from the previous minute). The distance from X(t) to x(1) should grow when t grows. It is like a reverse funnel or a tree if you were to picture it, but the average distance between open and close should be ZERO.
Do you see things a little different now?
First there is the question of profitable system. You can do it in two ways:
1. The QQQQ are showing auto-correlation, and you exploit it. My guess ia that a lot of people do that. So there is competition.
2. You have a rule to select a subset of data from the overall data, which shows some advantage.
You should note now that method 2 may work even if the original data is IID, but method 1 would work only if the IID is violated.
Which one do you think you should prefer? I would take 2. anytime, for many reasons. One was the competition, and two I am not concerned about an assumption (IID) on underlying data that might be wrong. Do you understand my point here?
Let me touch on the other point. You seem to be concerned with IID within the data. What I am trying to do is to put in your mind two set of questions:
1. Question 1: assuming my model (not the data), what is the probability to pick a top of a day.
2. Question 2: is IID valid or not valid in the data. That is the question you seem to be working on.
Question 1 should be very useful. Why? Because if I can answer it, then I will know the probability to pick the top of a day using a given time increment.
If I then find that I am picking tops of a day way better than the slim theoretical probability, then what do you think we should conclude?
For instance suppose you have a guy who wins lottery every week. Do not you think that the lottery and others will pose the question about whether it is luck, skill or something else?
My job is to be that sort of guy in the area of trading.
That is what methods of type 2 should be doing: generating a subset of data in a way that is significantly better than the theoretical probability, or why not close to "certainty?"
Think of the (t-1) to (t) transition as a white noise. Let us assume the variance is known (mean is zero). The question then becomes, what is the probability of picking the top or the bottom of a day?
The other point you raise is related to X(390) (X(390) is the price at the close not the change of price from the previous minute). The distance from X(t) to x(1) should grow when t grows. It is like a reverse funnel or a tree if you were to picture it, but the average distance between open and close should be ZERO.
Do you see things a little different now?
I think the most important probability is, of course, the final chance of success with a model. If a model has four trials, and is accurate with three successes, then there is a 95% chance that the correct proportion of successes to trials is between 28.36% and 94.73%, though 75% is most apparent at the moment.
But I'm sure what you're asking for is the probability of selecting each point correctly, given no model, given some assumptions, and finally, given your model. Given no assumptions should approach infinity, according to the number of dollar values the answer may assume (from zero to infinity).
Given some assumptions, such as iid, normality, etc., well, we could of course get a variety of probability estimates.
Given the model is best observed with predicted vs. actual, of course. Even though I've gotten 17 out of 25 binary predictions correct, my sample is small enough that I should only be confident that the accuracy of the method ranges from 48.21% to 82.79%, currently indicating 68%. I have much more sampling to do. And of course, I'll want to try different groups of stocks, different months of the year, etc., and track everything I can.
How long have you been using that model?
By the way, I'm curious what you think, but I'm thinking that indexes are too rich for my blood at the moment, and that increasing the number of shares is the best way to keep costs down regarding transaction fees, so it looks like low-dollar stocks with high volatility offer the best chances. For example, let's say there's a $10 brokerage fee, and you invest 100 shares at $5 each, then the price needs to climb to $5.20 to break even, not considering taxes. But if you held everything constant but instead invested 200 shares, now you only need the price to climb to $5.10 to break even.
And the reason I say high volatility, counter-intuitive as it may be, is because if it is reasonably predictable that it will increase, then the wider (hopefully positive) variance allows larger reasonable gain potential.
By the way, it looks like I may end up going with Ameritrade to start, but I'll be testing for a while first. Ameritrade has trade triggers for the software, so I can run a model in the morning and tell it to buy at a certain price while I'm at work.
If and when I move on to intraday trading, I'm looking at TradeStation. But I'll probably shift around on a lot of things, since I'm a newbie.
But I'm sure what you're asking for is the probability of selecting each point correctly, given no model, given some assumptions, and finally, given your model. Given no assumptions should approach infinity, according to the number of dollar values the answer may assume (from zero to infinity).
Given some assumptions, such as iid, normality, etc., well, we could of course get a variety of probability estimates.
Given the model is best observed with predicted vs. actual, of course. Even though I've gotten 17 out of 25 binary predictions correct, my sample is small enough that I should only be confident that the accuracy of the method ranges from 48.21% to 82.79%, currently indicating 68%. I have much more sampling to do. And of course, I'll want to try different groups of stocks, different months of the year, etc., and track everything I can.
How long have you been using that model?
By the way, I'm curious what you think, but I'm thinking that indexes are too rich for my blood at the moment, and that increasing the number of shares is the best way to keep costs down regarding transaction fees, so it looks like low-dollar stocks with high volatility offer the best chances. For example, let's say there's a $10 brokerage fee, and you invest 100 shares at $5 each, then the price needs to climb to $5.20 to break even, not considering taxes. But if you held everything constant but instead invested 200 shares, now you only need the price to climb to $5.10 to break even.
And the reason I say high volatility, counter-intuitive as it may be, is because if it is reasonably predictable that it will increase, then the wider (hopefully positive) variance allows larger reasonable gain potential.
By the way, it looks like I may end up going with Ameritrade to start, but I'll be testing for a while first. Ameritrade has trade triggers for the software, so I can run a model in the morning and tell it to buy at a certain price while I'm at work.
If and when I move on to intraday trading, I'm looking at TradeStation. But I'll probably shift around on a lot of things, since I'm a newbie.
It's tedious, but if you want to analyze tax you have to use the right terminology and understand the calculation mechanism.
The $10 brokerage fee is added to the tax basis of your asset.
The tax basis is the consideration in the acquisition of an asset.