backtesting vs real trades

d08 · Jun 21, 2015

DTorSwing said:
ok, but if I don't pre select stocks, how would I even run backtest on what instruments?

so 18months less, what period would you backtest, like 3 yrs or 5yrs? i doubt i have intraday data for that long

I'd use 5 years minimum and across all equities. You can filter for liquidity (price x volume for example).

Turveyd · Jun 21, 2015

Either 1 or 2 year periods, use 3 of them last 3 or 6 years.

Setup your system for the middle period, when profitable check its profitable once on the other 2 being before and most recent.

Its all curve fitting, just got to hope the curve fit is still valid and suck it and see.

goodgoing · Jun 21, 2015

IAS_LLC said:
The only thing I have stated is that you need X number of trades to claim your backtest is statistically significant....I've made no claims of profitability or anything further than that....Excuse me for attempting to help someone who had a question.

There is no way to claim backtest results are statistically significant. Nobody said you made claims of profitability but you made basic statistics mistakes. Your test was about testing whether a coin is fair (or unfair). This is not a proper tests as I already demonstrated to you. Win rate is irrelevant and your test is about that only, i.e., it is the wrong test. You have been misguiding yourself all along and you also tried to pass these misunderstanding on others. It's better to ask people who know these subjects than to present yourself as an expert. Some may believe you and end up paying the price for that. You have to take a course in statistics and hypothesis testing. Google these subjects and you can find many online courses. All the tests you talked about are not applicable to trading systems.

IAS_LLC · Jun 21, 2015

goodgoing said:
There is no way to claim backtest results are statistically significant. Nobody said you made claims of profitability but you made basic statistics mistakes. Your test was about testing whether a coin is fair (or unfair). This is not a proper tests as I already demonstrated to you. Win rate is irrelevant and your test is about that only, i.e., it is the wrong test. You have been misguiding yourself all along and you also tried to pass these misunderstanding on others. It's better to ask people who know these subjects than to present yourself as an expert. Some may believe you and end up paying the price for that. You have to take a course in statistics and hypothesis testing. Google these subjects and you can find many online courses. All the tests you talked about are not applicable to trading systems.

The test was to determine if the results are produced by random chance. It's completely relevant.

syswizard · Jun 22, 2015

It's amazing to me to see anyone attempt algo trading without a really solid background in statistics.

IAS_LLC · Jun 22, 2015

syswizard said:
It's amazing to me to see anyone attempt algo trading without a really solid background in statistics.

So the use of simple statistical tests for a simple problem implies a lack of solid statistical foundation? Interesting the conclusions that people can come to..

syswizard · Jun 22, 2015

IAS_LLC said:
So the use of simple statistical tests for a simple problem implies a lack of solid statistical foundation? Interesting the conclusions that people can come to..

So this response pretty much indicates you ARE a wise-ass (my impressions all along).
If these are such simple problems, why isn't everyone making money algo trading ? I mean it would be like taking candy from a baby, right ?

IAS_LLC · Jun 22, 2015

syswizard said:
So this response pretty much indicates you ARE a wise-ass (my impressions all along).
If these are such simple problems, why isn't everyone making money algo trading ? I mean it would be like taking candy from a baby, right ?

Noone said anything about algo trading being simple... The question was how much data is needed to produce a reliable backtest...and it appears I'm the only one who used any math (simple or otherwise) to produce a number. It's astonishing that my response receives more criticism than those that are along the lines of "oh, about 3 years should do the trick"

syswizard · Jun 22, 2015

I know, this stuff is just so......I see now why Simons hired PhD's.

8. Model Inference and Averaging
Note that we don’t need to know the explicit form of the conditional
densities, but just need to be able to sample from them. After the procedure
reaches stationarity, the marginal density of any subset of the variables
can be approximated by a density estimate applied to the sample values.
However if the explicit form of the conditional density Pr(Uk, |U`, ` 6= k)
is available, a better estimate of say the marginal density of Uk can be
obtained from (Exercise 8.3):
cPrUk (u) = 1 (M − m + 1) MX t=m Pr(u|U(t) ` , ` 6= k).
Here we have averaged over the last M −m+ 1 members of the sequence,
to allow for an initial “burn-in” period before stationarity is reached.
Now getting back to Bayesian inference, our goal is to draw a sample from
the joint posterior of the parameters given the data Z. Gibbs sampling will
be helpful if it is easy to sample from the conditional distribution of each
parameter given the other parameters and Z. An example—the Gaussian
mixture problem—is detailed next.

IAS_LLC · Jun 22, 2015

syswizard said:
I know, this stuff is just so......I see now why Simons hired PhD's.

I'm not sure why this is relevant? We are talking sample size determination, not Bayesian Inference or statistical model building.