My backtested system consistently beats the S&P500

"My question is, how do I verify that this backtest is legitimate? Are there statistical tests that I can run to ensure that the system's success would also apply to today's markets?"

AND

"And lastly, if I am confident in my system..."


HUGE GLARING CONTRADCTION.
 
Quote from rcanfiel:

Everyone's systems backtests better than the S&P. Then they crash in walk-forward real world testing. So what else is new?

There's nothing wrong with using S&P as a benchmark, based on what he mentions.

Institutionally... he needs to be outperforming the S&P every year. 60+% DD is way too high. Your datasets are way too small. The std. dev/volatility of the performance is way too high. Finally, it'll get killed with trading costs...

Anyways... it's not a marketable system.

Good luck to the OP.

PS. The problem isn't about whether the system is robust or tradable. It's just not a good system to start off with... Based on these replies on "viability", I wonder what kind of systems the people replying are developing...

:confused: :confused: :confused:

:(
 
"My question is, how do I verify that this backtest is legitimate? Are there statistical tests that I can run to ensure that the system's success would also apply to today's markets?"


you're looking for stochastic modeling such as the monte carlo method, that encompasses a technique of statistical sampling employed to approximate solutions to quantitative problems.
 
Quote from TSGannGalt:

There's nothing wrong with using S&P as a benchmark, based on what he mentions.

That had nothing to do with the reply to the post...
 
Quote from GTG:

You have to be careful when you use fundamental data for a backtest. A lot of the data gets revised after the fact, and the reported values in the database now, were unknowable at the time.

Well I'm going off reported financial information through Bloomberg. I guess the data could be flawed



Quote from h hubbins:

i'm no quant guru but his results might also improve if he dollar cost averages throughout the year right?

if the system is value based then test it against a value based large cap portfolio like the ishares s&p value etf.

well the way I do it is I assume all my stocks are equally valued. I don't know what the technical term for that is, but i choose 100 stocks that fit my criteria based all stocks' previous year's Q4 financial reports. Then, I see how all those 100 stocks performed over the year, and then find the mean of the returns

That's why I don't think slippage would be a huge deal. Tomorrow, I'll run through the backtest but I'll get the correct bid/ask prices for all the trades, that should take care of slippage. then I'll take commission into account. However, I was doing some rough calculations today and I don't think commissions would do much harm to my overall returns

Quote from phil1424:

stk trader how much are you asking for your system ? i take it still beating the pants off this lame market ? jake

Haven't thought about it yet. perhaps if there were some bids? :p (don't take this seriously please)


Quote from TSGannGalt:

There's nothing wrong with using S&P as a benchmark, based on what he mentions.

Institutionally... he needs to be outperforming the S&P every year. 60+% DD is way too high. Your datasets are way too small. The std. dev/volatility of the performance is way too high. Finally, it'll get killed with trading costs...

Anyways... it's not a marketable system.

Good luck to the OP.

PS. The problem isn't about whether the system is robust or tradable. It's just not a good system to start off with... Based on these replies on "viability", I wonder what kind of systems the people replying are developing...

:confused: :confused: :confused:

:(

well, I think you are a little confused, because I don't see how you could deduct that this system isn't tradable and that it's "not good". The stdev of the annual returns is only slightly higher than the stdev of the S&P over the same years. I beat the s&p virtually every year. The DD isn't ever 60%, don't know how you came up with that :confused:

Quote from traderdragon2:

"However, I've always been under the impression that slippage doesn't make a huge deal when backtesting equities."


Wow, just wow. :eek:

Like I said, I'll run through the numbers tomorrow taking slippage and commission into account. I still don't think it would be a huge deal, since I'm placing end of day trades, holding my trades throughout the course of 1 year (so the number of trades isn't very high)
 
Quote from StockTrader1985:

well the way I do it is I assume all my stocks are equally valued. I don't know what the technical term for that is, but i choose 100 stocks that fit my criteria based all stocks' previous year's Q4 financial reports. Then, I see how all those 100 stocks performed over the year, and then find the mean of the returns

Sure hope you do not use Q4 information to decide which stocks to buy at the beginning of the year. That earnings information is generally not released until some weeks/months later.
 
Large cap is good.

This should get you started on alpha.

http://www.investopedia.com/terms/a/alpha.asp

If you can show statistically risk adjusted outperformance (edit: in a diversified portfolio) over a reasonable period you should get someones attention.

Here's some standard portfolio statistics for a mutual fund -

http://finance.yahoo.com/q/rk?s=esmax

Obviously very good performance.

Good luck.

Quote from StockTrader1985:

1) Large cap, in fact one requirement is that the company's market cap must be at least a billion. I ran the system over small cap, and although I beat the s&p500, the results weren't as great as the large-cap test. However, since the two tests are mutually exclusive, I think that should add legitimacy to my system. What do you guys think?

2) how do you calculate return on risk adjusted basis? are you talking about drawdown, or standard deviation?

also, are there any objective tests I can run based on the stock's balance sheet and/or income statement? like I said, I'm looking for something objective

EDIT: I will also be running the tests on foreign equities in the near future. I will keep you all updated
 
Quote from StockTrader1985:

well, I think you are a little confused, because I don't see how you could deduct that this system isn't tradable and that it's "not good". The stdev of the annual returns is only slightly higher than the stdev of the S&P over the same years. I beat the s&p virtually every year. The DD isn't ever 60%, don't know how you came up with that :confused:

My bad. The DD of 60% was incorrect. 1998 and 2002 was underperforming and took 2002 and 2003 incorrectly... though...

Shouldn't the Std. Dev. be lower???? From my understanding...

Lower the Std. Dev., lower the volatility...

Am I missing something?

PS. I still think your model is useless. Please correct me.
 
Back
Top