Quote from Ghost of Cutten:
Don't forget the non-negligible chance of waking up to see the stock 50% against you (make that 100%+ for shorts), due to some overnight news. More risk means smaller size you can deploy, which means less returns. Every 1% extra you make from elite stockpicking skills is wiped out by the fact that a lazy indexer can put on 3-4 times the size per unit risk - and using futures he ties up less margin also, allowing him to overlay other unrelated trades, and avoid the risk of capital controls, broker bankruptcy, and other problems.
I'm your huckleberry...
I would say that "elite stockpicking skills" are worth a hell of a lot more than you give them credit for.
If you are that worried about blowup risk, you can still use stock selection to compile a basket of names that offers solid diversification and reduces exposure, while having far superior performance characteristics relative to an entire index.
Not only that, but there are plenty of instances where a sector or an industry leaves the broad market in the dust.
Oil refiners as a group, for example, more than doubled in price in a roughly six month period between Q410 and Q111.
A flip side of this would be solar stocks, which fell 50% as a group within a 2011 six-month period.
Lots of examples from the past. Oftentimes a left-for-dead industry will queue up for a double, again while the broad index gives 10% return at most. Steel stocks a few years back. Chicken producers after a health scare. Shippers (on the short side). On and on.
In all cases, the fundamental drivers of the move, the thesis behind it, and confirming chart action would have all been eminently accessible.
This type of thing happens a lot. There is often a lot of money to be made, long or short, on an industry or group level when the broad market isn't doing squat. You get exposure to those types of moves through a reasonable basket of industry-related names.
And while ETFs at least theoretically offer industry exposure (SLX, TAN etc), a lot of times the ETF's composition is absolutely shitty, or mystery meat, or messed up by lop-sided weightings that don't make any sense. Beef is beef, but there's a difference between hamburger helper and filet mignon.
And this doesn't even touch the value argument, e.g. buying cash boxes or Washington Post type names, where the disconnect between intrinsic valuation and market price only gets more extreme the more the stock drops, or alternatively shorting insanely valued growth names AFTER the sentiment and earnings engine has showed clear signs of breakdown. Stocks don't just drop 50% randomly, and shorts don't just double randomly, any more than a girlfriend turns randomly psycho. There are underlying accessible factors, warning signs there, and with reasonable diversification you can survive the adverse gaps and actually benefit from the ones that go in your favor. (Which will tend to happen more often, actually, if you are generally positioned correctly.)
Long story short, once again it just ain't that simple. While using an index can be a valid strategy, to say that it is superior to an excellent stock picking strategy makes no sense, because the excellent stock picker can still diversify, minimize individual exposure, and use indexes to hedge when he chooses, while gaining a superior exposure profile that no amount of "safe leverage" can match. You go ahead and leverage up total S&P exposure to turn a +5% return into +15% -- I'll stick with buying industry baskets that go up 50% and shorting ones that go down 50%.
All said, being able to go both ways is like being a kick-boxer, instead of just a regular boxer. You don't lose by having extra alpha generation capabilities on top of the other guy.
Nor do I understand the attitude that "stockpicking is so hard" etcetera. Sure, it ain't easy, but getting the macro right is??? Everything is deeply competitive about this game. It's one thing to admit a personal weakness in one's skill set (e.g. inability to pick stocks or industries), and another to irrationally say "nobody can really make money in that area" or "that skill set isn't so valuable as my ability to just add some more lazy leverage on an index."