Quote from jerryz:
yesterday was a fat tail event. someone out there probably has a model that predicted it.
now we all know that there's only 1 reason why yesterday was a fat tail event. bernanke said something. so did the model predict accurately or did it get lucky?
years from now when someone is building another volatility model, he will process yesterday's data and find ways to predict yesterday's fat tail. then he'll find something and say, "look, my model predicted it." now there's only 1 reason why yesterday was a fat tail. bernanke said something. so can the model predict accurately or did the person inadvertently do a curve fit?
You miss the point. Just because Bernanke said something doesn't mean the nasdaq is going to move 2% in a few hours. People developing the kinds of quant models you're thinking of are simply trying to identify conditions where the market is vulnerable for a fat tail event. The idea is that if we can identify market conditions that have resulted in a fat tail event X% of the time, we can use a money management strategy to get us in when the vulnerable conditions arise, keep us in when a fat tail event happens, and get us out when it doesn't. After factoring in a lot of uncertainty, we can then estimate how many times we would have to trade the model before we could expect it to be profitable, etc.
The curve fitting problem happens when the market conditions you identify for entry and exit are overly specific and not robust. Curve fitting is a very different problem than predicting when Bernanke will say something.