Let me reframe your question to be "why can't i reliably execute at the closing print?" Understand that the issue with BoWo goes beyond that, but let's deal with your question in simple form.
Basically, what you see with a lot of stocks is that they trade with normal liquidity up to 16:00, then at the closing bell the liquidity vanishes and the spreads widen. Let me just make up an example with a liquid stock trading 100.00 / 100.01 at 15:59:50. Let's say I know I can hit that bid for 10K shares and maybe drop it .02 at that time. Fine... but at 16:00:01 the spread will now be 99.10 / 101.25 and if I hit that bid for 2K shares it will drop and the next bid will be 97.50. Maybe there are more hidden bids there, but I can't bet on it. (Exact opposite example applies to buying.)
Then, particularly with NYSE listed stocks something curious happens. The specialist will settle his closing orders and do a closing print at, say 100.18 for half the day's volume.
Your data feed will quite correctly report the settlement price as 100.18, but you see there's no way you could have reliably bought there. Some days? of course. But others... no way.
The problem is if you do a backtest on those settlement prices you are using prices that you cannot possibly rely on getting. So why can't you enter market on close orders? Well, short answer is they have to be in well before the close and other restrictions apply. If you are trading a pairs system, you often don't know you need to execute until a few minutes or seconds before the close, so MOC is most definitely not an option.
In addition, a sizable MOC order will move that closing print... sometimes a lot (and that's never something you want (guess how I learned that one...)) Honestly that impact is dwarfed by the fact that you just discovered you needed to enter that MOC 20 minutes ago and now you gotta try to get as close to the 16:00 print as you can and you have no idea where they're gonna settle the damn thing lol.
And... maybe some stocks settle at 16:00:12 and another at 16:01:15. If you are looking at a ratio of settlements you are looking at prices that didn't happen at the same time... may not seem like a big deal but it is a HUGE deal.
Use intraday prices as someone else suggested... that's the best answer. If BoWo B-job were to rerun his QID QLD using 16:00 or 15:59 prints he would see a significant degradation in his backtested performance... but it would be a step closer to reality.
I think that was semi-coherent... ask more if I didn't answer well.
Quote from bs2167:
Weak. There was a nice combination of learning and entertainment there.
I had what I thought was a legitimate question on data in there...I hope you don't mind if I repost it in here in case someone wants to comment:
Wow - really surprised at the magnitude of the differences.
--------------------------------------------------------------------------------
Quote from talontrading:
Attached find a spreadsheet that covers the full history of the QID. It includes settlement prices and 1600 prices. BoWo's backtest was done to settlement prices, but you can see that at times this price is VERY different from the price you could have executed.
--------------------------------------------------------------------------------
For the unitiated like myself, here is some info on how the 16:00 vs. settlement discrepancy occurs:
http://www.sec.gov/answers/closepr.htm
In an earlier instance, Mike was nice enough to alert me to the dangers of using a daily data source. And to his point, I've yet to find an instance where a daily data provider does not use the settlement price as the daily close. Obviously this has major implications for constructing a realistic database.
However, there is still one thing I am unclear on regarding the ability to reliably transact at the settlement price. According to the paper below, it appears that market-on-close or transactable limit-on-close orders have/will always trade at the CTA's reported settlement price. Of course this ignores the possibility that your order would have moved the market. Is the potential for moving the market the only factor that makes using the settlement price unreliable - or is there something else I am missing?
http://www.sec.gov/rules/sro/nyse/2009/34-59345.pdf