How stupid are you, I mean really? If you take out all the non physical supply, then only physical supply stays. Hence oil would be $5,000 without speculators.
This reasoning is exactly the same as yours yet mine "sounds stupid" while yours "sounds smart" because you and everybody who listens...