When every millisecond counts the OS will be the least of your concerns. I would rather trade on a 5 year old computer plugged directly into a Nasdaq server than on a state of the art computer over dial up. The key is going to be in the latency of your internet connection. Don't confuse this with bandwidth though. (See bottom "Note" for more details). The faster you can get the data, and the faster you can get your orders to move to the brokerage server, the better off you will be. If at all possible try to get a full fiber optic internet connection directly to your modem. As mentioned you do not need a crazy amount of bandwidth so you don't have to pay more for a bigger package, just get 6-10MB/s DL speed and you will be fine....unless of course you need hundreds of streaming charts at the same time.
As far as OS goes both Linux and Windows can be stripped down to the point that any difference would be negligible. You would want to turn off and disable any background services and programs that are not required for trading. In fact if you know what you are doing in windows you can even disable the main Windows Explorer (Desktop, Icons, Start Bar, etc.) and actually boot windows so that your trading platform IS the OS....well at least the front end of it. Although to do this you would need a second machine for anything other than trading.
Note: Bandwidth is the quantity of data that can be downloaded per second) which is not really a major concern in today's broadband world, at least as far as trading goes. Latency on the other hand is how fast a small amount of data actually travels from one computer to another. Think of a network connection as a water hose. Bandwidth represents the diameter of the hose and the bigger the hose the more water you can push through it at a given time. Latency would be the speed with which the water moves through the hose. A smaller hose with more pressure can actually move a small amount of water from one place to another faster than a big hose that is only on at a trickle.
