I'm not going to pretend to be an expert, I just happen to know someone who was a developer for a few years in a true HFT firm and have talked about it.
As far as the lightweight linux build, I can think of a number of key things that are almost certainly changed from a standard linux system to reduce latency. I'd be surprised if any actually ran a standard linux kernel. In any case, it's not so much the hardware required that's the problem, it's the monthly expenditures (microwave access, exchange memberships, colocation).
Then there's the ongoing expenses that aren't fixed (clearing for example). To get good clearing agreements, you have to commit to substantial volume and have rather large capital (they don't just care about volume). Also because you're not going through a broker, there is not an API to use to get quote data and enter orders -- a lot of R&D in the work to deal with the messaging formats used by the exchanges (both for quote data and for orders) and these protocols do get updated and each exchange is different, so R&D has to be ongoing to keep up with these.
All of this is completely independent from the algorithms being used to determine when and what orders should be entered, which also need to be continually updated to keep with changing markets. It's really not a "go it alone" sort of thing. I really don't know that even an incredibly well funded individual could keep up with everything that needs to go in to running a true HFT operation -- they'd have to have a handful of people to assist in just keeping everything running as the exchanges themselves change.
As a sort of hobby project, I'm going to see how fast I can calculate the S&P 500 index from scratch (meaning no pre-computed values) from the time the last byte of the last quote comes in to the beginning of the first byte of the order is sent. I'm shooting for 2µs.