Try some broker and vendor systems to see if they satisfy your latency needs. Ask them if you are using a turnkey solution? co-lo? server side trigger of orders? etc.
i.e. TT, IB, Athena (although I think no many retail offer Athena), ORC, Pats, etc.
These guys all have server side solution. It all depends your broker on if they co-lo the solution, if they have a ton of users on each deployed solution or not, etc.