If the "contractor" is an employee (i.e. he gets W2 from you), you are responsible for one side of SS. But if he really gets W2 from you, he is not a "contractor", but an employee.
A true contractor will get a 1099 from you, and he is responsible to pay both sides of SS.