Your argument was used by every single colonizer. Your argument is typical.
Look at Vietnam, Algeria, Jewish Warsaw Ghetto and Occupied Europe during the Nazi invasion. In all examples, the occupier had people like you coming up with identical arguments.