You forgot the other part of that..."and give to God, what is Gods"
As far as Jesus was concerned, everything belongs to God, so how can you give something to Caesar when nothing belongs to him?
Also Matthew 17:24 After Jesus and his disciples arrived in Capernaum, the collectors of the...