I know a fellow who used the n-word about Obama twice in an hour, then proceeded to complain that people called him a racist.
I don't think not voting for Obama makes you a racist. After the last eight years of Republicans, cuckoo-crazy perhaps, but not by default racist.