Your problem seems obvious.
Even though you are not consciously aware of it, there might be a self image conflict between your conscious and subconscious mind that is affecting your judgment of others. The conflict can be a very subtle or insignificant to your conscious, but it's consequences...