When you defend yourself from a random accusation like racism or sexism, you tell the person that you are insecure about yourself. Non-racists are not afraid of being called racist because they know that they are not and that the other side is projecting.