They're probably staring because you're obviously trans.

That's how you know it's a mental thing, what they see in the mirror is not what the rest of the world sees.

What gets me is the ones who have say an obvious male hairline or jawline and expect you not to notice it. Women don't look like that, dude, how do you expect me to take you any way serious if you make no effort to fix that or disguise it?