I think part is a defensive reaction on part of these men; if you are attracted to women and you don't quite know what they will do and are dependent on their judgment, that can feel very dangerous and unsettling. A loss of control. So it's easier to just blame and defend your ego, to counter the dissonance.
Remembering that women are people is something that would likely make it easier