My guess is that the men who don’t think they’d be bothered by cat-calling are imagining a scenario where there are lots of other people around and the risk of being physically attacked is very low. (Something like the stereotypical image of construction workers whistling at a woman walking by them on a busy sidewalk.) Being on a nearly-empty subway platform with the only other guy nearby accosting you is a genuinely risky situation even without pretending that you’re a woman.

One time I was walking on the sidewalk when a car with several young women drove by and one of them leaned out the window and yelled something at me. I didn’t hear what she said but I like to think that it was positive and it made my day, but the caveat is that I did not feel like I was in any physical danger at all from them.