Mmm this is mostly, but not completely, true. I've had very concrete experiences when I was firm in my desire to only have a friendship with a woman, and it did not go over well. I ran into one woman at a nightclub that I'd rejected a relationship with (because I knew there would be issues to it working out), but nevertheless someone who's company I genuinely enjoyed as a friend, and she treated me like a pariah. That expression "Hell hath no fury like a woman scorned" is rooted in some degree of reality. I remember thinking, "I didn't take it further because I knew it wouldn't work, and yet I'm still the bad guy." Damned if I do, damned if I don’t.

I believe part of it is when a person of any gender is attracted to someone and they are rejected, rejection is not a pleasant emotion and it takes a lot of emotional intelligence to handle that rejection.

I also believe that the toxic masculinity aspect that you mention is also true- men are trained either via society or via genetics to pursue sexually. If I'm honest, frankly sometimes, we think with our phallus (I have no idea how to correctly pluralize that noun).