Preventing trans women from competing in women's sports is a ban on trans women in sports. Trans women do not have testosterone levels anywhere near cis male levels. And none of us are going to degrade ourselves by being categorized as men.
If you would make trans women compete against men then you're saying trans women aren't women. It's as simple as that.