Not just society. People with differing gender identities think differently, while hormones also influence how you think. Testosterone makes all people more aggressive and people with more masc identities think differently to those with less masc identities. It's a spectrum with multiple dimensions that all contribute to one's internal range of comfortable identities. Who fucking knows the specifics, but that's probably the general picture.
Gender identity kind of needs to be based in biology rather than culture, because otherwise conversion therapy would work and social contagion theories would hold water. Queerness is based in biology, but scientific labels of normality are also socially constructed tools of power that can be used for abuse. Queer theory and evolution are not incompatible. Evolutionary science doesn't inherently invalidate nonbinary gender identities, and doesn't validate most aspects of binary sex. Only sex cells evolved to be inherently binary, everything else varies based on incentives rather than rules.
Talking about biology and evolution in regards to gender feels so icky to us because it has been used as a tool for oppression so much in the past. It's like race science used to justify oppression of non white people. They use the authority of science to say we don't exist, refuse to accurately document our experiences, apply cisheteronormative assumptions even when they don't make sense, and claim that we're all freaks of nature instead of nature working as intended.
Just like how racial minorities have taken evolutionary theory from white supremacists as an authoritative tool of oppression, we need to take it back from transphobes and transmeds that want to invalidate and abuse. They assume that we should model our lives after nature, but we really shouldn't. Nature thrives on death and suffering. It isn't enviable; we just need to figure out how to live in it.