It's not that people's votes possibly don't matter because of their zip code. It's that peoples votes don't matter because of their zip code. There's a lot of bullshit going on with voter suppression, shittily written constitutions, and how representatives are calculated, but the electoral college itself is what decides the election, not the popular vote. And with the electoral college, your location is determinate of how much your vote matters in the election.
The thing is, that does refute the thesis.
If the thesis is "You withheld the one vote that would have prevented fascism", was true, we would either be using a popular vote system for president, which we literally don't, or the person would have to be in a singular swing state that decides the election. This means that the thesis is only true for those who live in swing states, due to the presence of the electoral college.
I googled the current swing states, and I got Georgia, North Carolina, Pennsylvania, Michigan, Wisconsin, Nevada, and New Mexico. The current population of the US is somewhere above 330 million, and the population of the swing states listed is, according to my napkin math, 55 million. I'm assuming the population of voters proportionally is roughly equivalent state to state.
That being said, this means there's roughly 1/6th of Americans whose votes would be representative of this meme.
To call the experience of 5/6ths of Americans tangential is a little silly.