That's been a continual strategy to try to deter and block RCV. They argued that in front of the courts in Maine when the state moved to RCV.
In the end, I feel there's one big defense: no matter where my vote ends up, I only get one for the last candidate standing that I voted for.
The other voting systems where you rate candidates on a scale, it's a bit muddier as to what a "vote" is. A vote should be your voice that's the same as anyone else's in the electorate. As long as all humans get the same voice, it should be able to take any form.