I actually try to use singular they for anyone who doesn't specifically request otherwise because of this.
In fact my concept for a conlang has this,
1st person (I, Me), 2nd Person (You, Y'all), third person (They, Them) and "4th person" or Object (It, Those)
Part of me wants to make some of the related particles also act as evidentiary cases for the past tense, Participant (I did), Witness (I saw), Testimony (They told me they did/saw), and Hearsay (I've heard).