It’s not especially harmful to white people to be denigrated for who they are…?
I already said it wasn't harmful to them, yes. But also, "White" is not an identity, as I have already said. It is a fucked up political category that serves to say, "these people are not any of the racialised categories". That's what I said at the start - it is only an exclusionary category. It does not identify any particular set of cultural or ethnic groups. There is no such thing as "white culture". There is Norwegian culture, British culture, Texan culture, etc, but no white culture. Black culture, on the other hand, very much does exist, because black people as a group have been forced to construct a culture in response to their racist oppression.
If society as a whole wasn’t horrifically anti-Black in its racism, would it then be okay to use anti-Black slurs? Because it wouldn’t be ‘especially harmful’?
Blackness as a concept was invented by racists for racist purposes. Anti-black slurs wouldn't exist without that, so you couldn't use them.